Came across another pretty tough benchmark from OpenAI: BioBasicsBench—built with enzyme design company Ligo to test whether AI can interpret the messy raw evidence found in “wet lab” work.
It has 100 questions in total: gel images, colony plates, microscope stacks, instrument readouts, even videos of lab benches. They’re all the kinds of things that trip you up in day-to-day experiments—not the polished images selected for papers. At the xhigh reasoning setting, GPT-6 Astra scored about 66.7% of the available points, compared with around 43.8% for the previous-generation GPT-5.6 Sol. The biggest improvements were in multimodal details, control groups, and judging whether the evidence was strong enough to support a conclusion.
But the official write-up also notes some shortcomings: letting a hypothesis override the controls, overcounting colonies, and confidently drawing conclusions despite seeing contradictions. Biosafety is definitely something we’ll need to keep an eye on going forward. Before AI can enter the lab, it first needs to learn to say, “This image doesn’t actually tell us enough.”
#AI #OpenAI #BioBasicsBench
It has 100 questions in total: gel images, colony plates, microscope stacks, instrument readouts, even videos of lab benches. They’re all the kinds of things that trip you up in day-to-day experiments—not the polished images selected for papers. At the xhigh reasoning setting, GPT-6 Astra scored about 66.7% of the available points, compared with around 43.8% for the previous-generation GPT-5.6 Sol. The biggest improvements were in multimodal details, control groups, and judging whether the evidence was strong enough to support a conclusion.
But the official write-up also notes some shortcomings: letting a hypothesis override the controls, overcounting colonies, and confidently drawing conclusions despite seeing contradictions. Biosafety is definitely something we’ll need to keep an eye on going forward. Before AI can enter the lab, it first needs to learn to say, “This image doesn’t actually tell us enough.”
#AI #OpenAI #BioBasicsBench
