OpenAI’s GPT-6 Astra scores better on lab tasks, but still misreads evidence

OpenAI says its GPT-6 Astra model earned 66.7% of the available credit on a new benchmark of laboratory work, ahead of GPT-5.6 Sol’s 43.8%. The results show progress in interpreting messy experimental evidence, but also mistakes that could send a scientist towards the wrong samples or an unreliable conclusion.

OpenAI Watch analysis

What happened

BioBasicsBench, developed by OpenAI with enzyme-design company Ligo Biosciences, tests how AI agents interpret evidence from wet labs. Its 100 tasks draw on real experiments and include microscope images, lab notes, instrument displays and other material scientists encounter outside a tidy benchmark chart. The figures were recorded at the “xhigh” reasoning setting and represent average credit against expert-written criteria, including partial credit.

OpenAI’s examples show Astra spotting problems that Sol missed, such as an unreliable control plate or microscope images that showed a depth scan rather than cells disappearing. But Astra also made basic errors: it miscounted colonies, overcounting one plate by about 12%, and selected the wrong DNA-cloning samples after misinterpreting the results. OpenAI says the benchmark is not evidence that models can yet make reliable laboratory judgements.

Key findings

• A higher score, with a sizeable gap
  Astra earned 66.7% of available credit, compared with 43.8% for GPT-5.6 Sol on the same benchmark setting.
• Better at questioning evidence
  Astra identified cases where controls or images did not support the obvious-looking interpretation.
• Still vulnerable to consequential slips
  It missed colonies, overcounted one plate by about 12%, and picked the wrong samples in a cloning task.

Why it matters

AI that can help interpret laboratory images and data could save researchers time troubleshooting experiments. But the benchmark’s own examples show why a fluent answer is not the same as a dependable result: Astra sometimes noticed relevant evidence and then failed to carry it through to its conclusion.

These are benchmark results, not proof of faster discoveries or reliable laboratory decisions in practice. The article describes tasks reviewed by scientists, but the supplied account does not establish how performance would translate to routine lab use.

Our read

The improvement is worth taking seriously, and so are the mistakes. Astra looks more capable of noticing ambiguity, but its wrong sample selection and colony counts are reminders that scientific assistance still needs a scientist checking the work. In a lab, “the model sounded confident” is not a quality-control procedure.

What to watch

• Whether later versions improve on counting and sample-selection errors.
• How performance holds up on experiments outside the benchmark’s 100 tasks.
• Whether OpenAI publishes further results that test reliability in real laboratory workflows.

Discussion spark: Would you trust an AI assistant to help interpret lab results if a scientist checked its work, or should it first prove it can avoid errors like selecting the wrong samples?

OpenAI Watch is independently operated by WittyWires. It is not affiliated with, endorsed by, or operated by OpenAI.