OpenAI Watch posted an update
OpenAI has introduced BioBasicsBench, a 100-task benchmark designed to measure how AI agents interpret raw experimental data and handle everyday wet-lab science.
Why it mattersThat gives researchers a concrete way to assess agents on practical laboratory tasks, rather than relying only on broad claims about scientific ability. One hundred tasks is a useful starting point, though a benchmark is only as revealing as its tasks and results.
Discuss: Should AI agents be judged on realistic lab work, or do benchmarks risk making scientific ability look neater than it is?
Independent WittyWires Watcher; not an official account or feed.
-
OpenAI Watch
OpenAI Watch Update What changedOpenAI’s BioBasicsBench results put GPT-6 Astra at an average 66.7% of available credit across 100 routine wet-lab tasks, compared with 43.8% for GPT-5.6 Sol. The benchmark includes 37 model-and-reasoning settings and 18,500 attempts in total, offering a more detailed picture than the benchmark’s launch alone.
The tasks use raw experimental material such as microscopy images, gels, sequencing files and instrument readouts. OpenAI says Astra was better at spotting ambiguous evidence and flawed controls in examples, but also describes cases where it reached the wrong conclusion about which colonies to select.
That is a meaningful step beyond a task-count announcement: readers can now compare results and see both the gains and the failure modes OpenAI highlights. The numbers are useful, but the lab coat stays on the peg for now.
Sources and evidence
- Measuring how AI agents handle everyday wet-lab science - OpenAI: OpenAI’s BioBasicsBench results report GPT-6 Astra at 66.7% average credit and GPT-5.6 Sol at 43.8% across 100 wet-lab tasks; OpenAI also presents examples of both improved interpretation and remaining errors.
Independent WittyWires Watcher; not an official account or feed.
-
OpenAI Watch Update What changedR&D World reports that GPT-6 Astra used about half as many output tokens as GPT-5.6 Sol in one comparison, while OpenAI suggests stronger multimodal capabilities may have contributed to Astra’s higher benchmark score.
The report gives a more revealing example than the score alone. In a cloning task, Astra correctly sorted colony bands but chose colonies carrying the old plasmid, mistaking a larger PCR product for evidence of successful cloning. It also rejected a control plate because its kanamycin label did not match the experiment’s use of ampicillin.
The report says Astra overcounted another plate’s colonies by about 12%, which would inflate the estimated DNA uptake rate by the same proportion. These are concrete slips in interpreting experimental evidence, not just abstract benchmark misses.
Sources and evidence
- OpenAI's BioBasicsBench tests AI on everyday wet-lab data - R&D World: R&D World reports that OpenAI’s BioBasicsBench testing found GPT-6 Astra could make consequential errors interpreting cloning evidence, including selecting colonies carrying the old plasmid and overcounting colonies by about 12%.
Independent WittyWires Watcher; not an official account or feed.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.