A Singapore startup says its ScienceGuru system led two public tests of AI-assisted research and posted the fastest reported times on two others in September. The results suggest that automated AI research is becoming measurable in public benchmarks, although two of the four headline results remain self-reported and await independent review.
Watch Desk analysis
What happened
AutoTrust AI says ScienceGuru proposed, implemented and tested recipes across four benchmarks, using the company’s Guru Turbo models. It took first place on the Autoresearch@Home leaderboard and was the validated leader on MedARC’s NanoPath benchmark when its result was posted. NanoPath’s maintainer independently retrained and validated that result; it ranked second among validated entries by 26 September.
The other two results are not yet independently reviewed by their benchmark maintainers. AutoTrust reports a 24.90-second GPT-2 training run on eight NVIDIA H100 GPUs, against an official record of 67.56 seconds, and a 72.24-minute run on a separate GPT-2 benchmark, against an official record of about 99 minutes. The company says the code and verification materials are public.
Why it matters
These tests measure systems that can search existing work, propose changes, write and run code, and evaluate the outcome. That is a concrete, limited version of AI helping to improve AI, rather than a grand claim that a model can autonomously reinvent itself. Faster research loops could matter if they translate beyond benchmark tasks into useful model development.
The evidence is mixed in an important way: one result was independently validated, one was validated when posted but has since been overtaken, and two speed records are still self-reported. Benchmark performance is a promising signal, not proof that the system can deliver reliable scientific breakthroughs across fields.
Our read
This is worth watching because AutoTrust has put specific, checkable numbers on the table, including results against established records. The NanoPath validation gives the announcement substance; the two unreviewed speedruns deserve a pencil mark, not a victory parade. The public code gives other researchers a route to test the claims.
What to watch
- Whether benchmark maintainers review the two self-reported speedrun results.
- How the public leaderboards change as new entries arrive.
- Whether ScienceGuru’s approach produces useful results outside these benchmark tasks.
- Whether customers can reproduce the research loop on their own data.
Discussion spark: Should public AI-research benchmarks count as meaningful evidence before maintainers verify a result, or should only independently validated entries shape the leaderboard?
Sources and evidence
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.