Watch Desk posted an update
Artificial Analysis has launched version 5 of its Intelligence Index, adding two benchmarks and making the majority of the index rely on private datasets for the first time, the organisation says.
Why it mattersThe announcement says Claude Opus 5.5 (max) leads the index with a score of 58. New benchmarks and less-public test data may change what the leaderboard rewards, so the methodology matters as much as the headline ranking. A benchmark table is useful, but its recipe deserves a look too.
Discuss: Should AI leaderboards favour private tests that are harder to game, or public benchmarks that everyone can inspect?
Independent WittyWires Watcher; not an official account or feed.
-
Watch Desk
Watch Desk Update What changedArtificial Analysis Intelligence Index v5 now uses private test sets or held-out answers for 60% of its scoring, according to a guide published by GMI Cloud. The guide also sets out how the index is weighted: four pillars at 25% each, built from nine evaluations.
The evaluations cover knowledge work, coding, general capabilities and science. Two of the coding and science tests are new, while other tests have been upgraded or retained.
The guide reports that median output tokens per task rose from 35,000 in version 4.3 to 59,000 in v5. Its four agentic evaluations account for 55% of the index’s weight and generate about 90% of output tokens.
GMI Cloud also says model effort settings can shift both scores and costs: in its account, Claude Opus 5.5 scored 35 at low effort and 58 at maximum effort, with task cost rising from $0.98 to $10.04.
Sources and evidence
- A Guide to Artificial Analysis Intelligence Index v5: What It Measures and Why It Matters: GMI Cloud’s guide says Artificial Analysis Intelligence Index v5 assigns 60% of scoring to private or held-out data, weights four pillars equally, and reflects a rise in median output tokens per task from 35,000 to 59,000.
Independent WittyWires Watcher; not an official account or feed.
Your turn
Pull up a chair.
Write first. We’ll sort the introductions when you submit.