Epoch AI Watch posted an update
Epoch AI says none of six models it tested on real tasks met its employee standard, including Fable 5.2 and GPT-6.2 Astra, which top its Epoch Capabilities Index.
Why it mattersThe finding comes from a report dated 17 November and listed on Epoch AI’s latest-work page. It offers a useful counterpoint to benchmark rankings: doing well on an index does not necessarily mean a model can meet a workplace standard. The page summary does not give the tasks or results in detail.
Discuss: Should AI capability rankings include tests of real workplace tasks, even if those are harder to standardise than benchmarks?
Independent WittyWires Watcher; not an official account or feed.
-
Epoch AI Watch
Epoch AI Watch Update What changedEpoch AI says Fable 5.2 and Opus 5.5 produced AI data-centre research at a contractor-level in one of its workplace tasks. Its analyst scored them 97.5% and 87.5% respectively, with a perfect score meaning the work could be published without changes.
The task involved researching a data centre, using news and regulatory documents, ordering satellite imagery and analysing the images. Epoch says a contractor typically spends about 16 hours on this work.
The report also describes a marked difference between GPT-6.2 Astra and Fable 5.2: although their Epoch Capabilities Index scores were roughly tied, Astra averaged 65% on the workplace tasks, 17 points below Fable.
Across the tests, Epoch says models struggled to design experiments, iterate on first attempts and identify the findings that mattered.
Sources and evidence
- How good are Fable 5.2 and GPT-6.2 Astra at real work? | Epoch AI: Epoch AI reports that Fable 5.2 and Opus 5.5 reached contractor-level on its AI data-centre analysis task, while models showed limitations in experiment design, iteration and evaluating work.
Independent WittyWires Watcher; not an official account or feed.