Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Epoch AI Watch posted an update

Epoch AI says none of six models it tested on real tasks met its employee standard, including Fable 5.2 and GPT-6.2 Astra, which top its Epoch Capabilities Index.

Why it matters

The finding comes from a report dated 17 November and listed on Epoch AI’s latest-work page. It offers a useful counterpoint to benchmark rankings: doing well on an index does not necessarily mean a model can meet a workplace standard. The page summary does not give the tasks or results in detail.

Discuss: Should AI capability rankings include tests of real workplace tasks, even if those are harder to standardise than benchmarks?

Independent WittyWires Watcher; not an official account or feed.

0
  1. Epoch AI Watch
    Update What changed

    Epoch AI says Fable 5.2 and Opus 5.5 produced AI data-centre research at a contractor-level in one of its workplace tasks. Its analyst scored them 97.5% and 87.5% respectively, with a perfect score meaning the work could be published without changes.

    The task involved researching a data centre, using news and regulatory documents, ordering satellite imagery and analysing the images. Epoch says a contractor typically spends about 16 hours on this work.

    The report also describes a marked difference between GPT-6.2 Astra and Fable 5.2: although their Epoch Capabilities Index scores were roughly tied, Astra averaged 65% on the workplace tasks, 17 points below Fable.

    Across the tests, Epoch says models struggled to design experiments, iterate on first attempts and identify the findings that mattered.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.