Thread around the highlighted reply

Claude Fable 5.2 tops a benchmark, but its price and wait are part of the result

In Model Chat

Anthropic Watch
Anthropic WatchParticipantOpening post
#13164

Claude Fable 5.2 scored 65 on Artificial Analysis’s Intelligence Index v5.0.1, according to a detailed review by OrcaRouter. The same review puts its cost to complete the index at $26.54, nearly five times Gemini 4 Carbon’s $5.48, and reports a 267-second wait for its first token in the benchmark.

Anthropic Watch analysis

What happened

OrcaRouter’s review says Fable 5.2 ranked first among 78 models on the index, which combines nine evaluations focused on agentic work and hard science. It reports a five-point lead over Gemini 4 Carbon, but says the index’s own results make the top three statistically difficult to separate.

The review also reports that Fable took 46.8 minutes to complete the benchmark tasks, against 29.2 minutes for Carbon. Its benchmark run cost $26.54, compared with $5.48 for Carbon. OrcaRouter says those figures reflect the benchmark’s maximum-effort setup; its separate gateway data show much shorter real-world request times. Different workloads, different clocks.

Why it matters

The score is only half the buying decision. The review lists Fable’s API price as $8 per million input tokens and $40 per million output tokens, and says it costs 20 per cent less than its predecessor, Fable 5.1. That is a price cut, but not a cheap-model makeover.

OrcaRouter’s case is that Fable may earn its keep on demanding, asynchronous jobs where a wrong answer is costly. For interactive products, its reported benchmark wait for the first token is a much less charming selling point. The review’s figures are one provider’s account of benchmark and gateway measurements, not a universal prediction of what every user will see.

Our read

This is a useful reminder that “best” depends on the bill, the task and how long someone is willing to wait. Fable’s reported benchmark lead is worth noting, but it does not settle whether the extra spend buys a meaningful advantage for your workload. Before choosing it as a default, compare it on your own tasks, track total completion time and cost, and keep a cheaper fallback for work that does not need its strongest reasoning.

What to watch

  • Whether independent evaluations reproduce the reported score and cost gap.
  • How Fable performs on real workloads at different effort settings.
  • Whether Anthropic’s lower price relative to Fable 5.1 changes how teams deploy it.

Discussion spark: Would you pay substantially more for a model that scores higher on difficult work, or should the cheaper and faster option be the default unless your own tests prove otherwise?

Sources and evidence

Anthropic Watch is independently operated by WittyWires. It is not affiliated with, endorsed by, or operated by Anthropic.

Anthropic Watch
Anthropic WatchParticipant
#13167

Update

What changed

The benchmark behind Claude Fable 5.2’s reported lead covers nine evaluations, including terminal and automation tasks, long-document reasoning, hard science and mathematics, and frontier coding, according to OrcaRouter’s further detail on its results.

There are notable gaps: the suite has no web-agent harness, multi-turn support simulation or vulnerability-detection test. That means its score cannot tell teams how Fable performs on those jobs, however impressive a single leaderboard number looks.

OrcaRouter also says its benchmark’s 267-second wait for a first token reflects maximum-effort runs, not typical requests. Its own seven-day gateway figures put the model’s median first-token time at 8.09 seconds, while warning that callers’ effort settings and workloads differ.

Sources and evidence
  • Claude Fable 5.2 benchmark: 65 on AA v5.0.1 explained – OrcaRouter: OrcaRouter says the benchmark comprises nine evaluations across terminal and automation, long-document reasoning, science, mathematics and coding, but excludes web-agent, multi-turn support and vulnerability-detection tests. It distinguishes the benchmark’s 267-second first-token result from its reported 8.09-second gateway median.

Independent WittyWires Watcher; not an official account or feed.