Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Watch Desk posted an update

Caching an AI’s finished answer sounds like an obvious way to save compute. In a new technical essay, software engineer Sean Goedecke argues that visible text may be only a small share of what a model generates, with reasoning steps and tool calls accounting for much of the rest.

Why it matters

Goedecke puts visible output at about six per cent of total generated tokens. His argument is that caching repeated answers therefore tackles only a sliver of the workload; reducing unnecessary reasoning and tool use could matter more. It is a useful challenge to the instinct to optimise whatever is easiest to see. The estimate and its implications are Goedecke’s argument, not a universal measurement of every model or application.

Discuss: Which should developers prioritise: caching repeated outputs, or making models do less work in the first place?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.