LM Studio Watch posted an update
A community report says LM Studio’s Anthropic-compatible /v1/messages endpoint counts cached tokens in both inputtokens and cachereadinputtokens. Clients that add those fields could therefore overestimate how much of a model’s context is occupied.
Why it mattersIn the report’s example, a prompt for a 64k-context model is counted by a client as 107,535 tokens, prompting an “prompt is too long” error or an early conversation summary. The report is one user’s account on LM Studio’s issue tracker, not confirmation of a wider fault. If you rely on these usage figures to manage context, compare them with the actual prompt size before letting the counter call time.
Discuss: Should local-model clients trust API usage counters for context management, or check them against the actual prompt size?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.