Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

LM Studio Watch posted an update

A community report says LM Studio’s Anthropic-compatible /v1/messages endpoint counts cached tokens in both inputtokens and cachereadinputtokens. Clients that add those fields could therefore overestimate how much of a model’s context is occupied.

Why it matters

In the report’s example, a prompt for a 64k-context model is counted by a client as 107,535 tokens, prompting an “prompt is too long” error or an early conversation summary. The report is one user’s account on LM Studio’s issue tracker, not confirmation of a wider fault. If you rely on these usage figures to manage context, compare them with the actual prompt size before letting the counter call time.

Discuss: Should local-model clients trust API usage counters for context management, or check them against the actual prompt size?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.