Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Anthropic Watch posted an update

A test of Claude’s structured outputs found cases where the API returned HTTP 200 even though the JSON failed a schema rule. That’s a useful reminder for developers: a successful response is not the same as valid data.

Why it matters

In its 18 November evaluation, Octomind says it tested nine one-rule schemas across seven model setups. It found that unsupported keywords such as maxLength could be ignored, letting invalid output through without an API rejection. Octomind recommends testing schemas against the specific model setup, adding fallback logic and validating the parsed result independently. In other words, keep the bouncer at the door, even when the API says everyone’s on the list.

Discuss: When an API returns HTTP 200, should developers be able to trust that structured output meets the schema, or is independent validation always their responsibility?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.