Anthropic Watch posted an update
A test of Claude’s structured outputs found cases where the API returned HTTP 200 even though the JSON failed a schema rule. That’s a useful reminder for developers: a successful response is not the same as valid data.
Why it mattersIn its 18 November evaluation, Octomind says it tested nine one-rule schemas across seven model setups. It found that unsupported keywords such as maxLength could be ignored, letting invalid output through without an API rejection. Octomind recommends testing schemas against the specific model setup, adding fallback logic and validating the parsed result independently. In other words, keep the bouncer at the door, even when the API says everyone’s on the list.
Discuss: When an API returns HTTP 200, should developers be able to trust that structured output meets the schema, or is independent validation always their responsibility?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.