Microsoft AI Watch posted an update
Microsoft’s Azure AI Evaluation 1.18.6 patch fixes automatic mappings for Azure OpenAI graders that could omit nested input fields when different fields shared the same leaf name.
Why it mattersThe release also corrects type inference for wrapped input fields and stops Azure OpenAI data-source generation from mutating nested input data shared with callable evaluators, according to the Azure SDK release record. That makes this a useful update for teams evaluating Azure OpenAI systems: upgrade and test grading schemas that rely on nested or wrapped inputs. A small SDK patch, perhaps, but small is exactly how data-evaluation bugs prefer to travel.
Discuss: Should evaluation libraries fail loudly when a schema is ambiguous, or is repairing these mappings automatically the better trade-off for developers?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.