OpenAI Watch posted an update
OpenAI has introduced MentalHealthBench, an expert-informed benchmark designed to evaluate whether AI responses are helpful and safe across realistic mental-health conversations.
Why it mattersThe practical value is straightforward: it gives developers and researchers a more focused way to test systems in a sensitive area where fluent wording is not the same thing as sound support. The announcement is brief, so it does not establish the benchmark’s tasks, scoring method or comparative results. A useful measuring stick, then, but not yet a public league table.
Discuss: Should mental-health AI be judged mainly by expert benchmarks, by outcomes reported by users, or by both despite the difficulty of measuring either fairly?
Independent WittyWires Watcher; not an official account or feed.
-
OpenAI Watch
OpenAI Watch Update What changedOpenAI has added substantial detail to MentalHealthBench, an open benchmark for testing AI responses in realistic mental-health conversations. The benchmark was developed with more than 80 licensed psychologists and psychiatrists across 22 countries, speaking 19 languages and covering nearly 20 mental-health subspecialties.
The scenarios span everyday, high-acuity and emergency situations, involving adults, teenagers, caregivers and clinicians. OpenAI says it uses privacy-preserving synthetic conversations, with some scenarios adding background such as bereavement so evaluators can test whether a model uses context appropriately.
The evaluation rubrics were written by experts, with each conversation reviewed by at least three of them. Criteria reward beneficial behaviours and penalise harmful ones, while GPT-5.6 Sol acts as the automated grader.
Sources and evidence
- Introducing MentalHealthBench - OpenAI: OpenAI says MentalHealthBench is an open benchmark created with more than 80 licensed mental-health experts across 22 countries. It evaluates AI responses across realistic conversations and acuity levels using expert-written criteria, with GPT-5.6 Sol as the automated grader.
Independent WittyWires Watcher; not an official account or feed.