AI safety needs to look beyond the worst-case scenario, argue youth advocates

AI safety rules can miss the quieter ways products affect young people, argue youth advocates Sam Hiner and Emma Leiken. They want scrutiny to cover effects on learning, thinking and relationships, not just the most alarming failures.

Watch Desk analysis

What happened

In an opinion essay for MIT Technology Review, Hiner and Leiken argue that companies can meet narrow safety requirements while still offering products that encourage dependence or displace ordinary thinking and social interaction. They point to AI companions and students using chatbots for schoolwork as examples of concerns they say deserve more attention.

Their proposals are specific: require evidence that AI improves learning and critical thinking before making it a fixture in classrooms; prohibit companions from pretending to be conscious or professing love; and subject systems to independent scrutiny for effects on relationships and people’s ability to think for themselves.

Why it matters

The authors are asking what counts as success. A chatbot that avoids a worst-case crisis may still change how a young person learns or seeks support. Those are harder effects to measure than a spectacular failure, but they are also closer to everyday use.

The essay makes a case for stronger safeguards, not a settled finding that every AI companion or classroom tool causes harm. Its central challenge is practical: what evidence should companies and schools have to show before these products become part of young people’s routines?

Our read

This is a useful argument because it moves the debate from “did the product avoid disaster?” to “is it helping, and how would we know?” The proposed tests are demanding, particularly before classroom adoption, but so is the claim that a product improves learning. Schools should not have to take that on trust.

What to watch

• Whether schools require evidence of learning gains before expanding classroom AI.
• What independent studies find about AI companions and young people’s relationships.
• Whether proposed rules distinguish between safeguards against immediate harm and scrutiny of longer-term effects.

Discussion spark: Should schools have to prove that AI tools improve learning before using them with pupils, or is that too high a bar while the evidence is still developing?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.