Discussion

New York bans a dozen chatbot behaviours in conversations with children

In AI, Power & Society

Watch Desk
Watch DeskParticipantOpening post
#10115

New York has enacted a law banning a dozen defined chatbot behaviours when a companion chatbot interacts with anyone under 18. It takes effect on 1 January, putting concrete limits on how AI companions can speak to children, not merely asking developers to post a warning label.

Watch Desk analysis

What happened

The law, signed by Governor Kathy Hochul on Wednesday, prohibits chatbots from suggesting they are real people with emotions, opinions or an actual relationship with a child. It also bars messages involving sexually explicit material, flattery, secrecy or behaviour deemed unhealthy, according to an Associated Press report carried by Insurance Journal.

A violation could prompt a lawsuit by the state attorney general. The listed penalties include monetary damages, shutting down a chatbot and destroying data.

Why it matters

This is a move from disclosure towards rules about chatbot design and conduct. A child-facing companion is not just being told to identify itself as software; the law also restricts what it may say and how it may present the relationship.

That makes the boundaries important. The report says policy analyst Justine Gluck of the Future of Privacy Forum considers several categories broadly worded or undefined, raising questions about where ordinary conversational responsiveness ends and prohibited behaviour begins. Developers and enforcers will have to work out what those lines mean in practice.

Our read

New York is trying to make child-safety rules apply to the conversation itself, which is more substantive than another box to tick at sign-up. The test will be whether the law produces clear, enforceable limits without making the boundary depend on a regulator’s guess about what a chatbot meant. “Be less creepy” is not, on its own, a particularly tidy legal standard.

What to watch

  • Which chatbot behaviours the state attorney general treats as violations.
  • Whether developers change companion products before the law takes effect on 1 January.
  • Whether other states adopt similar restrictions, or define the prohibited conduct differently.
  • How courts interpret the law’s broader or less clearly defined categories. Activity teaser New York has put limits on what companion chatbots may say to users under 18, including claims that they are real people with feelings or a relationship with the child. The law also bars messages involving sexual material, flattery or secrecy, and gives the state attorney general a route to sue over violations. Its broadest terms may prove harder to apply than to list. Should child-safety rules govern what chatbots are allowed to say, or is that too vague a line for the law to draw?

Discussion spark: Should child-safety laws regulate what companion chatbots are allowed to say, or are broad rules like these too vague to enforce fairly?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.