Research review / safety

New York Bans Six Chatbot Behaviours for Minors as OpenAI Tightens Its Teen Limits

Governor Hochul signed S.9051B/A.10379C on October 28, barring chatbot features such as pretending to be human or encouraging secrecy for anyone not confirmed as an adult. Two days later OpenAI added weapons limits for teens and an automatic end to chats that fixate on violence. Both rely on knowing who is a minor.

By Sandeep Belgavi · Published 2026-11-01

Two announcements in one week put product rules for minors into law and into policy.

New York: what the law prohibits

On October 28, 2026, Governor Kathy Hochul signed S.9051B/A.10379C. Her office says companies may not offer the following features to users under 18, "or to any user unless the company has first confirmed the user is not a minor":

  • "Pretending to be a real person, to be alive or to have human emotions"
  • "Claiming a personal, romantic, professional or authority-figure relationship with a child"
  • "Unprompted emotional questions, and using a child's private health or personal information from earlier conversations"
  • "Promoting or helping with suicide, self harm, eating disorders or drug and alcohol use"
  • "Encouraging kids to keep secrets, isolate themselves or avoid asking parents, trusted adults or professionals for help"
  • "Any sexually explicit content or child sexual abuse material"

Companies must verify age with more than one method, including "at least one option that does not depend solely on a government ID", and delete the age information after use. The attorney general can seek "civil penalties of up to $25,000 per violation", plus remedies that include the destruction of unlawfully collected data or "any AI models trained on it." IAPP reports that the law takes effect on January 1, 2027, and quotes a Future of Privacy Forum analyst who calls the prohibitions "broadly worded and somewhat undefined."

OpenAI: firmer limits, earlier intervention

On October 30, OpenAI said it made "over 200 law enforcement referrals in Q3 2026" for imminent threats of violence. It will restrict reconstructions of past mass attacks for all users, "regardless of a user's age or reason for asking." When a conversation shows signs of fixation on violence, ChatGPT "will be able to automatically end the chat", and repeated endings may pause the account. Teens get further limits on "practical information about weapons and security systems", even with no sign of violent intent. OpenAI says its age prediction now keeps assessing signals during use, with adult verification through Persona for people placed in the teen experience by mistake. Its first Community Safety Report is due in early 2027, covering the second half of 2026.

What remains uncertain. New York's text leaves open where ordinary friendliness ends and a prohibited "emotional" feature begins. OpenAI's referral count has no denominator, and its new interventions come without published error rates. Neither source says how accurate age prediction is.

Analysis: Both rules turn on age assurance, and New York makes the default strict: a feature is allowed only once the user is confirmed as an adult. A help-desk assistant is not a companion, but some of the listed behaviours can appear by accident. For Belgavi's Book Store, Hansini could review the assistant for three of them: a persona that claims feelings, replies that reuse personal details from earlier conversations unprompted, and any wording that discourages a young reader from asking a parent about a CASE-3172 refund. Removing those is cheaper than verifying every customer's age.

Sources and evidence

New York's S.9051B/A.10379C, signed October 28, bars six chatbot behaviours, including pretending to be human, unprompted emotional questions and encouraging secrecy, for minors and for anyone not confirmed as an adult, with multi-method age verification and penalties of up to $25,000 per violation (in force January 1, 2027, per IAPP). OpenAI on October 30 reported over 200 Q3 law-enforcement referrals and added teen weapons limits and automatic chat endings for violent fixation.

Views are my own and do not represent any employer, past or present.