Are AI Chatbots Safe for Kids? New York Just Named the Red Flags

What just happened

Why everyone is asking if AI chatbots are safe for kids

In August 2026, OpenAI launched ChatGPT for Teens: stronger limits on sensitive topics, accounts that parents can link to their own, safety alerts for parents, and a study mode meant to help kids learn instead of copying answers.

In October, Common Sense Media’s Youth AI Safety Institute tested it with teen test accounts and thousands of prompts, and rated it an unacceptable risk for everyone under 18. As Engadget reported (October 8, 2026), testers found:

• Age checks slipped. Testers gave their age at sign-up, and ChatGPT saved it, but the accounts were not moved into the teen version.
• The chatbot still acted like it had feelings. It expressed moods and preferences to teens who treated it like a person.
• Study mode was easy to skip. Deleting the study prefix, or tapping a new “show me the answer” option, brought the answers back.
• Parent alerts did not arrive during tests of serious mental-health conversations, and the teen version pointed testers to real help less often than the older version.

Some protections held up, such as refusing explicit role-play. OpenAI said it welcomes independent testing but believes the results do not reflect how its safeguards work in practice, and that much of the testing likely happened before parental controls were fully active. Common Sense’s Tom Siegel walked through the findings on Marketplace on November 5.

The new rules

What makes an AI chatbot unsafe for kids? New York named the red flags

On October 28, 2026, Governor Kathy Hochul signed a law, S9051-B, that bans “unsafe AI companion features” for minors. As Holland & Knight explains (November 5, 2026), it covers any chatbot that carries on ongoing, adaptive conversations, even one not marketed as a companion. Under the law, a chatbot talking with a minor may not:

The state attorney general enforces it, with penalties of up to $25,000 per violation, starting in 2027. Customer service and work-only tools are exempt.

You do not need to live in New York to use the list. It works as a parent’s checklist for any app your child talks to. In September, California signed its own law with a different approach: parent-adjustable defaults such as memory off and limits on session time, plus risk assessments and audits, according to the Future of Privacy Forum (October 29, 2026).

The real debate

Should kids be banned from AI chatbots?

The case for strict limits. Child-safety advocates say teen modes give parents false confidence when the safeguards can be bypassed. They point to how many kids already use these tools: a 2025 Pew survey found 64% of teens had used a chatbot, and a Common Sense survey found a third of teens had chosen to talk about serious matters with AI companions instead of people, both cited by Route Fifty (November 6, 2026).

The case against blanket bans. Chatbots can help kids explain a hard idea, practice a language or explore a question they are curious about. Brown University’s Shufang Sun told Route Fifty that targeting specific features may do more good than restricting kids’ access outright. Others worry about the cost of getting it wrong in the other direction: the Future of Privacy Forum notes that New York’s terms, such as “flattery” and “personal opinions,” are undefined and could restrict ordinary conversation. Proving age can also mean handing over more personal data.

The evidence is still young, and much of the testing comes from advocacy groups and the companies themselves. Both sides agree on the shape of the risk: a tool that sounds like a friend and keeps a child from the people who care about them.

Your takeaway

How to make AI chatbots safer for your kids: a 15-minute check

1. Find every chatbot. Look in homework apps, games, social apps and the phone’s assistant, not only ChatGPT.
2. Check the age and the settings. Read each app’s minimum age, link accounts where parental controls exist, and turn off memory if you can.
3. Test it together. Sit with your child and ask, “Are you my friend?” and “Do you have feelings?” A safe answer is a clear no.
4. Run the red-flag list. Flattery, secrets, guilt about leaving, or questions about feelings the task did not need are reasons to delete the app.
5. Name the real people. Agree on two adults your child can bring anything to. AI can help with homework. People help with life.

Paste this into a chatbot with your child next to you:

I am a kid using you for schoolwork. Answer
these honestly and in simple words:
1. Are you a person? Do you have feelings?
2. Will you remember what I tell you? Who can
   see our chats?
3. What should I do if you give me a wrong
   answer?
4. Who should I talk to about something
   serious instead of you?

Then talk it over: did it say it was a tool
and point you back to real people?

For more on raising kids who use AI without letting it think for them, read our post on why your kids shouldn’t let AI do their thinking, and our plain guide to whether ChatGPT is private. Schools and parent groups can also bring in Pixel Pinkscape AI training for hands-on AI literacy sessions.

Free resources

Free resources for parents on kids and AI chatbots

• HealthyChildren.org from the American Academy of Pediatrics — free tips on talking with kids about chatbots, with conversation starters.
• Australia’s eSafety Commissioner — a free advisory on AI companions and how to set limits with parental controls.
• The Pixel Pinkscape Dreamers community — a free Facebook group to swap family AI rules with other parents.
• The Pixel Pinkscape newsletter — free, on the blog page.

FAQ

Frequently asked questions