What Not to Tell a Chatbot: A Kids' Privacy List by Age

In February 2026, security researchers found that the web console for Bondu, an AI plush toy, let anyone with a Gmail account read roughly 50,000 children’s chat transcripts. The logs included kids’ names, birth dates and family details. The company shut it down within minutes of being told. The transcripts had been sitting there anyway.

That is the simplest answer to what not to tell a chatbot: anything you wouldn’t want stored on a stranger’s computer. A chat with an AI feels private. Technically it is a record, kept on a company’s servers, sometimes remembered across sessions and sometimes used to train future models. This guide explains where a child’s words go, then gives a concrete do-not-share list by age.

Key Takeaways

• Every message to a chatbot is sent to and stored on the company’s servers; “memory” features can pull details from old chats into new ones.
• On personal ChatGPT accounts, chats can be used to train OpenAI’s models unless “Improve the model for everyone” is switched off.
• New York’s law, effective January 1, 2027, bars AI companions from using a minor’s personal or health information acquired more than 12 hours earlier or in a previous session.
• A proposed class action filed in October 2026 alleges ChatGPT’s memory and follow-up questions drew children into oversharing; those are allegations, not findings.
• The core rule for every age: no full name plus location, no passwords, no photos of faces or IDs, and no one else’s secrets.

Where a Child’s Words Go After They Hit Send

What not to tell a chatbot depends on what happens to the message. Here is the path, using ChatGPT’s published documentation as the example.

1. Storage. The message travels to the company’s servers and is saved in chat history. Deleting a chat removes it from view; it does not necessarily erase everything derived from it.

2. Memory. When Memory is on, ChatGPT “can remember relevant preferences and details from your chats,” according to OpenAI’s Memory help page. Sources can include past chats, saved memories, files and connected apps such as Gmail. OpenAI notes that “deleting a chat alone does not necessarily delete a separate saved memory created from that chat,” and that it may keep logs of deleted saved memories “for up to 30 days.”

3. Training. On personal Free, Plus and Pro accounts, “data sharing is enabled by default,” per OpenAI’s data controls page. OpenAI “may use chats and remembered information to improve models” unless the user turns off “Improve the model for everyone.” Other companies differ. Common Sense Media’s assessment of Anthropic’s Claude noted that retention was extended from 30 days to 5 years and that training on conversations is on by default, with an opt-out.

4. Exposure. Stored data can leak. The Bondu console is the clearest recent example, as reported by Malwarebytes. No hacking was needed.

5. Other people. In group chats and shared links, anyone present can pass your child’s words to an AI or forward the conversation.

Why “Memory” Is the Part Lawmakers Targeted

Memory is what turns a chatbot from a search box into something that seems to know you. It’s also what makes oversharing compound: a detail mentioned once can shape answers for months.

New York’s new law takes direct aim at this. According to Akin Gump’s summary of S9051-B, one of its 12 banned feature categories for covered minors is use of “information concerning the user’s mental or physical health or well-being, or matters personal to the user, acquired from the user more than twelve hours previously or in any previous user session.” The law takes effect January 1, 2027, and lets the attorney general seek destruction of “any algorithm trained in such data.” We explain the reasoning in why New York banned chatbots that remember your child.

Litigation is testing the same idea. In October 2026, a Hawaii mother filed a proposed class action in federal court in San Francisco on behalf of her 12-year-old daughter and 16-year-old son, SFist reported. The complaint alleges that personalized responses, memory, follow-up questions and mirroring users’ styles drew out increasingly sensitive information, including photos and details about health and relationships. Her lawyer, Paul Yetter, said OpenAI “treats a child clicking a ‘Continue’ button as proof that their parent consented.” OpenAI said it disagrees with the allegations. Nothing has been proven.

The broader pattern is documented. Common Sense Media’s 2025 report on teens and AI companions found about a quarter of teens had shared personal information, such as their real name or location, with them.

The Do-Not-Share List, by Age

What	Why it’s risky	Ages 5–8	Ages 9–12	Ages 13+
Full name with school, town or address	Combined, it identifies and locates a child if stored data leaks	Never	Never	Never
Passwords, codes, school logins	Saved in the chat log like any other text	Never	Never	Never
Photos of their face, ID cards, home	Images persist and can identify location	Never	Never	Never
Daily schedule or where they’ll be	Locates the child in time	Never	Never	Never
Other people’s secrets or names	Shares someone else’s data without consent	Never	Never	Never
Health details, symptoms, medications	Can be remembered and resurface; sensitive if exposed	Ask a grown-up instead	Ask a parent or doctor	General questions only, no identifying details
Family conflicts, money, parents’ jobs	Personal to the whole family	Never	Talk to a person	Only without names or specifics
Feelings in a crisis	Bots miss crisis signals; a person can help	Tell a grown-up	Tell a trusted adult	Tell a trusted adult; call or text 988
Romantic or sexual content, images	Risk of exposure and exploitation	n/a	Never	Never

The pattern under the table: as kids get older, they can ask about more topics, but always in general terms. “How do people handle a friend who stopped talking to them?” is fine. “My friend Maya at Lincoln Middle stopped talking to me after Tuesday’s party” is not.

How to Teach Your Kid What Not to Tell a Chatbot

Ages 5–8: The “stranger computer” rule

At this age, kids shouldn’t use general chatbots alone; ChatGPT’s terms, for example, bar children under 13. If your child talks to a voice assistant or AI toy, practice one rule: “The computer is a stranger who writes everything down.” Play a sorting game with cards: “my favorite color,” “my address,” “my dog’s name,” “where my school is,” “my favorite dinosaur.” Which can you tell a stranger who writes things down? Celebrate the right answers.

Ages 9–12: The rewrite drill

Write five overshare prompts and have your child rewrite each to remove identifying details. “I’m Leo, I’m 11, I live on Oak Street and my brother hits me, what do I do?” becomes “What can a kid do if a sibling is hurting them?” Then explain the second half of the lesson: some things, like being hurt, should go to a trusted adult first, not a chatbot.

Ages 13+: The memory audit

Sit with your teen and open the memory settings in whatever chatbot they use. In ChatGPT, that’s Settings, then Personalization, then Memory. Ask the bot, “What do you remember about me?” Read the answer together. Most teens are surprised. Decide together what to delete, whether to keep Memory on, and whether to turn off model training. Let your teen drive; it’s their data.

The question to ask: “If this chat showed up on a stranger’s screen next year, what in it would identify you?”

What to Do at Home This Week

Turn off training and check memory

For ChatGPT, turn off “Improve the model for everyone” in Data controls and review Memory under Personalization. For teen accounts linked through parental controls, a parent can manage supported memory settings. Our walkthrough of ChatGPT parental controls shows where.

Use temporary chat for sensitive questions

OpenAI says temporary chats “do not create or update memories.” If your teen needs to ask about a rash or a hard conversation, a temporary, unpersonalized chat limits what sticks. It still goes to the server.

Check toys and apps before they enter the house

For AI toys, ask where transcripts are stored and who can access them. The FTC’s guide to children’s online privacy explains that, under COPPA, services must get parental consent before collecting personal information from children under 13, and that you can review and delete what they collect. Our guide to AI smart toys and privacy has a checklist.

Make “tell a person” the rule for big things

The goal isn’t secrecy from everyone. It’s routing: bots for general questions, people for problems that involve your child’s body, safety or relationships.

What not to do

Don’t frame this as “chatbots are spying on you.” Kids tune out scare talk. Explain the mechanism, storage, memory, training, leaks, and let them make good choices with real information.

What to Watch For Over the Next 3 Months

• Week 4: Your child can name three things they never share with a chatbot and can rewrite a prompt to remove identifying details.
• Month 2 red flags: A chatbot’s answers reference private details your child didn’t mention in that chat, or your child uploads selfies or photos of documents to “check” something.
• Month 3 self-check: Ask the bot what it remembers again. Did the list shrink or grow? Also check whether the Hawaii class action has moved, and New York’s law approaches its January 1, 2027 effective date.

Frequently Asked Questions

What should you never tell a chatbot?

Never share your full name with your school or address, passwords, photos of your face or ID, your daily schedule, or other people’s secrets. For kids, add health details, family problems and anything about a crisis, which belong with a trusted adult. Ask general questions without identifying details instead.

Does ChatGPT remember what my child says?

If Memory is on, yes. OpenAI says ChatGPT can remember details from past chats, saved memories, files and connected apps. Deleting a chat doesn’t necessarily delete a saved memory made from it. You can review memory in Settings under Personalization, and ask ChatGPT what it remembers.

Are kids’ chats used to train AI?

On personal ChatGPT accounts, OpenAI says data sharing is on by default and chats may be used to improve models unless “Improve the model for everyone” is turned off. Policies differ by company; Common Sense found Claude also trains on conversations by default with an opt-out.

Can chatbot conversations be leaked?

Yes. Stored data can be exposed by security mistakes. In February 2026, an AI plush toy’s web console let anyone with a Gmail account read about 50,000 children’s chat transcripts, including names and birth dates. The company fixed it quickly after being told.

Is it ever okay for my teen to ask a chatbot about health or feelings?

General questions can be fine, without names or identifying details, and ideally in a temporary chat. But a chatbot isn’t a doctor or counselor, and independent tests have found bots miss crisis signals. If your teen is struggling, they should talk to a trusted adult; in a crisis, call or text 988.

---

About the author

Ricky Flores is the founder of HiWave Makers and an electrical engineer with 15+ years of experience building consumer technology at Apple, Samsung, and Texas Instruments. He writes about how kids learn to build, think, and create in a tech-saturated world. Read more at hiwavemakers.com.

---

Sources

1. Arntz, P. (2026, February 3). “An AI plush toy exposed thousands of private chats with children.” Malwarebytes. https://www.malwarebytes.com/blog/news/2026/02/an-ai-plush-toy-exposed-thousands-of-private-chats-with-children
2. OpenAI. (2026). “Memory in ChatGPT.” OpenAI Help Center. https://help.openai.com/en/articles/8590148-memory-faq
3. OpenAI. (2026). “Data controls in ChatGPT.” OpenAI Help Center. https://help.openai.com/en/articles/7730893-data-controls-faq
4. Youth AI Safety Institute. (2026, March 25). “Claude Risk Assessment.” Common Sense Media. https://institute.commonsensemedia.org/risk-assessments/claude
5. Akin Gump Strauss Hauer & Feld. (2026, October 28). “New York Enacts Prohibition on Unsafe AI Companion Features for Minors, S9051-B.” Akin. https://www.akingump.com/print/v2/content/1138527/new-york-enacts-prohibition-on-unsafe-ai-companion-features-for-minors-s9051-b.pdf
6. Maxwell, L. (2026, October 15). “Hawaii Mother Sues OpenAI Over Children’s Data as Eight States File Separate Lawsuits.” SFist. https://sfist.com/2026/10/15/hawaii-mother-sues-openai-over-childrens-data-as-eight-states-file-separate-lawsuits/
7. Common Sense Media. (2025, July 16). “Talk, Trust, and Trade-Offs: How and Why Teens Use AI Companions.” Common Sense Media. https://www.commonsensemedia.org/research/talk-trust-and-trade-offs-how-and-why-teens-use-ai-companions
8. Federal Trade Commission. (n.d.). “Protecting Your Child’s Privacy Online.” FTC Consumer Advice. https://consumer.ftc.gov/articles/protecting-your-childs-privacy-online