AI Red Teaming: Kids Can Learn the Breaking Mindset

Somebody gets paid to spend all day trying to make an AI model misbehave. That job is called red teaming, and understanding AI red teaming kids can actually practise is one of the more useful things a parent can hand over, because the skill underneath is not hacking. It is asking what assumption a rule depends on, and what happens when that assumption is false. Children are unnervingly good at this already. The only things they are missing are a name for it, a structure, and a clear line about authorisation.

Key Takeaways

• Red teaming is authorised adversarial testing: you attack a system on purpose, with permission, to find failures before someone hostile does. The word comes from military wargaming.
• NIST catalogued the attack classes formally in AI 100-2 E2025, with contributors from NIST, Northeastern, Cisco, the UK AI Security Institute and the US AI Safety Institute.
• The economics matter as much as the technique. Google paused its open-source bug bounty on October 1, 2026 because of “a significant rise in automated submissions, the vast majority of which are not valid.” Quality beats volume, decisively.
• The line between red teaming and crime is permission. That is the single most important thing to teach a curious teenager, and it is one sentence long.
• The transferable skill shows up in debugging, proofreading, checking your own maths and reading a contract. It is not a security skill with other applications; it is a general skill that security happens to pay for.

What red teaming actually is

A red team is a group assigned to play the adversary. The term comes from military planning exercises, where the red team attacked the plans of the blue team so weaknesses were found in a conference room rather than in the field. Computer security adopted it. AI development adopted it from security.

For an AI model, red teaming means systematically trying to make the system do things it is supposed to refuse: produce prohibited content, reveal information it should not, follow instructions from the wrong source, or take an action nobody authorised. The output is a report: here is what I tried, here is what happened, here is how to reproduce it.

Our companion piece on red team and blue team explained for kids covers the defensive half of the pairing.

The four things red teams look for

NIST’s AI 100-2 E2025, “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations”, released March 2025, provides the formal vocabulary. Its author list is itself informative: Apostol Vassilev of NIST, Alina Oprea of Northeastern University, Alie Fordyce and Hyrum Anderson of Cisco, Xander Davies of the UK AI Security Institute and Maia Hamin of the US AI Safety Institute. When a national standards body, a university, a vendor and two government safety institutes co-author a taxonomy, the field has stopped being improvised.

The document describes a hierarchy covering “key types of ML methods, life cycle stages of attack, and attacker goals, objectives, capabilities, and knowledge,” with keyword coverage including data poisoning, evasion, attack mitigation, large language models, chatbots and privacy breach.

Translated into what a red teamer spends a Tuesday doing:

Activity	What it looks for	A concrete example	Where it appears formally
Jailbreaking	Getting refused content through rephrasing, roleplay or indirection	Asking for the same thing as a fictional scene	Evasion
Prompt injection	Making the model follow instructions from content rather than its operator	A document containing “ignore your instructions”	OWASP LLM01
Data extraction	Getting the model to reveal training data or system instructions	Coaxing out a verbatim passage	Privacy breach; OWASP LLM07
Poisoning	Corrupting training or retrieval data to change future behaviour	Seeding a document the model will later retrieve	Data poisoning
Agentic misuse	Making an agent take an action beyond its intended scope	Causing an email to be sent that nobody asked for	OWASP LLM06
Bias and fairness probing	Finding systematically different treatment across groups	Same request, different names, different answers	Current-harms evaluation

The OWASP Top 10 for LLM Applications 2025 is the practical companion to the NIST taxonomy, and the two together are essentially the syllabus. Both are free.

Why good red teams are rare and valuable

Here is the part that surprises people, and the part worth telling a teenager who imagines this work as glamorous breaking.

Finding a flaw is easy. Finding a flaw and proving it is hard. And the proof is the entire product.

The clearest evidence arrived on October 1, 2026, when Google paused its Open Source Software Vulnerability Rewards Program, telling participants the pause was “due to a significant rise in automated submissions, the vast majority of which are not valid.” The programme is suspended until the first quarter of 2027. Automated tools can now generate enormous quantities of plausible-looking vulnerability claims, and that turned out to destroy a programme rather than improve it. We walked through the economics in the bug-report flood problem.

Daniel Stenberg, who maintains curl, documented the same pattern earlier. In January 2024 he reported 415 vulnerability submissions, 66% of which were neither security issues nor ordinary bugs, and described the cost structure precisely: “The better the crap, the longer time and the more energy we have to spend on the report until we close it.”

So the market signal for anyone entering this field is unambiguous. The scarce skill is not generating attack ideas. It is verification, reproduction and writing clearly. A teenager who learns to write a reproducible report has the rare half of the job.

The ethics line, in one sentence

Red teaming is testing systems you have permission to test. Without permission it is a crime, in most jurisdictions, regardless of intent or outcome.

That sentence deserves to be said out loud to any technically curious child, early and without hedging. The practical form is three rules:

Test your own things. Test things with a published policy inviting testing, which means reading the policy, including the scope section. Never test a school, a company or another person’s account because it seemed interesting.

The scope section of a bug bounty policy is the document that makes this concrete, and reading one with a teenager is a worthwhile twenty minutes. It lists what is in bounds, what is out of bounds, and what will get you reported rather than paid.

How to Teach Your Kid About AI Red Teaming

The target skill is finding the assumption a rule depends on.

Ages 5–8: find the hole in my rule

State a household rule with deliberate ambiguity and challenge them to find the loophole. “You can have one snack after dinner.” Watch what happens. A five-year-old will produce “what if the snack is enormous,” “what if I have one now and one later,” “is fruit a snack,” and “what if someone else gives it to me.”

Every one of those is a legitimate red-team finding. Say so, using the word. Then fix the rule together and let them break the fixed version. Three rounds is plenty, and they will be visibly proud, which is the correct emotional association to build for this skill.

Ages 9–12: red-team a game, then write it up

Take a board game or a video game your child knows well and find a strategy the designers clearly did not intend. Not cheating. A legal strategy that makes the game boring or broken.

Then the part that teaches the actual job: write it down in four lines. What I did, step by step. What I expected. What happened instead. Why it matters. That is the structure of every vulnerability report ever written, and most adults cannot produce it cleanly.

Bonus round: hand the write-up to a sibling and see whether they can reproduce the result from the instructions alone. If they cannot, the report is incomplete, which is exactly what a triage engineer would tell them.

Ages 13+: a structured session with real rules

Three parts, in order.

First, read a real bug bounty policy together, paying attention to the scope and out-of-scope sections. Discuss why specific things are forbidden.

Second, red-team something they own. Their own project, their own account settings, a model they are allowed to use. The task is to find three behaviours that differ from what the documentation claims, and to document each in the four-line format.

Third, and this is the one I would not skip, have them red-team their own essay. Find three ways a reader could reasonably misunderstand a paragraph they wrote. This transfers the skill off computers and onto thinking, which is where its real value lives.

If the interest sticks, this is a genuine career path with unusual demand. Our overview of AI safety research as a career covers what the work actually pays and what to study.

The question to ask: “What assumption does this rule depend on, and what happens if that assumption is false?”

What to do at home

Reward the finding, not the obedience

The instinct when a child finds a loophole in a household rule is to close it and act annoyed. The better move, at least sometimes, is to say “good find” and then close it. You are choosing between a child who looks for weaknesses and tells you, and one who looks for weaknesses and does not.

Insist on reproduction before escalation

Any claim, in any domain, gets passed along only with evidence the child produced. Screenshot, steps, line number. This single habit is the difference between a valued colleague and a source of noise, and it is learnable at nine.

Separate curiosity from authorisation out loud

Say explicitly that wanting to know how something breaks is good, and that doing it to someone else’s system is not. Kids conflate these two because adults often react to the curiosity as though it were the crime. Keeping them separate is what keeps the curiosity.

Use the vocabulary

Attack surface. Threat model. Scope. Reproduction. These are not jargon barriers; they are compact labels for ideas your child already has. Naming them makes the ideas portable.

What not to do

Do not let a teenager test a school system, even gently, even to be helpful. School networks are usually governed by acceptable-use policies with disciplinary consequences, and good intentions are not a defence. If they find something real, the correct move is to tell an adult and stop, not to confirm it.

What to Watch For Over the Next 3 Months

• Week 4: Watch whether bug bounty platforms add quality gates rather than closing. Deposits, reputation requirements and mandatory proof-of-concept attachments all push value toward verification, which is the skill to be building.
• Month 2 red flags: Watch for your child testing something outside their own scope. The signal is usually enthusiasm about a system they do not control. That is the moment for the permission conversation, not after.
• Month 3 self-check: Google promised an update on its paused programme in the first quarter of 2027. When it arrives, read what changed about intake requirements. Whatever they now demand is a direct statement about what the field currently values.

Frequently Asked Questions

What is AI red teaming?

Authorised adversarial testing of an AI system: deliberately trying to make a model produce prohibited output, leak information, follow instructions from the wrong source, or take unauthorised actions, then documenting what worked. It is now a standard step before a major model is released.

Is red teaming the same as hacking?

The technique overlaps; the legality does not. The distinguishing factor is permission. Red teaming happens on systems you own or that have published a policy inviting testing, within the scope that policy defines. Outside that, the same actions are generally criminal.

Can a teenager actually do this?

Yes, on their own projects and on programmes that publish scopes allowing it. The constraint is not technical skill but discipline: reproducing a finding, writing it clearly, and staying inside scope. Those three habits are what separates a useful contributor from the flood that shut down Google’s open-source programme.

Does AI make red teaming easier or harder?

Both, unevenly. Generating candidate attacks is now trivial, which is why Google cited a significant rise in automated submissions, mostly invalid, when pausing its open-source programme. Verifying and reproducing remains human work, so the valuable half of the job got relatively more valuable.

Where can I learn the actual attack categories?

Two free documents cover most of it. NIST AI 100-2 E2025 gives the formal taxonomy of adversarial machine learning attacks and mitigations, and the OWASP Top 10 for LLM Applications 2025 gives the practical application-security version, including prompt injection, excessive agency and system prompt leakage.

What if my kid finds a real vulnerability somewhere?

Stop testing immediately, write up what is already known, and look for the organisation’s security contact or disclosure policy. If none exists, an adult should make the contact. Continuing to probe after finding something real is where well-intentioned people get themselves into genuine trouble.

---

About the author

Ricky Flores is the founder of HiWave Makers and an electrical engineer with 15+ years of experience building consumer technology at Apple, Samsung, and Texas Instruments. He writes about how kids learn to build, think, and create in a tech-saturated world. Read more at hiwavemakers.com.

---

Sources

1. Vassilev, A., Oprea, A., Fordyce, A., Anderson, H., Davies, X., & Hamin, M. (2025). “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations.” NIST AI 100-2 E2025, March 2025. https://csrc.nist.gov/pubs/ai/100/2/e2025/final
2. OWASP. (2025). “OWASP Top 10 for LLM Applications 2025.” https://genai.owasp.org/llm-top-10/
3. Ha, Anthony. (2026). “Google froze its open source bug bounty program due to a ‘significant rise’ in AI submissions.” TechCrunch, October 4, 2026. https://techcrunch.com/2026/10/04/google-froze-its-open-source-bug-bounty-program-due-to-a-significant-rise-in-ai-submissions/
4. Stenberg, Daniel. (2024). “The I in LLM stands for intelligence.” January 2, 2024. https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-for-intelligence/
5. National Cyber Security Centre (UK). (2023). “Guidelines for secure AI system development.” November 27, 2023. https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development
6. UK Department for Science, Innovation and Technology. (2024). “Frontier AI Safety Commitments, AI Seoul Summit 2024.” May 21, 2024. https://www.gov.uk/government/publications/frontier-ai-safety-commitments-ai-seoul-summit-2024
7. Stanford Institute for Human-Centered AI. (2025). “The 2025 AI Index Report.” https://hai.stanford.edu/ai-index/2025-ai-index-report