Learning to Code With AI Help: The Lost Debugging Skill

Learning to code with AI help feels great. The code runs, the errors vanish, the project ships. But a randomized study published in 2026 found that people who learned a new programming library with an AI assistant scored 17% lower on a follow-up quiz than people who learned without one. They weren’t significantly faster either. And the biggest gap was on debugging questions.

The reason is almost too simple: the people without AI ran into more errors, and fixing those errors is how they learned. The AI group’s errors got fixed for them. If your kid is learning to code with AI today, that’s the skill to protect.

Key Takeaways

• In Shen and Tamkin’s randomized study (52 developers, preprint, 2026), the AI group scored 17% lower on a quiz about the library they’d just used, with no significant time savings.
• The largest score gap was on debugging questions. The authors say the no-AI group “encountered more errors during the task and became more capable at debugging.”
• The AI group found the task easier, and the no-AI group reported learning more. Easy and learned didn’t go together.
• Three AI habits preserved learning: asking only conceptual questions, asking for code with explanations, and generating code then asking follow-ups to understand it.
• Kids can use AI while learning to code if the AI explains and the kid writes, runs and fixes.

What the study did and found

“How AI Impacts Skill Formation” is a preprint by Judy Hanwen Shen and Alex Tamkin, posted to arXiv in January 2026. Shen worked through the Anthropic Fellows Program and Tamkin is at Anthropic, which also published a summary of the study. It has not been peer-reviewed.

The setup: participants with at least a year of Python experience learned Trio, a Python library for asynchronous programming, chosen because it would be new to them. They completed two coding tasks covering seven core concepts, either with an AI coding assistant or without one. Then everyone took a 27-point quiz with conceptual, code-reading and debugging questions, but no code-writing questions. The grading rubric was pre-registered.

The main results, from 52 participants (26 per group):

• Lower scores with AI. The AI group averaged 4.15 points lower, about 17% or “two grade points” (Cohen’s d = 0.738, p = 0.010).
• No real speed gain. AI use “did not significantly improve task completion time.”
• Biggest gap in debugging. In an analysis the authors label exploratory and not pre-registered, “the largest score gap occurs in the debugging questions and the smallest score gap in the code reading questions.”
• Easier is not the same as learned. The AI group found the task easier. The no-AI group reported higher self-reported learning. Both found the quiz similarly hard.

One more detail from the pilots is worth knowing: in an early round, 35% of people told not to use AI used it anyway, copying the instructions into a chatbot. The researchers had to switch platforms and use screen recordings to make the no-AI condition real. That tells you how strong the pull is, even for adults who were paid to resist it.

Why errors are where the learning happens

Here’s the mechanism, in engineering terms. To fix a bug, you must form a model of how the system works, compare it to what actually happened, and find where they disagree. Each fixed bug updates your model. That’s the learning.

The study’s error logs make it visible. The no-AI group “encountered many more errors related to key Trio concepts,” like TypeError and RuntimeWarning, errors you only get past by understanding how the library works. With AI, many of those errors either never appeared, because the AI wrote correct code, or got fixed by the AI.

This matches decades of education research. A 2008 literature review in Computer Science Education by Renée McCauley and colleagues called debugging “both difficult for novice programmers to learn and challenging for computer science educators to teach.” In high school e-textiles, Deborah Fields, Yasmin Kafai and colleagues (British Journal of Educational Technology, 2021) went as far as having students design buggy projects on purpose, arguing it makes learners “agents of their own learning and, more importantly, of making and solving mistakes.” Our companion piece on why debugging builds ownership covers that research in depth.

The six ways people used AI, and which ones kept learning

Shen and Tamkin watched screen recordings of the AI group and sorted their behavior into six patterns. The groups are tiny (two to seven people each), so treat this as a set of useful hypotheses.

Pattern	What the person did	Quiz result	Kid-friendly version
AI Delegation (4 people)	Had AI write all the code	Low; fastest overall	”Make it work for me”
Progressive AI Reliance (4)	Asked a question or two, then delegated everything	Low	”Just one hint… okay, just write it”
Iterative AI Debugging (4)	Used AI to debug or verify code, without building understanding	Low; also slow	”Fix my error” over and over
Generation-Then-Comprehension (2)	Generated code, then asked follow-ups to understand it	High	”Write it, then teach me how it works”
Hybrid Code-Explanation (3)	Asked for code plus explanations of it	High; slower	”Show me and explain each line”
Conceptual Inquiry (7)	Asked only conceptual questions; wrote code and fixed errors alone	High; fastest high-scoring group	”Explain the idea; I’ll build it”

Notice two things. “Iterative AI Debugging” scored poorly even though the person was busy with AI the whole time. Activity isn’t learning. And “Conceptual Inquiry,” the group that hit many errors and resolved them independently, was the fastest of the high-scoring patterns. Understanding pays off in speed too.

How to Teach Your Kid to Learn Coding With AI Without Losing Debugging

Ages 5–8: unplugged bugs

Coding at this age is mostly about sequences and instructions. Play “robot”: your kid gives you step-by-step directions to walk to the kitchen, and you follow them literally. When you crash into a chair, they find the wrong step and fix it. Tools like Scratch Jr. work similarly on a tablet. No AI needed. The goal is to make “find the broken step” feel like a game, not a failure.

Ages 9–12: AI as explainer only

When your kid uses Scratch or a beginner Python course and gets stuck, set one rule: they can ask an AI (with you nearby) to explain a concept or an error message in kid language, but they can’t ask for the code. Have them say the explanation back in their own words, then try the fix themselves. That’s the Conceptual Inquiry pattern. Keep a “bug log” in a notebook: the error, what they thought, what fixed it.

Ages 13+: the two-pass rule

Teens working on real projects can use AI more, with a structure. Pass one: they write and debug the code themselves, using AI only for conceptual questions. Pass two, only if they’re truly stuck after a set time: AI can suggest code, but they must then ask it to explain every line and rewrite it in their own style before it goes in the project (Generation-Then-Comprehension). Once a month, have them do a no-AI “debugging drill”: take a working program, break it in three places, and fix it.

The question to ask: “When the error showed up, what did you think was wrong before anyone, or anything, told you?”

Learning to code with AI: how to set it up for a kid

Write the system prompt together

Many chatbots let you set standing instructions. Write one with your kid: “I’m learning to code. Don’t write code for me. Explain concepts and error messages. Ask me questions. If I ask for code, remind me of this rule.” This is the same principle as the hint-only AI tutor that protected learning in a PNAS math study (Bastani et al., 2025).

Separate learning projects from shipping projects

If your teen is building something to share and also learning, decide which is which. For a learning project, the rules above apply strictly. For a project where the goal is just getting it done, more AI help may be fine. Being clear keeps the learning projects honest.

Let errors sit for a few minutes

The urge to fix an error immediately, yours or the AI’s, is strong. Set a timer: five minutes for a 10-year-old, fifteen for a 15-year-old, before any help. The study suggests the errors are where learning happens. Struggling with them for a bit is the point.

Ask for explanations, not results

When your kid shows you working code, ask them to walk you through it. If they can’t explain a part, that part isn’t learned yet, no matter who wrote it. For a broader look at AI in kids’ coding, see our earlier explainer on AI code debugging and kids’ learning.

What not to do

Don’t assume AI coding help is harmless because the project works. The study’s AI group felt the task was easier and produced working code, and still learned less. A working program is not evidence of a learned skill. Don’t ban AI outright either; the high-scoring patterns show it can help when it explains instead of replacing.

What to Watch For Over the Next 3 Months

• Week 4: Your kid has a bug log with at least a few entries they solved alone, and their AI chats are mostly questions, not “write this for me.”
• Month 2 red flags: Projects that work but your kid can’t explain, or a pattern of pasting errors into AI and pasting back the fix (the “Iterative AI Debugging” pattern). Reset with the explainer-only rule.
• Month 3 self-check: Can your kid fix a simple bug in their own code without help? Can they explain a concept they learned this month in their own words? Did they complete a no-AI debugging drill?

Frequently Asked Questions

Is it bad for kids to learn to code with AI?

Not necessarily, but it depends on how. In a 2026 randomized study with adults, AI help led to 17% lower quiz scores, with the biggest gap in debugging. Yet people who used AI to ask conceptual questions or get explanations learned well. Kids should use AI as an explainer, not as the one writing the code.

What did the Anthropic study on AI and coding skills find?

Shen and Tamkin’s preprint found that developers learning a new Python library with AI scored 17% lower on a follow-up quiz and weren’t significantly faster. The largest gap was in debugging. Three of six usage patterns, all involving active thinking, preserved learning. It’s an unreviewed study of 52 adults.

Should my kid use GitHub Copilot or ChatGPT for coding homework?

Check the class rules first. If allowed, set the tool to explain rather than generate, and have your kid write and debug the code themselves. Science fairs like ISEF allow AI-written code only with explicit citation of which parts AI wrote and a prompt log.

Why does debugging matter so much?

Debugging is how you prove you understand a system. Fixing a bug requires comparing your mental model to what really happened. In the study, the group without AI hit more errors and became better at debugging. Professional engineers spend much of their time debugging, and it’s the skill AI help most easily replaces.

How much AI is too much for a beginner?

A useful line: if AI is writing code your kid can’t explain, it’s too much. Beginners benefit most from asking concepts and fixing their own errors. As skills grow, teens can use AI to generate code, but only if they follow up until they understand every line.

---

About the author

Ricky Flores is the founder of HiWave Makers and an electrical engineer with 15+ years of experience building consumer technology at Apple, Samsung, and Texas Instruments. He writes about how kids learn to build, think, and create in a tech-saturated world. Read more at hiwavemakers.com.

---

Sources

1. Shen, J. H., & Tamkin, A. (2026). “How AI Impacts Skill Formation.” arXiv preprint, not peer-reviewed. https://arxiv.org/abs/2601.20245
2. Anthropic. (2026, January 29). “How AI assistance impacts the formation of coding skills.” https://www.anthropic.com/research/AI-assistance-coding-skills
3. McCauley, R., Fitzgerald, S., Lewandowski, G., Murphy, L., Simon, B., Thomas, L., & Zander, C. (2008). “Debugging: a review of the literature from an educational perspective.” Computer Science Education, 18(2), 67-92. https://doi.org/10.1080/08993400802114581
4. Fields, D. A., Kafai, Y. B., Morales-Navarro, L., & Walker, J. T. (2021). “Debugging by design: A constructionist approach to high school students’ crafting and coding of electronic textiles as failure artefacts.” British Journal of Educational Technology, 52(3), 1078-1092. https://doi.org/10.1111/bjet.13079
5. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). “Generative AI without guardrails can harm learning: Evidence from high school mathematics.” Proceedings of the National Academy of Sciences, 122(26). https://doi.org/10.1073/pnas.2422633122
6. Society for Science. (2025, October). “Use of generative AI to support a research project.” Regeneron ISEF. https://sspcdn.blob.core.windows.net/files/Documents/SEP/ISEF/2026/Rules/Generative-AI-Use-Table.pdf