Table of Contents
Terminal-Bench-Science Explained: How AI Is Scored on Science
Terminal-Bench-Science explained for parents: 70 expert-written research tasks, what a 52.6% score means, and why that is not the same as doing science.
Here is a number worth understanding: 52.6%. That is what Anthropic reported for Claude Fable 5.1 on Terminal-Bench-Science 0.1 when the model launched September 1, 2026, up from 24.7% for Fable 5 and 29.0% for Opus 5. With Terminal-Bench-Science explained properly, that jump is interesting, but it is not “AI does science now.” It is a score on 70 specific tasks, graded by automated tests, in a terminal.
The difference between those two readings is the most useful thing a parent can teach a kid about AI progress.
Key Takeaways
- Terminal-Bench-Science 0.1 contains 70 expert-curated tasks across life, physical, Earth, mathematical, and engineering sciences, built by 376 contributors across 22 countries.
- Tasks run in containerized environments and are graded by programmatic, task-specific tests on concrete artifacts: analyses, simulations, proofs, code, and data products.
- Anthropic reported 52.6% for Claude Fable 5.1 on September 1, 2026, against 24.7% for Fable 5, 29.0% for Opus 5, and 22.4% for GPT-5.6 Sol; the public leaderboard’s harness settings differ, which is why its posted numbers are lower.
- A benchmark score is not a research result. The tasks have known answers and automated graders; real science has neither.
- Every number here is vendor-reported except the public leaderboard, and even independent leaderboards depend on which harness and effort setting was used.
Terminal-Bench-Science explained: what the benchmark actually is
A benchmark is a fixed set of tasks with automatic grading, used to compare AI systems on the same work. Terminal-Bench-Science is one of the more interesting ones because its tasks are real research chores rather than trivia.
The 0.1 release, published in August 2026, holds 70 tasks written and reviewed by domain experts across five broad areas: life sciences, physical sciences, Earth sciences, mathematical sciences, and engineering sciences. The published task types read like a graduate student’s week: scientific data analysis, statistical inference, simulation, optimization, theorem proving, image reconstruction, signal processing, inverse problems, sensor calibration, model fitting, classification, and scientific machine learning. The project reports 376 contributors from 22 countries, with affiliations including Stanford, the Laude Institute, Stanford HAI, and the Allen Institute for AI.
Two design choices make it credible. First, the agent works in a real containerized environment with a terminal, not a multiple-choice form; it has to actually run code. Second, grading is programmatic and task-specific rather than a language model judging another language model’s essay. A score means a test passed, not that an answer sounded good.
What 52.6% means, and the caveat about harnesses
Anthropic’s September 1, 2026 launch of Claude Fable 5.1 reported 52.6% on Terminal-Bench-Science 0.1, with Fable 5 at 24.7%, Opus 5 at 29.0%, and GPT-5.6 Sol at 22.4%. Roughly doubling the previous best in one model generation.
Now the caveat, because this is exactly where parents get misled. The public Terminal-Bench-Science leaderboard, as of its posted results, listed Claude Opus 5 with Claude Code at 30.0%, GPT-5.6 Sol with Codex at 22.4%, and Claude Fable 5 with Claude Code at 21.4%. Those are lower than Anthropic’s numbers for the same models because a benchmark score depends on the harness (the software scaffolding the model runs inside), the effort or reasoning setting, and how many attempts are allowed. VentureBeat’s coverage of the launch made the point directly: these are “vendor-reported results rather than independent proof of superiority,” and “production safeguards can affect scores.”
So the honest statement is: on this benchmark, in Anthropic’s harness at high effort, Fable 5.1 solved a bit more than half of 70 expert-written research tasks, which is roughly double what the previous generation managed. That is a real and large improvement. It is also a sentence with four qualifications in it, and the qualifications are the literacy lesson.
Where AI has actually produced science, and where it has not
Benchmarks are proxies. The record of real results is more mixed, and more interesting.
On the plus side, there are documented wins. A May 20, 2026 OpenAI result disproved a 1946 Erdős conjecture in discrete geometry with a 125-page proof, borrowing machinery from algebraic number theory; the same write-up noted the model was correct in only about 50% of repeated runs, and that earlier claims of “solving Erdős problems” had turned out to be rediscovery of existing literature. At Boston Children’s Hospital, OpenAI’s o3 Deep Research helped clarify 18 rare-disease diagnoses. In materials science, the GNoME project predicted 2.2 million structures, of which about 380,000 were flagged stable and 736 actually synthesized, a funnel worth staring at: 0.03% of predictions became real materials.
On the minus side, the failure modes are documented too. UC Berkeley Law banned AI for exams and credited coursework in May 2026 after hallucinated citations. And Anthropic’s own reporting on Project Glasswing, the security program using restricted Mythos models, showed partners found more than 10,000 high-severity vulnerabilities while critics counted only about 10% disclosed and under 1% fixed by September. Finding is easier than finishing.
Both columns matter. A kid who hears only the wins learns hype; a kid who hears only the failures learns cynicism. The actual skill is reading the qualifications.
How to Teach Your Kid About Benchmark Scores
Ages 5–8: The spelling test that is not spelling
Give your child ten words to spell, then tell them they scored 70%. Now ask: does 70% mean you know 70% of all the words in the world? They will say no, quickly. Explain that computers get tested the same way: someone picks the questions, and the score is about those questions only. That is the whole idea of a benchmark, and a six-year-old can hold it.
Ages 9–12: Build a benchmark and break it
Have your kid write five “tasks” to test a sibling or a parent, with an answer key. Then have them notice the two problems every benchmark has: some tasks are easier than others, and the person being tested might have seen the answers before. Try it twice, once cold and once after the test-taker has peeked. The score change is what researchers call contamination.
Ages 13+: Read the fine print on a real result
Give your teen the 52.6% figure and ask them to list everything they would need to know before trusting it. A good list includes: how many tasks, who wrote them, who graded, which harness, what effort setting, how many attempts, who published the number, and what the same model scored on an independent leaderboard. Then show them that the public leaderboard listed 21.4% and 30.0% for closely related configurations. Watching a teen work out why both numbers can be honest is worth more than a semester of “be skeptical of AI.”
The question to ask: “If a model scores 52.6% on 70 science tasks, what does the other 47.4% tell you, and what does the benchmark not measure at all?”
Reading a benchmark table without getting fooled
| Benchmark | What it tests | Claude Fable 5.1 (Anthropic-reported) | What it does not tell you |
|---|---|---|---|
| Terminal-Bench-Science 0.1 | 70 expert-written research tasks in a terminal | 52.6% | Whether the work would pass peer review |
| Terminal-Bench 4.0 | General agentic terminal tasks | 55.8% (Mythos 5.1: 60.9%) | How it handles an unfamiliar tool |
| CursorBench 3.2.0 | Real coding-assistant work | 73.4% | Code quality over months of maintenance |
| Humanity’s Last Exam | Hard expert questions across fields | 60.9% without tools, 65.0% with | Anything about doing research, only answering |
| AutomationBench | Workflow automation | 31.4% | Reliability on your specific workflow |
| OSWorld 2.0 | Operating a computer | 41.7% strict | Whether it can recover from a mistake it caused |
The right-hand column is the one to read with a teen. Every benchmark has a scope, and the scope is always narrower than the headline.
What this means for a kid who likes science
Two practical implications, and they point in opposite directions from the usual takes.
First, the tasks in this benchmark are the tasks that fill a working scientist’s day: calibrating a sensor, fitting a model, cleaning data, running a simulation. If AI agents get good at those, the human value shifts toward choosing which question to ask and deciding whether the answer is believable. That is an argument for teaching experimental design and error analysis, not an argument against science careers.
Second, the verification skill is now the scarce one. When the Erdős result landed, the newsworthy detail was not that a machine produced a proof; it was that mathematicians had to check a 125-page argument that succeeded in only half of runs. Kids who learn to verify, in math with proof assistants, in science with replication, are learning the part that is getting more valuable, not less. For an adjacent explainer, see how mathematicians verify an AI proof, and for the reliability angle, what a 50% success rate teaches kids.
The classroom-evidence picture remains what it was. The OECD’s PISA 2025 release on September 8, 2026 reported science scores stable between 2022 and 2025 while reading and math fell, and found students who used AI daily to draft written work scored 481 in science against 509 for those who never did. Benchmark scores rising and student scores not rising are both true at the same time.
What to Watch For Over the Next 3 Months
- Week 4: Next time your kid quotes an AI capability claim, ask for the benchmark name. If there is not one, that is information.
- Month 2 red flags: Treating a benchmark score as a measure of trustworthiness; assuming a model good at coding tasks is good at lab reasoning; quoting a vendor number as independent.
- Month 3 self-check: Terminal-Bench-Science is growing toward 100-plus tasks. When a new version ships, scores will shift and vendors will publish new highs. Ask your kid whether the model got better or the test changed.
Frequently Asked Questions
What is Terminal-Bench-Science?
A benchmark of 70 expert-curated scientific research tasks that AI agents perform in a containerized terminal environment, spanning life, physical, Earth, mathematical, and engineering sciences. Grading is programmatic and task-specific, checking concrete artifacts like analyses, simulations, proofs, and code.
What does 52.6% actually mean?
Anthropic reported that Claude Fable 5.1 completed 52.6% of Terminal-Bench-Science 0.1 tasks successfully in its own test harness, against 24.7% for Fable 5 and 29.0% for Opus 5. The public leaderboard lists lower figures for related configurations, because scores depend on harness and effort settings.
Does this mean AI can do science now?
No. It means AI agents can complete about half of a fixed set of well-specified research chores with known answers and automated graders. Real science involves choosing the question, designing the experiment, and judging whether a result is believable, none of which the benchmark measures.
Why do different sources report different scores for the same model?
Because a benchmark measures a model plus its harness, effort setting, and number of attempts. Anthropic reported 52.6% for Fable 5.1; the public leaderboard listed 21.4% for Fable 5 with Claude Code and 30.0% for Opus 5. Both can be accurate measurements of different configurations.
Should my kid still study science if AI keeps improving?
The benchmark tasks are the routine parts of research: data cleaning, model fitting, calibration. If those get automated, the human premium moves to question selection, experimental design, and verification. That is an argument for teaching those skills harder, not for skipping them.
Is there an independent leaderboard I can check?
Yes. Terminal-Bench publishes results at tbench.ai, and Snorkel AI maintains a leaderboard view. Read the harness column alongside the score; the configuration matters as much as the model name.
About the author
Ricky Flores is the founder of HiWave Makers and an electrical engineer with 15+ years of experience building consumer technology at Apple, Samsung, and Texas Instruments. He writes about how kids learn to build, think, and create in a tech-saturated world. Read more at hiwavemakers.com.
Sources
- Terminal-Bench. (2026, August). “Terminal-Bench-Science 0.1.” https://www.tbench.ai/news/terminal-bench-science-0-1
- VentureBeat. (2026, September 1). “Anthropic’s Claude Fable 5.1 and Mythos 5.1 arrive with a 75% cost reduction for Fable cache reads.” https://venturebeat.com/technology/anthropics-claude-fable-5-1-and-mythos-5-1-arrive-with-a-75-cost-reduction-for-fable-cache-reads
- Anthropic. (2026, September 1). “What’s new in Claude Fable 5.1.” Claude Platform Documentation. https://platform.claude.com/docs/en/models/fable-5-1/whats-new-fable-5-1
- OpenAI. (2026, May 20). “A model disproves a discrete geometry conjecture.” https://openai.com/index/model-disproves-discrete-geometry-conjecture/
- Quanta Magazine. (2026, August 3). “Why the legendary Erdős problems are falling to AI.” https://www.quantamagazine.org/why-the-legendary-erdos-problems-are-falling-to-ai-20260803/
- NBC News. (2026). “AI helped Boston Children’s Hospital diagnose rare diseases in kids.” https://www.nbcnews.com/tech/innovation/ai-boston-childrens-hospital-diagnose-rare-diseases-kids-openai-rcna350387
- OECD. (2026, September 8). “PISA 2025: Students’ reading and mathematics performance declined sharply across the OECD.” https://www.oecd.org/en/about/news/press-releases/2026/09/pisa-2025-students-reading-and-mathematics-performance-declined-sharply-across-the-oecd.html
- Anthropic. (2026, April 7). “Project Glasswing.” https://anthropic.com/glasswing
