Mixture of Experts Explained: Why One AI Can Be Many

Mixture of experts explained in one sentence: it’s a model architecture where a large network is split into many smaller specialist sub-networks, and a routing layer sends each piece of text to only a few of them instead of running the whole thing. That’s why Gemini 3.7 Flash, announced in August 2026, launched at half the per-million-token price of the 3.6 Flash it replaced, three weeks after it. Not because Google got generous. Because the model only wakes up a fraction of itself for any given word.

If your kid’s school picked a cheaper AI tool this fall, this architecture is most of the reason it exists.

Key Takeaways

• A mixture-of-experts (MoE) layer contains many feed-forward sub-networks (“experts”) plus a small gating network that picks which few to use for each input.
• The 2017 paper that introduced the modern version (Shazeer et al., with Hinton, Le, and Dean as co-authors) reported over 1000x improvements in model capacity with only minor losses in computational efficiency, in models up to 137 billion parameters.
• The key distinction is total parameters versus active parameters. A model can “know” a lot while only “using” a little per token, and you pay for what’s active.
• Google’s August 2026 roundup described Gemini 3.7 Flash as its “most intelligent workhorse model yet for coding and agents,” at an introductory price of half the original 3.6 Flash cost per million tokens.
• MoE is why cheap tiers got good. It’s also why a cheap model can be strangely uneven: routing decisions are a source of failure that dense models don’t have.

What the architecture actually does

Start with the thing MoE replaced. A dense transformer runs every parameter for every token. If the model has 70 billion parameters, all 70 billion participate in predicting the next word, whether the word is part of a Python function or a poem about frogs. That’s simple, and it’s wasteful.

A mixture-of-experts layer changes one piece. Inside each layer where the dense model would have a single feed-forward network, MoE puts many of them side by side, and adds a small gating network whose only job is to decide which ones to use.

The 2017 paper that made this practical, “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer” by Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean, describes a layer with “up to thousands of feed-forward sub-networks” where a trainable gating network determines “a sparse combination of these experts to use for each example.” Their headline claim: “greater than 1000x improvements in model capacity with only minor losses in computational efficiency on modern GPU clusters,” demonstrated in models up to 137 billion parameters.

The word doing the work is sparse. The gate typically picks two experts out of dozens or hundreds. So a model with a trillion total parameters might activate only 30 billion of them for any given token. Total parameters determine how much the model can store. Active parameters determine what you pay and how fast it runs.

One thing to correct right away, because the name invites it: the “experts” are not specialists in human subject areas. There is no “math expert” and no “French expert.” What each expert learns is opaque, emerging from training, and often corresponds to patterns no human would name. The routing is learned, not designed.

Why this shows up in your family’s AI bill

The economics are the whole story. Because only a fraction of the network runs per token, an MoE model of a given quality costs far less to serve than a dense model of that quality. That cost saving flows straight into pricing tiers, and pricing tiers are what schools and families actually choose between.

Google’s August 2026 AI roundup put Gemini 3.7 Flash at “an introductory price of half the original 3.6 Flash cost per million tokens,” describing it as “our most intelligent workhorse model yet for coding and agents” and noting it shipped three weeks after 3.6 Flash. Release cycles that fast, at falling prices, are a signature of architectural efficiency rather than incremental training.

You can see the same pattern on the Anthropic side. Claude Haiku 4.5 runs at $1 per million input tokens against Claude Fable 5.1 at $10, per the published pricing. A tenfold price gap between tiers is not a marketing decision; it reflects a real difference in compute consumed per token. Our comparison of what the flash, mini, and nano tiers actually run maps the landscape.

For a school district deciding whether 680,000 students can have an AI tool, the difference between $1 and $10 per million tokens decides the answer. That’s not an abstraction. Utah’s statewide Gemini rollout covers roughly 680,000 students and 28,000 educators at no cost for 2026–27, and deployments at that scale only pencil out when inference is cheap. Google’s August 2026 Classroom expansion, which turned on Gemini for students of all ages by default, is the same economics applied at product scale.

Where MoE models get weird

Here’s the part that’s useful to a parent and almost never covered. MoE introduces a failure mode dense models don’t have: bad routing.

If the gate sends a token to the wrong experts, the model produces a worse answer than its parameter count would suggest. Several consequences follow.

Uneven performance. A cheap MoE model can be excellent on common tasks and surprisingly weak on unusual ones, because routing is learned from training data distribution. Rare phrasings get routed less reliably.

Load balancing artifacts. Training MoE models requires auxiliary losses to keep experts from being ignored or overloaded, a problem worked through in detail in Fedus, Zoph, and Shazeer’s Switch Transformer paper. These are engineering compromises, and they can leave odd gaps.

Harder to reason about. With a dense model, “the model doesn’t know this” is the explanation for a failure. With MoE, “the model knows it but didn’t route there” is also possible, and there’s no way for a user to tell the difference.

The practical upshot for a family: if a cheap model gives a weirdly bad answer on something it should handle, retrying with different phrasing is more likely to help than it would be with a dense model. That’s a genuinely useful tactic, and it comes directly from understanding the architecture.

How to Teach Your Kid About Mixture of Experts

The analogy that works is a hospital front desk, a phone tree, or a library reference desk. Someone routes you; you don’t see everyone.

Ages 5–8: The hospital front desk

Set up four “doctors” in your house (four chairs, four labels: bones, tummy, eyes, skin). You’re the front desk. Your kid describes a pretend problem, and you send them to one or two doctors, not all four. Ask: “Did all the doctors have to work?” No. Then say: “Computers that answer questions work like this. They have lots of little helpers inside and only wake up two of them for each word. That’s why they’re cheap and fast.” Do a round where you send them to the wrong doctor on purpose and let them notice the answer got worse.

Ages 9–12: Build the router

Write 20 questions on slips of paper across four topics. Make four “expert” envelopes. Your kid’s job is to be the gating network: read each slip and route it to the right envelope, fast, without reading the whole thing carefully. Time them. Then count the misroutes. Now tell them the real numbers: the 2017 paper used up to thousands of experts, and models today typically activate two per token out of dozens or hundreds. Ask what happens when the router is wrong, and connect it to the fact that a cheap AI model is sometimes oddly bad at one specific thing.

Ages 13+: Total vs. active parameters

Have your teen look up two model architectures (published MoE models list total and active parameters) and build a small table comparing total parameters, active parameters, and price per million tokens. Then have them calculate the ratio of active to total. Their finding: a model can be enormous in what it stores and modest in what it runs. Finish by having them read the abstract of the Shazeer et al. paper and find the phrase “sparse combination.”

The question to ask: “If a model has a thousand helpers inside but only uses two at a time, what could go wrong?”

Dense vs. mixture of experts: the honest comparison

Property	Dense model	Mixture of experts
Parameters used per token	All of them	A small fraction (often 2 experts out of dozens or hundreds)
Cost per token served	Scales with total size	Scales with active size, not total
How much it can store	Limited by what you can afford to run	Can be far larger than what runs per token
Training difficulty	Straightforward	Needs load balancing so experts aren’t ignored or swamped
Failure mode	”Doesn’t know it"	"Doesn’t know it” or “knew it but routed elsewhere”
Performance consistency	More uniform across task types	Can be uneven, especially on rare phrasings
Interpretability	Hard	Harder (routing decisions are opaque)
Typical 2026 use	High-end reasoning tiers	Cheap, fast workhorse tiers (Flash-class and similar)

The row that matters most for a family is the second-to-last one in the cost column. MoE is why your school can afford a decent AI tool for every student. The trade-off shows up as inconsistency rather than as being uniformly worse.

What to do at home

Rephrase before you conclude the model is dumb

If a cheap model fails on something simple, try saying it differently before assuming a capability limit. Different phrasing routes differently. This is a five-second experiment that saves a lot of frustration and teaches a real fact about the system.

Match the tier to the task on purpose

A Flash-class model is excellent for summarizing, formatting, extracting, and answering factual questions. A high-end reasoning tier is worth it for multi-step math, proofs, and long agent tasks. Teaching your kid to pick deliberately is the same skill as knowing when to think longer, covered in test-time compute explained.

Don’t read “experts” literally

If your kid thinks MoE means there’s a math expert and a history expert inside, correct it gently. The experts are learned patterns nobody named. Getting this right prevents a whole family of wrong intuitions about why models fail.

Watch the price per million tokens, not the model name

Model names change monthly. Price per million tokens is the number that tells you what tier you’re on and what your school could afford. It’s also the number that made this architecture worth inventing.

What not to do

Don’t assume cheaper means worse across the board. Google described a Flash-tier model as its most intelligent workhorse yet for coding and agents, at half the prior price. The honest framing is that MoE trades consistency for cost, and for most homework the trade is a good one. But don’t put a cheap tier on your kid’s hardest math either.

What to Watch For Over the Next 3 Months

• Week 4: Your kid can explain that a big model only uses a small part of itself for each word, and can name the router’s job.
• Month 2 red flags: They think “experts” means human subjects like math and history. Or they’ve concluded the cheap tier is useless after one bad answer.
• Month 3 self-check: Ask them to predict which of three tasks (summarize an article, solve a two-step algebra problem, write a formal proof) needs the expensive tier. Correct on the proof, with a reason, is a pass.

Frequently Asked Questions

Are the “experts” actually experts in subjects?

No. Each expert is a feed-forward sub-network whose specialization emerges from training and usually corresponds to statistical patterns no human would name. There is no math expert or French expert. The routing is learned, not hand-assigned.

Why is a mixture-of-experts model cheaper to run?

Because only a small fraction of its parameters activate for each token. You pay for compute actually used, not for total parameters stored. That’s the direct reason Flash-class tiers cost a fraction of high-end reasoning tiers.

Is Gemini 3.7 Flash a mixture-of-experts model?

Google’s August 2026 announcement describes it as its most intelligent workhorse model for coding and agents at half the prior tier’s per-token price, but does not publish the architecture. Sparse MoE is the standard approach for cheap, fast tiers across the industry; treat the specific architecture as unconfirmed unless the lab states it.

Does MoE make models worse at anything?

It makes them less consistent. Routing can send an input to suboptimal experts, which produces failures that a dense model of similar quality wouldn’t have. In practice that shows up as unexpected weakness on unusual phrasings, which is why rewording often helps.

Should I pay for a more expensive model for my kid?

Match tier to task. Cheap tiers handle summarizing, formatting, and factual questions well. Reserve the expensive tier for multi-step reasoning, proofs, and anything where a subtle error matters. Paying top rates for everything is the most common way families overspend on AI.

---

About the author

Ricky Flores is the founder of HiWave Makers and an electrical engineer with 15+ years of experience building consumer technology at Apple, Samsung, and Texas Instruments. He writes about how kids learn to build, think, and create in a tech-saturated world. Read more at hiwavemakers.com.

---

Sources

1. Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017). “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer.” arXiv:1701.06538. https://arxiv.org/abs/1701.06538
2. Google. (2026, August). “Google AI updates, August 2026.” The Keyword. https://blog.google/innovation-and-ai/technology/google-ai-updates-august-2026/
3. Anthropic. (2026). “Pricing.” Claude Platform Documentation. https://platform.claude.com/docs/en/about-claude/pricing
4. Fedus, W., Zoph, B., & Shazeer, N. (2022). “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.” Journal of Machine Learning Research, 23(120). https://arxiv.org/abs/2101.03961
5. Snell, C., Lee, J., Xu, K., & Kumar, A. (2024). “Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters.” arXiv:2408.03314. https://arxiv.org/abs/2408.03314
6. Google Workspace Updates. (2026, August 4). “Gemini in Google Classroom is expanding to users of all ages.” https://workspaceupdates.googleblog.com/2026/08/gemini-in-google-classroom-is-expanding-to-users-of-all-ages-with-contextualized-Gemini-starter-prompts-for-students.html
7. MarkTechPost. (2026, September 1). “Anthropic releases Claude Fable 5.1 and Claude Mythos 5.1.” https://www.marktechpost.com/2026/09/01/anthropic-releases-claude-fable-5-1-and-claude-mythos-5-1-52-6-on-terminal-bench-science-and-75-cheaper-cache-reads/