What it is
A large language model’s “parameters” are the numerical weights it learned during training. In older, “dense” models like the original GPT-3, every one of those parameters is used to process every piece of text the model reads or generates. Total parameter count and computational cost moved together: a bigger model was a slower, more expensive one.
Mixture-of-experts (MoE) architectures break that link. Instead of one large network, an MoE model contains many smaller sub-networks, called “experts,” alongside a small router that decides, for each token, which handful of experts should process it. A model might store hundreds of experts — its total parameters — while activating only a handful of them, its active parameters, for any given token. Hugging Face’s technical primer on MoE architectures gives the illustrative case of Mixtral 8x7B: the model stores 47 billion parameters but computes with roughly 12 billion of them per token, because only two of its eight experts activate at a time. DeepSeek-V3 stores 671 billion parameters across 256 experts but activates about 37 billion per token — 5.5% of the total.
The practical consequence is that total parameters mainly govern how much memory a model needs to be loaded (VRAM), while active parameters mainly govern how much arithmetic — and therefore how much time and energy — it takes to generate each token. A model can have the stored “knowledge” of a very large dense model while running, in the best case, at something closer to the speed of a much smaller one. Epoch AI’s analysis of MoE inference economics adds an important caveat: the active-parameter metric is a reasonably good proxy for compute cost only when a system is arithmetic-bound (large batches, long context). At high decoding speeds with small batches, memory bandwidth and network communication dominate instead, and active parameter count becomes a much weaker predictor of real-world cost — a technical qualifier that gets dropped when the figure is used as a marketing headline.
Why it matters for AI governance and narratives
Active-parameter counts have become a competitive marketing metric in their own right, particularly among Chinese labs releasing open-weight models: a claim that a model with, say, 6 billion active parameters “outperforms” a much larger closed model is simultaneously a technical claim and a framing move, implying efficiency and value-for-compute that the raw benchmark score alone doesn’t convey. Because MoE model cards typically report both numbers, ecosystems can select whichever supports their narrative — total parameters to claim scale and capability, active parameters to claim efficiency and accessibility — often within the same release announcement.
The distinction also exposes a gap in current compute governance. The EU AI Act’s threshold for presumed “systemic risk” in general-purpose AI models is defined by cumulative training compute — 10^25 floating-point operations — not by parameter count of any kind, active or total, according to the European Commission’s official guidance on the Act’s general-purpose AI provisions. That means the efficiency gains MoE architectures advertise at inference time say little about whether a model crosses regulatory thresholds set at training time. A model boasting a low active-parameter count for inference efficiency can still have been produced by an enormous training compute budget, and vice versa. This mismatch between the metric labs promote publicly and the metric regulators actually use is itself a small but recurring feature of the AI governance information environment.
Key facts and dates
The mixture-of-experts concept dates to 1991 (“Adaptive Mixture of Local Experts”), but its large-scale application to language models began with Google’s Switch Transformer in January 2022 (1.6 trillion parameters) and became a mainstream industry pattern following Mistral AI’s release of Mixtral 8x7B in December 2023, per Hugging Face’s architecture history. Since then, MoE has become the default architecture for the largest frontier and open-weight releases from multiple labs, including DeepSeek, Alibaba’s Qwen series, and Zhipu’s GLM series, precisely because it allows total capacity to scale without a proportional increase in per-token compute cost.
Where to learn more
- Mixture of Experts Explained — Hugging Face — the standard technical reference for MoE architecture, terminology, and history.
- MoE vs. dense models: How do they compare in inference? — Epoch AI — analysis of when active-parameter counts do and don’t predict real inference cost.
- General-Purpose AI Models in the AI Act — Questions & Answers — European Commission — primary source on the EU’s training-compute-based (not parameter-based) regulatory threshold.
- What is mixture of experts? — IBM — accessible secondary overview for readers wanting a gentler introduction.