Qwen3: Qwen 3 broadens open reasoning

The event and the specific claim

On 29 April 2025, the Qwen team at Alibaba released Qwen3, a family of open-weight language models spanning six dense sizes and two mixture-of-experts (MoE) designs, and made the release notable for one design choice above the rest: a single model that can operate in a deliberate "thinking" mode that emits an extended chain of reasoning before answering, or a "non-thinking" mode that responds directly, with the switch exposed to the user. The official launch post describes the lineup, the dual-mode behavior, and Apache 2.0 licensing for the open weights, while the accompanying Qwen3 Technical Report documents the architecture, training procedure, and evaluations in detail.

The narrow, documented claim is precise and worth separating from the broad historical judgment. The narrow claim is that a well-resourced lab shipped downloadable reasoning models at many sizes, unified thinking and non-thinking behavior inside each model, and reported competitive results against both leading closed systems and the strongest open reasoning model of the moment. The broader claim, that this reshaped how reasoning-capable models are deployed, is an interpretation that only holds up when placed against what came before it and what followed. This report keeps those two levels of claim distinct.

The constraint it addressed: reasoning that was closed, and modes that were split

Two frictions defined the field Qwen3 entered. The first was access. When OpenAI introduced its o1 model in September 2024, described in Learning to reason with LLMs, the strongest demonstration that reinforcement learning could teach a model to spend inference-time compute on a chain of thought before answering arrived as a closed, hosted product. Practitioners could call it but could not download it, inspect it, fine-tune it, or run it on their own hardware. In January 2025 DeepSeek-R1, later published in Nature, changed that by showing reasoning behavior could be incentivized largely through reinforcement learning and by releasing open weights, but it did so at a single very large scale that most users could not host.

The second friction was structural: reasoning capability tended to live in a separate model from the everyday instruction-following one. A team wanting both a fast conversational assistant and a slow deliberate solver typically deployed two different checkpoints, such as Qwen's own instruct models alongside its QwQ reasoning model. That split doubled the operational burden and forced a routing decision outside the model. Qwen3's stated contribution is to collapse both frictions at once: to offer reasoning across a wide range of downloadable sizes, and to fold the deliberate and the direct behaviors into one set of weights whose mode the caller chooses at request time.

The lineup: from a phone-sized model to a 235-billion-parameter sparse network

Qwen3 shipped as a graded family rather than a single flagship. The dense models are Qwen3-0.6B, 1.7B, 4B, 8B, 14B, and 32B, sizes that run from a laptop or phone up to a single high-end accelerator. The two MoE models are the larger of the two contributions. Qwen3-30B-A3B has 30 billion total parameters but activates only about 3 billion for any given token, and the flagship Qwen3-235B-A22B has 235 billion total parameters and activates roughly 22 billion per token. The naming convention encodes the idea directly: "A22B" means twenty-two billion active parameters.

That sparsity is the mechanism that makes a 235-billion-parameter model economical to serve. A mixture-of-experts layer replaces one dense feed-forward block with many parallel "expert" blocks and a small router that sends each token to only a few of them, an idea developed in Outrageously Large Neural Networks and scaled by work such as Switch Transformers. The result is a model with the knowledge capacity of its full parameter count but the per-token compute cost of a much smaller one. Mistral's Mixtral of Experts had already popularized the pattern in the open-weight world, and DeepSeek's large MoE systems, including DeepSeek-V3, pushed it to frontier scale. Qwen3 adopts the same lever so that its top model can be broadly capable while activating less than a tenth of its parameters at inference. Most family members support a 128K-token context window, while the two smallest dense models are capped lower, so context length is a per-variant property rather than a family-wide guarantee.

How the unified thinking switch actually works

The distinctive engineering claim is that thinking and non-thinking behavior live in the same weights and are selectable per request. In thinking mode the model produces an internal reasoning trace, wrapped in dedicated markup, before its final answer; in non-thinking mode it skips the trace and replies directly, trading depth for latency and token cost. The launch post frames this as letting users "control how much thinking the model performs based on the task at hand," and exposes soft switches such as /think and /no_think that can flip the behavior turn by turn within a conversation, alongside a request-level flag.

Layered on top is a "thinking budget," a cap on how many tokens the model may spend reasoning before it must answer. The Qwen team reports that raising this budget produces, in their phrasing, "scalable and smooth performance improvements that are directly correlated with the computational reasoning budget allocated." This matters because it turns inference-time compute into a dial the operator can set per task rather than a fixed property of the model: a hard math problem can be given a large budget, a routine reply a small one. The reasoning trace itself is a descendant of chain-of-thought prompting, which showed that eliciting intermediate steps improves performance on multi-step problems; Qwen3's move is to make that behavior a trained-in, budgetable mode rather than a prompt trick.

The training recipe: a large pretraining corpus and a four-stage post-training

The launch post reports that Qwen3 was pretrained on approximately 36 trillion tokens, close to double the roughly 18 trillion used for its predecessor, and that its multilingual coverage was expanded to 119 languages and dialects, up from 29 in the Qwen2.5 Technical Report. These are scale figures reported by the developer, and the honest reading is that they describe the training budget, not a guaranteed uplift in any particular capability.

The more interesting part of the recipe is how the two modes were fused. The team describes a four-stage post-training pipeline: a long chain-of-thought cold start to seed reasoning behavior, a reasoning-focused reinforcement-learning stage to sharpen it, a "thinking mode fusion" stage that blends the reasoning model with a non-thinking instruction model so both behaviors coexist in one checkpoint, and a general reinforcement-learning stage across more than twenty domains to align broad behavior. The team also reports using its strongest models to distill capability into the smaller ones, which is the plausible route by which a small model can punch above its parameter count. The pipeline is the concrete answer to the question the unified switch raises: how do you get deliberate and direct behavior out of the same weights without one degrading the other.

The evidence, and what the numbers do and do not say

The strongest quantitative claims are comparative and size-relative. The Qwen team reports that the flagship Qwen3-235B-A22B "achieves competitive results" on coding, math, and general benchmarks against a specific set of contemporary systems: DeepSeek-R1, OpenAI's o1 and o3-mini, xAI's Grok-3, and Google's Gemini 2.5 Pro. Because that set mixes open and closed models, the claim is best read as "in the same competitive band" rather than "strictly superior," and the detailed per-benchmark figures live in the technical report rather than the announcement.

Two efficiency comparisons are more concrete and more revealing. The team states that Qwen3-30B-A3B "outcompetes QwQ-32B with 10 times of activated parameters," meaning the new sparse model matches or beats the prior reasoning model while doing a fraction of the per-token work. And it claims that "even a tiny model like Qwen3-4B can rival the performance of Qwen2.5-72B-Instruct," a striking generational jump attributable to the larger corpus and the distillation from stronger models. These are the kind of results that make the family, rather than any single model, the contribution.

Three cautions apply to all of these numbers. First, they are developer-reported at launch and were not, at that moment, independently reproduced. Second, benchmark scores are task-specific measurements on particular datasets under particular decoding settings; they are not measures of general intelligence, and a lead on one suite does not transfer automatically to another. Third, reasoning benchmarks such as competition math and code are especially vulnerable to training-data contamination, so a high score is evidence about a specific test, not proof of a capability in the wild.

The competitive field of early 2025

Qwen3 arrived in the middle of a fast, crowded contest over open reasoning models, which is the right frame for judging its significance. On the open-weight side, DeepSeek-R1 had demonstrated frontier-adjacent reasoning that anyone could download, and Meta's Llama 3 herd of models had established that a major lab would ship large, generally capable open weights with permissive terms. Mistral's Mixtral had shown that sparse MoE could be released openly and run efficiently. On the closed side sat o1 and o3-mini, Gemini 2.5 Pro, and Grok-3, hosted systems that set the capability ceiling but offered no weights.

Against that backdrop, Qwen3's differentiation was neither "first to reason" nor "first to open weights," both of which had already happened. It was breadth and packaging: a single coherent family that offered reasoning at eight scales, from models small enough for on-device use to a sparse flagship, all under a permissive license, all sharing the same mode-switching interface. The competing approaches of the era had each solved a piece: closed labs proved the reasoning frontier, DeepSeek proved reasoning could be opened, Mixtral proved sparse MoE could be shipped. Qwen3's bet was that combining wide size coverage, a unified thinking switch, and open weights into one release would matter more to practitioners than any single record score.

Downstream influence and the open-weight ecosystem

The clearest evidence of influence is adoption infrastructure. The launch guidance points to serving stacks such as SGLang and vLLM for deployment and to local runtimes including Ollama, LM Studio, MLX, and llama.cpp for individual use, and the weights and code were published through the Qwen3 GitHub repository and public model hubs. That the models were released under the Apache 2.0 license, which permits commercial use and modification, is the specific legal fact that lets companies build on them without negotiating terms, and it is a large part of why the Qwen family became a common base for downstream fine-tunes, quantizations, and derivative models.

The unified-mode idea also proved directionally influential rather than merely one lab's quirk. The pattern of a single model that can be asked to reason more or less, with inference-time compute as a tunable dial, aligns with a broader industry movement in 2025 toward "hybrid" or budget-controlled reasoning across both open and closed systems. It would overstate the evidence to attribute that movement to Qwen3 alone, since multiple labs converged on it. The honest claim is that Qwen3 was an early, widely-downloaded, open instantiation of the pattern, which made it easy for the rest of the ecosystem to build on and copy.

The honest limits and caveats

Several caveats keep the significance in proportion. The family is not uniform: capability, memory footprint, and hardware requirements vary enormously between Qwen3-0.6B and Qwen3-235B-A22B, so any statement about "Qwen3" should name the variant it means. Language coverage of 119 languages and dialects is a coverage claim, not a promise of equal quality across all of them; performance on lower-resource languages should be checked on the actual task. The competitive benchmark claims are self-reported and comparative, and the impressive small-model results depend on distillation from larger teachers, which is a legitimate technique but one that ties the small models' quality to the quality of the large ones.

There is also a subtlety in the word "open." Apache 2.0 covers the released weights and code, which is genuinely permissive, but open weights are not the same as a fully open training pipeline: the pretraining data composition and the full reinforcement-learning details are described at a high level rather than released for exact reproduction. That is the norm for the category, but it means "open" here refers to downloadable and commercially usable artifacts, not end-to-end reproducibility. Finally, extended thinking has a direct cost: the reasoning trace consumes tokens and latency, so the thinking budget is a genuine trade-off to manage, not free additional accuracy.

How to place Qwen3 in AI history

The most defensible placement treats Qwen3 as a consolidation event, not an invention. Chain-of-thought reasoning, reinforcement learning for reasoning, sparse mixture-of-experts, and permissive open-weight releases were all established before April 2025, each traceable to specific prior work cited above. Qwen3's documented contribution was to assemble those ingredients into one graded, openly licensed family with a unified, budgetable thinking switch, and to report that the resulting models were competitive with the strongest systems of their moment while remaining downloadable and runnable at many scales.

A reader navigating the timeline can locate Qwen3 with three questions. Which earlier idea did it inherit? Reasoning models, from o1 through DeepSeek-R1, and sparse MoE, from Shazeer's work through Mixtral and DeepSeek-V3. What did it recombine? Wide size coverage, open weights, and a single model that switches between deliberate and direct answering on demand. What did it leave unfinished? Independent verification of its benchmark claims, the cost and reliability of extended reasoning, and the gap between downloadable weights and a fully reproducible pipeline. Read that way, Qwen3 matters because it made capable open reasoning broadly deployable and gave the operator explicit control over how much the model thinks, not because it was the first to do any single one of those things.

References

• Qwen Team, Alibaba (2025). Qwen3: Think Deeper, Act Faster.
• Yang et al., Qwen Team (2025). Qwen3 Technical Report.
• DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.
• OpenAI (2024). Learning to reason with LLMs.
• Qwen Team, Alibaba (2024). Qwen2.5 Technical Report.
• Jiang et al., Mistral AI (2024). Mixtral of Experts.
• DeepSeek-AI (2024). DeepSeek-V3 Technical Report.
• Fedus, Zoph and Shazeer (2021). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.
• Shazeer et al. (2017). Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer.
• Wei et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.
• Grattafiori et al., Meta AI (2024). The Llama 3 Herd of Models.
• The Apache Software Foundation (2004). Apache License, Version 2.0.
• Qwen Team, Alibaba (2025). Qwen3 GitHub repository.