Claude 3 arrives

The launch and the narrow documented claim

On 4 March 2024, Anthropic introduced the Claude 3 family: three models named Opus, Sonnet, and Haiku, released as one generation and ordered deliberately from the most capable and expensive to the fastest and cheapest. The narrow, verifiable claim is specific and modest. One provider shipped three models trained with the same generation of methods, differing mainly in size, price, latency, and measured capability, and made two of them, Opus and Sonnet, immediately available in the claude.ai product and the API, with Haiku to follow shortly. Opus was positioned as Anthropic's most capable model to date, Sonnet as the balanced default, and Haiku as the near-instant, low-cost option.

That is the event. The broader historical judgment, which the rest of this report tries to keep separate from the announcement copy, is why a tiered family, rather than any single benchmark score, was the consequential move. Anthropic reported strong results on standard evaluations, but a model maker's own benchmark numbers do not settle real-world performance, and this entry describes the March launch, not the many later updates to each model. Reading Claude 3 well means holding those two levels of claim apart.

From a single Claude to a generation

Anthropic had shipped Claude before, and the predecessor makes the design problem concrete. Claude 2, released in July 2023, was a single model with a 100,000-token context window that reported 71.2% on the HumanEval coding benchmark and 88.0% on the GSM8K grade-school math set, alongside a 76.5% score on the multiple-choice section of a bar exam. It was one model asked to serve every use, from a throwaway classification call to the analysis of a book-length document. That is the constraint Claude 3 was built to relax. A lone model forces a single operating point on the cost, speed, and quality curve onto every request, whether the task is a cheap autocomplete or a careful legal review, and it forces the provider to choose whether that one model should be optimized for depth or for throughput.

The way to relax the constraint was not a new architecture but a product and training decision: train several models of different sizes as one coordinated release, evaluate them against the same suite, and let the caller pick the point that fits the job. This is a familiar move in engineering, offering a range rather than a single artifact, but applying it to frontier language models at one moment, with shared branding and a shared evaluation table, is what made Claude 3 legible as a family rather than three unrelated products.

The tiered family as the actual innovation

The three names map to a single axis. Opus is the largest and most capable, Sonnet sits in the middle, and Haiku is the smallest and fastest. Anthropic priced them to match that ordering: at launch, Opus cost 15 dollars per million input tokens and 75 dollars per million output tokens, Sonnet 3 dollars and 15 dollars, and Haiku 0.25 dollars and 1.25 dollars. The output price therefore spanned a factor of sixty across the family, which is the whole point: a task that a small model can handle should not be charged at the rate of the most capable one.

Framing model selection as a cost, speed, and quality decision, rather than a binary of using the frontier model or not, is the structural contribution of the release. It reflects a maturing view of how these systems get deployed. In production, most requests are not the hardest requests, and paying frontier prices and latency for every call is wasteful. Anthropic also distributed the family beyond its own surface, making the models available through cloud platforms including Amazon Bedrock and Google Cloud's Vertex AI, which mattered for enterprises that consume models where their data already lives.

What the benchmarks actually reported

Anthropic accompanied the launch with a table of standard evaluations, and the headline figures belonged to Opus. It reported 86.8% on MMLU, the multitask knowledge benchmark measured five-shot, 95.0% on GSM8K with chain-of-thought prompting, 84.9% on HumanEval, and 60.1% on the MATH competition set. On GPQA, a deliberately Google-proof graduate-level science benchmark, Opus reported 50.4% on the hard Diamond subset in a zero-shot chain-of-thought setting, a result Anthropic emphasized because GPQA is designed so that non-experts with web access still score poorly. The company presented these as improvements over the figures reported for OpenAI's GPT-4 in its technical report and for Google's Gemini 1.0 Ultra, citing, for example, GPT-4 at 86.4% on MMLU and 35.7% on GPQA.

Several caveats belong right next to those numbers. They are self-reported by the model's maker, and the measurement conditions differ across labs: the number of examples shown, whether chain-of-thought is used, and whether multiple samples are aggregated all move a score by points, so a like-for-like comparison across three companies' tables is not guaranteed. A benchmark figure is a task-specific result under stated conditions, not a percentage of human ability or a general competence rating. Sonnet and Haiku sat lower on the same suite, as the tiering implies, with Sonnet near 79% and Haiku near 75% on MMLU. The honest reading is that Opus was competitive with the strongest models publicly available in early 2024 on these particular tests, which is a narrower statement than "the best model," and one that later independent use both partly supported and partly complicated.

Multimodal input and the 200,000-token window

Two capabilities distinguished Claude 3 from its immediate predecessor beyond raw scores. First, the models added vision: they could accept images, including photos, charts, graphs, and technical diagrams, and reason over them alongside text. Anthropic reported results on the multimodal MMMU benchmark, with Opus near 59% on its validation set, and described the vision capability as on par with other leading models rather than ahead of them. This mattered less as a benchmark milestone than as a practical one, because many enterprise documents are not clean text but scanned forms, slides, and screenshots.

Second, the family launched with a 200,000-token context window, double Claude 2's 100,000, and Anthropic noted that the underlying models could accept inputs exceeding one million tokens for select customers. On the needle-in-a-haystack retrieval test, which hides a target sentence inside a long body of unrelated text and asks the model to recover it, Anthropic reported that Opus exceeded 99% recall. A widely circulated anecdote from the same evaluation, in which Opus remarked that the planted sentence seemed out of place, was interesting but should be treated as an informal observation rather than a measured result about self-awareness. The durable point is that long, reliable recall over a large window changed which workflows, such as querying an entire codebase or a long contract in one pass, became practical.

How the models were shaped: Constitutional AI

The Claude line is trained with a method Anthropic has described publicly, which distinguishes it from competitors that rely more heavily on human preference labels. Constitutional AI, introduced in a 2022 paper, uses a written set of principles, a "constitution," and has the model critique and revise its own responses against those principles, generating AI feedback that supplements or partly replaces human labeling of harmful outputs. The stated aim is a harmless but non-evasive assistant that engages with a difficult query by explaining its objection rather than refusing flatly.

It is important not to overstate what a training method determines. Constitutional AI shapes the model's behavior and safety profile, but the capability gains reported for Claude 3 came primarily from a new generation of pretraining and scale, not from the alignment method alone. The two are separate levers: pretraining and scale largely set what the model can do, while the alignment procedure sets how it behaves when it does it. Conflating them produces the mistaken idea that a safety technique was responsible for the benchmark jump. The more defensible claim is that Anthropic paired a stronger base generation with its established behavioral training approach.

That approach fed a specific behavioral claim about Claude 3: the models were "significantly less likely to refuse to answer prompts that border on the system's guardrails than previous generations." This addressed a real and well-documented failure mode of earlier assistants, over-refusal, in which a model declines a benign request because it superficially resembles a prohibited one, such as refusing to explain a security concept or discuss a sensitive historical topic. Reducing that friction is a genuine usability improvement and a difficult calibration problem, because the same dial that lowers false refusals can, if turned too far, raise genuinely harmful compliance. The claim is best read as a reported direction of travel, not a solved problem. Refusal behavior is hard to measure in the open, depends heavily on the exact phrasing of a prompt, and trades off against safety in ways no single number captures. Anthropic framed the improvement as making Claude more useful while keeping its guardrails, but the balance between helpfulness and caution remained, and remains, an active and contested area rather than something Claude 3 finished.

The competitive field of early 2024

Claude 3 did not arrive into an empty field, and placing it among its contemporaries is essential to reading the event. The reference point was OpenAI's GPT-4, roughly a year old by March 2024 and still the model most others measured against. Google had released Gemini 1.0, including its Ultra tier, in December 2023 and February 2024, pursuing multimodality from the start as a competing family. On the open-weights side, the landscape was moving quickly: Meta released Llama 3 in April 2024, with 8-billion and 70-billion-parameter models and a 400-billion-plus model still training, and Mistral had shipped strong smaller and mixture-of-experts models that many teams could run themselves.

These were genuinely different bets, not just different scores. Google integrated multimodality and its own infrastructure tightly; Meta and Mistral bet on open weights that others could host, fine-tune, and inspect; OpenAI and Anthropic offered closed, hosted frontier models with safety framing and API access. Anthropic's distinctive positions in that mix were the explicit tiered family, the emphasis on long-context reliability, and the Constitutional AI training story. None of these choices was preordained to dominate, and evaluating Claude 3 against real, live competitors is more informative than treating it as a solitary leap.

Safety framing: ASL-2 and the scaling policy

Anthropic released Claude 3 under its own governance framework. The company had published its Responsible Scaling Policy in September 2023, which defines AI Safety Levels modeled loosely on biosafety standards, and it classified the Claude 3 models as ASL-2: systems that show early signs of dangerous capabilities but do not yet meaningfully raise catastrophic risk relative to existing tools. The policy commits the company to defined evaluations and stronger safeguards before deploying models that would cross into ASL-3.

Two honest observations follow. First, the classification is Anthropic's own, made against its own criteria, so it is a statement of the company's assessment and commitments rather than an external certification. Second, the value of such a framework is mostly prospective: it matters when a future model actually approaches a dangerous threshold and the pre-committed evaluations force a decision. For Claude 3 specifically, the safety framing is best understood as the release fitting comfortably within a policy designed to bind harder later, not as evidence that the model itself was near any dangerous frontier.

The downstream influence

The tiered family became a durable pattern, both for Anthropic and as an idea others echoed. Anthropic iterated fast on the same three-name structure. In June 2024 it released Claude 3.5 Sonnet, reporting that the mid-tier model now outperformed Claude 3 Opus on a range of evaluations while running at roughly twice Opus's speed and at Sonnet's lower price. That is a striking illustration of how quickly the frontier moved: within months, the balanced tier of the next generation reportedly surpassed the flagship of the previous one, which is exactly why a timeline entry should mark the March 2024 family launch as a starting point and not a settled ranking.

The deeper influence is conceptual. Offering a coordinated range of models at different capability and price points, evaluated on a shared table and selectable per request, became a standard way to ship frontier models, and the Claude 3 launch was one of the clearest early statements of it. The specific numbers dated quickly; the shape of the offering did not.

Honest limits and how to place the event

Several limits keep the event in proportion. The benchmark figures were self-reported and measured under conditions that differ from competitors', so the "state of the art" framing was provisional and partly commercial. Claude 3 introduced no new architecture; it was a strong generation of an existing approach, packaged well. The vision capability was described as on par with, not ahead of, rivals. And the whole family was superseded within a year by Anthropic's own faster iterations, which means any capability claim tied to March 2024 should be read as a snapshot.

So how should a reader place Claude 3 in the history of AI? Not as a scientific breakthrough, and not as the moment any particular benchmark was conquered, but as a clear instance of a product and deployment maturation: the point at which shipping frontier language models as a deliberate, priced range of capability tiers, with long context and multimodal input, became a normal and legible thing for a leading lab to do. The narrow claim is that Anthropic launched three coordinated models on 4 March 2024. The wider judgment is that the family format, more than any single score, is what the entry is really about, and that judgment is supported by how quickly both Anthropic and the field settled into the same pattern afterward.

References

• Anthropic (2024). Introducing the next generation of Claude.
• Anthropic (2024). The Claude 3 Model Family: Opus, Sonnet, Haiku.
• Anthropic (2023). Claude 2.
• Anthropic (2024). Introducing Claude 3.5 Sonnet.
• Bai et al. (2022). Constitutional AI: Harmlessness from AI Feedback.
• Anthropic (2023). Anthropic's Responsible Scaling Policy.
• OpenAI (2023). GPT-4 Technical Report.
• Gemini Team, Google (2023). Gemini: A Family of Highly Capable Multimodal Models.
• Meta AI (2024). Introducing Meta Llama 3.
• Hendrycks et al. (2020). Measuring Massive Multitask Language Understanding.
• Rein et al. (2023). GPQA: A Graduate-Level Google-Proof Q&A Benchmark.
• Cobbe et al. (2021). Training Verifiers to Solve Math Word Problems.
• Chen et al. (2021). Evaluating Large Language Models Trained on Code.
• Yue et al. (2023). MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark.