AI Hallucination Rates,
Statistics & Benchmarks in 2026

The Benchmark Problem: Why the Numbers Contradict Each Other

Before looking at any hallucination data, you need to understand why different benchmarks give wildly different scores for the same model.

Grok-3 scores 2.1% on the Vectara summarization benchmark. Excellent. That same model scores 94% on the Columbia Journalism Review citation accuracy test. Catastrophic. Same model, same time period, opposite conclusions.

This isn't an error. It's measuring different things. And treating any single benchmark as "the hallucination rate" will mislead you.

The matrix below summarizes what each benchmark actually tests. Click any benchmark name to jump to its dedicated section.

Vectara AI Hallucination Leaderboard (HHEM)

Vectara's leaderboard is the most cited hallucination benchmark in the industry. It measures summarization faithfulness: given a source document, does the model's summary stick to what's actually in the document, or does it add unsupported facts? This makes it a direct proxy for how AI behaves in RAG pipelines, enterprise search tools, and document analysis workflows. The leaderboard exists in two versions, and the gap between them tells an important story. ^[1]

Original Dataset: ~1,000 Documents (April 2025)

This is the dataset most articles reference when they quote hallucination rates. The documents are relatively short and the summarization tasks are straightforward.

On this dataset, the numbers look encouraging. Ant Group's finix_s1_32b and two Gemini 2.0 models take the top three spots, with OpenAI's o3-mini-high tied for third. OpenAI's GPT models in this table sit between 1.2% and 2.0%. Every model in this table stays under 15%, though the full archived board runs up to 29.9% for older small models.

Correction, August 2026: an earlier version of this page listed finix_s1_32b at 1.8% on the original dataset. That is its rate on the new 7,700-article dataset, where it also leads. On the archived original dataset it scores 0.6%, one tenth of a point below Gemini-2.0-Flash. It is the first time a Chinese enterprise model has topped either Vectara table. Vectara froze the original dataset in October 2025, so no model released after that date will appear here. ^[100]

But this dataset is easy. The documents are short, the summarization tasks are clean, and the real world is neither.

New Dataset: 7,700 Articles (November 2025 to October 2026)

Vectara launched a refreshed benchmark in late 2025 with longer documents (up to 32K tokens) spanning law, medicine, finance, technology, and education. This version better reflects what enterprise AI systems actually face.

Rates rose for most models:

October 2026 Updates: Gemini 4 Argon, Claude Opus 5.5 and DeepSeek V4.1 Flash

Vectara added Gemini 4 Argon on October 16, 2026 at 12.1%, then Claude Opus 5.5 at 9.9% and DeepSeek V4.1 Flash at 4.2% on October 22. Opus 5.5 is the first Claude Opus under 10% on the new dataset (Opus 4.7: 12.0%). Among Anthropic's models only Claude Haiku 4.5 reads lower there, at 9.8%. DeepSeek V4.1 Flash has the sixth-lowest rate of the 111 models on the board. Argon shows how far apart the two kinds of hallucination can sit: it has the lowest AA-Omniscience v1.1 hallucination rate of any public model, 4.3%, yet on Vectara it reads 12.1%, higher than Gemini 3.1 Pro Preview's 10.4%. Declining to guess on knowledge questions does not stop a model from adding unsupported detail to a summary. ^[1][2]

The September 22 update had added three OpenAI models: GPT-6 Sol at 6.5%, GPT-6 Astra at 8.7% and GPT-5.6 Sol at 12.4%. Both GPT-6 models score better than GPT-5.6 Sol, in line with their lower hallucination rates on AA-Omniscience. Claude Fable 5.1, Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 4 Carbon and every Grok released since May 2026 have no score yet, so the board's latest Grok is still Grok 4.1 Fast (19.2% with reasoning). Earlier snapshots are covered in our February 2026 research report. ^[1]

The Reasoning Tax

When Vectara launched the new dataset in November 2025, the thinking models it named, including GPT-5, Claude Sonnet 4.5 and Grok-4, all came in above 10%, and Gemini-3-Pro read 13.6%. Grok-4-fast-reasoning reads 20.2% on the current board. Several of OpenAI's newer models have since come in under 10% on the same dataset: GPT-6 Sol at 6.5%, GPT-5.4 at 7.0%, GPT-6 Astra at 8.7% and GPT-5.5 at 9.3%, though GPT-5.6 Sol reads 12.4%. Anthropic's newest entry, Claude Opus 5.5, is its first Opus under 10%, at 9.9%, just above Claude Haiku 4.5 (9.8%). Google's newest, Gemini 4 Argon, reads 12.1%, above Gemini 3.1 Pro Preview's 10.4%. ^{[102][1][48][49]}

The hypothesis is straightforward. Reasoning models invest computational effort into "thinking through" answers. During summarization, this thinking leads them to add inferences, draw connections, and generate insights that go beyond what's in the source document. That's helpful for analysis. It's hallucination on a summarization benchmark.

This creates a critical decision for enterprise teams: reasoning mode helps on open-ended tasks and hurts on grounded tasks. Knowing when to enable it and when to turn it off is not optional.

AA-Omniscience Benchmark (Artificial Analysis)

AA-Omniscience asks a fundamentally different question than Vectara. Instead of "can you summarize without adding stuff," it asks "when you don't know something, do you admit it or make something up?" ^[2]

Artificial Analysis moved the benchmark to version 1.1 on October 27, 2026. It removed saturated questions and its old public set, added new questions and revised ambiguous or erroneous ones, which leaves 3,000 questions in six domains, 300 of them public. Version 1 had 6,000 questions across 42 topics. The Omniscience Index (scale: -100 to +100) penalizes wrong answers and doesn't penalize refusal. This makes it the only major benchmark that explicitly rewards models for knowing their own limits. ^[2][124]

How to read the three numbers together. Accuracy is the share of questions answered correctly. Hallucination rate is the share of non-correct responses that were wrong answers rather than refusals. The Index is correct minus incorrect. So a model can post the highest accuracy while still fabricating most of the time it doesn't know. That is Claude Fable 5.1's profile: 68.4% accuracy and a 70.0% hallucination rate. On Artificial Analysis's v1.1 board on November 7, 2026, 72 of 124 models have a positive Index. The other 52 give more wrong answers than right ones. ^[2]

Snapshot policy. Artificial Analysis re-runs models and adds effort configurations over time, so figures drift after launch, usually by a point or two and sometimes by much more. The two tables below hold every model in our table that has a v1.1 record, read from Artificial Analysis's own model records on November 7, 2026. Its v1.1 board lists 124 models, most of them released from July 2026 on. Models it did not re-run keep their final v1 record, read October 4, 2026, in the v1 archive under the second table and in the cross-benchmark table, marked v1. Artificial Analysis reports that models score about 2 points lower on the v1.1 Index on average, with a median drop of 0.6 points. The question sets still differ, so compare a v1.1 figure only with another v1.1 figure. ^[2][124]

Models Ranked by Accuracy, With Hallucination Rates (AA-Omniscience v1.1, November 2026)

Hallucination Rates Ranked, Lowest to Highest (November 2026)

All 32 models in the cross-benchmark table that have an AA-Omniscience v1.1 hallucination rate, with accuracy alongside. A low rate on its own can mean a model declines most questions, so read the two columns together.

How the Leaders Changed, Newest First

The notes below run from the newest frontier releases back to Gemini 3 Pro in November 2025. Each one quotes the figures Artificial Analysis published at the time. The November note uses AA-Omniscience v1.1. The notes from September back use version 1, and where they mention a later record they mean the final v1 record, read October 4, 2026. The tables above carry v1.1 values.

Most of those re-scores happened on one day. On August 6, 2026, Artificial Analysis moved AA-Omniscience to a new grader, GPT-5.6 Luna in place of Gemini 3 Flash Preview, and said all its published results reflected the change. All 20 models in the chart below read higher in their final v1 record than when their first figure was published, by about 2 points at the median, and about three quarters of that movement came with the grader change. The three largest moves on that day were Anthropic models: Claude Opus 5 (+10.7 points), Claude Fable 5 (+8.7) and Claude Opus 4.7 (+6.1). Compare launch figures with launch figures, v1 records with v1 records, and v1.1 records with v1.1 records. ^{[2][122][123]}

November 2026: Version 1.1 and Gemini 4 Carbon

Artificial Analysis launched AA-Omniscience v1.1 on October 27, 2026, as part of its Intelligence Index v5, where the benchmark now carries 15% of the weight: 10% for accuracy and 5% for not hallucinating. It reports that models score about 2 points lower on the v1.1 Index on average. On the 19 models in our table that it re-ran, the hallucination rate fell for 12, rose for 4 and stayed within half a point for 3, a median move of 1.5 points down. The biggest moves: Gemini 4 Argon from 15.1% to 4.3%, Grok 4.7 from 29.3% to 20.0% and Claude Opus 5.5 from 58.6% to 50.3%. GLM-5.3 went the other way, from 29.6% to 38.9%. ^[2][124]

A day later Google announced Gemini 4 Carbon, open first to a subset of cyber defenders and then to more trusted partners. At high effort it takes the top Index on the v1.1 board, 53.5, 5.4 points clear of Claude Opus 5.5 (48.1), with 58.3% accuracy and an 11.4% hallucination rate. At medium effort it hallucinates 4.2% while answering 48.9% correctly, the lowest rate of any model that answers at least 45% of questions correctly, just ahead of Gemini 4 Argon (4.3% at 48.3%). Across their effort settings, Google's two Gemini 4 models hold the four best-calibrated spots among models that answer at least 45% correctly, and Argon, public since October 13, is the best public model on that measure. ^[2][125]

The other October releases spread across the board. GPT-6.1 Terra (October 26) reads 47.6% accuracy and 34.5% hallucination, the best calibration of any OpenAI model on v1.1. Kimi K3.1 Pro (October 23) reads 48.9% and 37.3%, and Claude Haiku 5.5 (October 7) 34.7% and 34.2%. Meta's Muse Wonder Full and Muse Wonder (October 12) sit with the cautious models at 17.6% and 20.9%, and Muse Spark Open (October 15) at 28.0%. At the far end, MiniMax-M3.1-Flash (October 19) and Tencent Hy4 (November 4) answer 45.1% and 50.0% of questions correctly and hallucinate 93.7% and 89.1% of the time they don't know. All figures in this subsection are v1.1 records read November 7, 2026. ^[2]

The September 2026 Releases Split the Field

September's frontier releases moved in different directions on AA-Omniscience. GPT-6 Astra (September 3) shows the biggest drop against a direct predecessor in this data: 62.6% accuracy and 51.3% hallucination at max, against 59.4% and 92.2% for GPT-5.6 Sol. OpenAI's flagship went from the highest flagship hallucination rate among the four US frontier labs to below both new Claude flagships, and gained accuracy doing it. Grok 4.7 (September 21) cut xAI's rate for the second release in a row, to 29.3% at 47.4% accuracy. Gemini 4 Argon, announced September 30 for an initial group of cyber defenders and public since October 13, posted the best calibration of any high-accuracy model in the v1 data: 15.1% hallucination at 49.9% accuracy, Index 42.4. ^[2]

Anthropic's top models mostly kept their answer-first profile. Claude Opus 5.5 (September 22) took the highest v1 Index in the table, 46.4, on 66.2% accuracy, with a 58.6% hallucination rate, slightly below Opus 5's 60.8%. Claude Fable 5.1 (September 1) went the other way: the highest v1 accuracy in the table, 67.2%, and a 72.6% hallucination rate, 9 points above Fable 5. The Index still rewards Anthropic's knowledge. On calibration alone, OpenAI, Google and xAI each fielded a model by early October that hallucinates less than Claude Opus 5.5 or Fable 5.1 while answering at least 47% of questions correctly. All figures in this subsection are Artificial Analysis v1 records read on October 4, 2026. ^[2]

The other September releases mostly followed their labs' direction. GPT-6.1 Sol (September 29) improved on GPT-6 Sol (September 22) on both counts a week later: 62.1% accuracy and 54.3% hallucination, against 54.5% and 60.1%. Claude Sonnet 5.5 (September 28) lifted accuracy to 53.9% from Sonnet 5's 40.1%, and its hallucination rate rose from 39.4% to 47.0%. Gemini 3.8 Flash (September 2) cut the Flash line's rate to 55.2% from Gemini 3.7 Flash's 64.5% at almost the same accuracy, 54.6% against 55.3%. Muse Spark 1.3 and Qwen3.8 Max (0902), also out on September 2, stayed with the cautious models at 32.9% and 28.8%. ^[2]

Grok 4.6 Breaks the 2026 Pattern (August 2026)

Grok 4.6 shipped on August 12, 2026. On AA-Omniscience it posted 48.2% accuracy and a 34.3% hallucination rate, for an Index of 30.5, fourth on the board at the time behind Fable 5, Opus 5 and Gemini 3.1 Pro. Against Grok 4.5 (52% accuracy and 54% hallucination at launch, 51.6% and 54.1% in its final v1 record) that was a 20-point drop in hallucination for less than four points of accuracy. It was one of the few 2026 flagship releases to cut hallucination instead of buying accuracy with it, and Grok 4.7 repeated the move in September (29.3%). Its final v1 record read 48.2%, 34.3% and 30.5. Artificial Analysis has not re-run the high setting on v1.1, where the xhigh setting reads 46.2% accuracy and 39.7% hallucination. ^[86][88][2]

One oddity worth knowing about. xAI's own model card reports the opposite direction on its internal factuality evaluation: Grok 4.6 hallucinates 1.7% at high effort where Grok 4.5 scored 0.98%, and its MASK-Rectified dishonesty score rose from 0.67% to 1.90%. Internal and independent benchmarks disagree not just on magnitude but on which way the model moved. The internal test is a different task on a different scale, so neither number is wrong. It is a reminder that "hallucination rate" without a benchmark name means nothing. ^[85]

Claude Opus 5 Moves the Opus Line (July 2026)

Claude Opus 5 launched on July 24, 2026 at $5 / $25 per million tokens with a 1M-token context window and four effort levels, and at launch posted 54% accuracy on AA-Omniscience, 7 points above Opus 4.8, for an Index of 31, third behind Fable 5 and Gemini 3.1 Pro. After Artificial Analysis's August 6 grader change its v1 record read 61% accuracy and Index 37, second only to Fable 5 at the time. On v1.1 it reads 60.9% accuracy and Index 37.6. Its Intelligence Index score of 63 was the highest of any Anthropic model at launch, one point above Fable 5. ^{[83][84][99][123]}

The calibration price was steep. At launch Artificial Analysis put Opus 5's hallucination rate at 50%, 14 points above Opus 4.8's 35.9%. After the August 6 grader change the two read 60.8% and 39.3%. On v1.1 Opus 5 reads 59.4%, and Opus 4.8 has no v1.1 record. Anthropic's launch post reports accuracy gains in chemistry, biology, finance and law, and says nothing about hallucination. ^[2][122][83]

So the Opus line split into two profiles. Opus 4.7 and 4.8 refuse rather than guess. Opus 5 answers like Fable 5, and Opus 5.5 kept that profile in September at 58.6% on v1. On v1, Opus 4.8 (39.3%) and Sonnet 5 (39.4%) had the lowest rates of any Claude apart from Haiku 4.5, which answers only 14.4% correctly. Neither has a v1.1 record. On v1.1 the best-calibrated current Claude is Claude Haiku 5.5 at 34.2%, then Sonnet 5.5 at 43.3%, against 50.3% for Opus 5.5 and 70.0% for Fable 5.1.

The GPT-5.6 Sol Data Point (July 2026, Rate Published August 2026)

GPT-5.6 Sol, the flagship tier of OpenAI's Sol / Terra / Luna family (general availability July 9, 2026), posts an AA-Omniscience accuracy of 59.4% in its max configuration, third at the time behind Fable 5 and Opus 5. At launch Artificial Analysis described it only as "a small accuracy uplift coupled with an increase in hallucination rate" versus GPT-5.5, and this page withheld a number. The AA-synced board added one in August: 92.2% hallucination, Omniscience Index 22.0. When Sol doesn't know, it invents an answer more than nine times out of ten. That was the highest hallucination rate of any flagship from the four US frontier labs, above GPT-5.5's 86% at launch (89.0% in its final v1 record), and it left Sol outside the top ten on the Index despite third-place accuracy. GPT-6 Astra succeeded it in September at 51.3%. ^[73][88][2]

The two cheaper tiers followed the same shape. GPT-5.6 Terra: 46.8% accuracy, 87.9% hallucination, Index 0.1. GPT-5.6 Luna: 42.7% accuracy, 92.6% hallucination, Index -10.3. Across all three, OpenAI's mid-2026 models answered almost everything and admitted almost nothing, the pattern GPT-6 Astra broke in September. ^[88]

Claude Fable 5 Resets the Ceiling (June 2026, Re-Scored August 2026)

Claude Fable 5, Anthropic's first public Mythos-class model, launched on June 9, 2026 and immediately took the top of the AA-Omniscience Index at 40, seven points clear of Gemini 3.1 Pro, with a record 61% accuracy. Artificial Analysis tested it with adaptive reasoning, max effort and Opus 4.8 as the fallback model, and its numbers moved up when AA re-graded every model with a new grader on August 6, 2026: Index 43, 65% accuracy, ten points clear of Gemini 3.1 Pro and six clear of the new Claude Opus 5 at the time. Artificial Analysis's records still read 43 and 65% on October 4, 2026. In September, Claude Fable 5.1 (43.5), GPT-6 Astra (43.4) and Claude Opus 5.5 (46.4) passed it on the Index. ^{[72][2][122][88]}

Here is the twist that matters, and the re-score made it sharper. Fable 5's Index lead was driven by accuracy, not by calibration. Its hallucination rate is 63.6%, up from 54.9% at launch, far above Claude Opus 4.8's 39.3% and nowhere near the abstention-first models. It could still top the Index because the Index rewards correct answers and penalizes incorrect ones while ignoring refusals, and Fable 5's 65% correct-answer base is large enough to outweigh the penalty. The 63.6% applies only to its non-correct answers, not to every question. Roughly: it gets 65 of 100 questions right, and of the 35 it gets wrong or skips, it guesses wrong on 22 and admits it doesn't know on 13.

This is a real strategic shift for Anthropic, and it breaks the tidy "Claude refuses rather than guesses" story that held through Claude 4.1 Opus and Opus 4.7/4.8. That story still holds for Opus 4.7 and 4.8. It is not true of Fable 5, and, as the Opus 5 notes above show, it is not true of Opus 5 either. (A note on one stray figure: a 36.18% hallucination rate has circulated for Fable 5. That number belongs to Claude Opus 4.7, misattributed. Artificial Analysis's figure for Fable 5 is 63.6% as of October 2026, 54.9% at launch.)

The GPT-5.5 Data Point (April 2026)

GPT-5.5, released by OpenAI on April 23, 2026, posted the highest accuracy recorded on AA-Omniscience up to then, 57%. It also posted an 86% hallucination rate on the same benchmark, the most extreme accuracy-versus-calibration gap observed at the time: when GPT-5.5 didn't know an answer, it fabricated one 86% of the time. Its final v1 record read 58.0% accuracy and 89.0% hallucination. The Gemini 3 Pro pattern (knowledge without self-awareness) intensified with that generation of high-capability models. ^[2][63]

Claude Opus 4.7, released by Anthropic on April 16, 2026, took the opposite tradeoff: a 36% hallucination rate at launch (42.3% in its final v1 record), with somewhat lower raw accuracy. The two release decisions, a week apart, were the clearest split up to then between prioritizing what a model knows and what a model knows about its own limits. ^[58][63][2]

Anthropic held that line with its next release. Claude Opus 4.8, which became the new flagship on May 28, 2026, launched with a 35.9% hallucination rate, 46.6% accuracy and an Omniscience Index of 27, at the time second only to Gemini 3.1 Pro. Its final v1 record read 39.3%, 48.8% and 28.8. The numbers barely moved from Opus 4.7, which is the point: a full flagship generation passed with calibration held steady rather than traded away for raw knowledge. Early testers described the model as more likely to flag uncertainty and less likely to make unsupported claims. It was the last Opus to make that trade. ^[64][65][2]

The Gemini 3 Pro Paradox (November 2025)

Gemini 3 Pro told the first version of this story. At launch it posted the highest accuracy on the board, 53%, 14 points ahead of the next model. It also showed an 88% hallucination rate: when it didn't know an answer, it fabricated one 88% of the time rather than admitting uncertainty. Its final v1 record read 55.8% accuracy and 91.5% hallucination. ^[2]

High knowledge + low self-awareness = a model that's brilliant when it's right and dangerous when it's wrong.

The Gemini 3.1 Pro update partially addressed this. At launch, Artificial Analysis measured its hallucination rate at 50%, down from 88% for Gemini 3 Pro, at about the same accuracy, because the model guessed less often when it did not know. Its Omniscience Index rose 17 points, the highest of any model until Claude Fable 5 arrived at 40 in June, since re-scored to 43. In their final v1 records the two read 91.5% and 50.9% hallucination, with Index values of 15.3 and 31.9. This proved that dramatic hallucination reduction is possible without meaningful accuracy sacrifice. ^[15][2]

Domain-Specific Leaders

No single model dominated all knowledge areas in Artificial Analysis's November 2025 launch analysis:

Source: Artificial Analysis AA-Omniscience, launch analysis, November 2025. Domain leaders have not been re-published for the 2026 flagships. ^[2]

Claude models led in domains where precise reasoning and citation accuracy matter. Grok led in domains where broad knowledge coverage matters. GPT led in business applications. This fragmentation is itself data. It means no single model is the safest choice for every professional use case.

One Statistic That Matters More Than the Rest

In Artificial Analysis's launch analysis, accuracy tracked model size. Hallucination rate did not. The 2026 releases have not broken that finding. The three most accurate models in our table, Claude Fable 5.1 (68.4%), Claude Opus 5.5 (65.5%) and GPT-6.1 Sol (61.8%), have AA-Omniscience v1.1 hallucination rates of 70.0%, 50.3% and 51.9%: each still gives a wrong answer more often than it declines. Google's Gemini 4 models are the exception: Gemini 4 Carbon answers 58.3% of questions correctly with an 11.4% hallucination rate, though it is not yet publicly available, and Gemini 4 Argon 48.3% with 4.3%.

Throwing more parameters at the problem increases knowledge without increasing self-awareness. This is why the hallucination problem won't simply disappear with the next model generation.

FACTS Benchmark (Google DeepMind)

Google DeepMind's FACTS benchmark suite, published in December 2025, takes a different approach to most evaluations: instead of producing one hallucination score, it breaks factuality into four distinct dimensions and reports the overall as an average across them. This multi-dimensional view exposes that models have dramatically different strengths depending on task type. Grok 4 scores 75.3 on Search but just 25.7 on Multimodal, a 50-point gap within the same model. The figures here are the December 2025 Suite launch snapshot. The overall is the four-slice average (Gemini 3 Pro's 68.8 is the mean of its four slice scores, GPT-5's 61.8 the mean of its four). ^[3]

What the Four Slices Measure

Grounding: Can the model faithfully use information from provided documents? Tested through summarization and extraction tasks with source material.

Multimodal: Can the model accurately describe and reason about visual content alongside text?

Parametric: Does the model's internal knowledge (stored in its weights from training) produce correct answers without external tools?

Search: How accurate is the model when it has access to web search and retrieval tools?

Model Scores Across All Four Slices

The 2026 Kaggle Board (Updated November 5, 2026)

Google DeepMind now runs FACTS as a public leaderboard on Kaggle, and that is where the 2026 models are scored. The board's average covers only the slices a model has been scored on, and 10 of the 18 models listed on November 5, 2026 have fewer than four. GPT-5.6 Sol tops the average column at 81.1 on a Parametric score alone, so the table below ranks only the eight models scored on all four slices. GPT-6 Astra leads at 78.1, ahead of GPT-6.1 Sol at 76.8 and Gemini 4 Argon at 72.4. Claude Opus 5.5 is the best Anthropic model at 66.5. None of the December 2025 launch-table models is on the current board. When we read it on October 7, the ones still listed scored lower there than in the launch table, for example GPT-5 at 56.5 against 61.8 and Claude Opus 4.5 at 43.9 against 51.3. The search slice uses a newer version of FACTS Search, so treat the 2026 board as a separate scale. ^[3]

What This Data Reveals

No model broke 70% at launch. The best score in the December 2025 table was Gemini 3 Pro's 68.8, so every model was wrong more than 30% of the time on this multi-dimensional factuality evaluation. On the 2026 board GPT-6 Astra reaches 78.1, on a scale that is not directly comparable.

Search is the strongest slice for everyone. Gemini 3 Pro hits 83.8 and GPT-5 hits 77.7 on search-enabled factuality. When models can look things up, they're materially more accurate. When they rely on stored knowledge alone, accuracy drops. This matches the browse-on vs browse-off findings from OpenAI's system cards. It still holds on the 2026 board: all eight fully scored models do best on Search, from 80.3 to 93.3. The gap to Parametric is widest for Claude Haiku 5.5, 80.3 against 13.1.

Grok 4 has a 50-point internal gap. It scores 75.3 on Search but 25.7 on Multimodal, a massive inconsistency that means it can find facts well but struggles with visual content. Any evaluation that averages these into a single score hides this gap.

Gemini 3 Pro's improvement is real. Compared to Gemini 2.5 Pro, Gemini 3 Pro reduced error rates by 55% on the Search slice and 35% on the Parametric slice. That's a large generation-over-generation improvement in factual accuracy, driven primarily by better search and grounding capabilities.

Frontier Model Hallucination Profiles

Every model below is profiled across multiple benchmarks. Single-benchmark comparisons mislead. The profiles show where each model is reliable and where it isn't.

Models added in the November 2026 update come first in each family below. AA-Omniscience figures are v1.1 records, read from Artificial Analysis on November 7, 2026, for every model it has re-run on v1.1. For the others, the figure is the final v1 record, read October 4, 2026, and is labeled that way. Launch figures stay where the story depends on them, so you can see how a model's record moved after release.

To see any two of these models side by side on every benchmark, use the head-to-head comparison further down the page.

GPT Family (OpenAI)

GPT-6.1 Terra (October 26, 2026): the mid tier of the GPT-6.1 generation, below Sol and Astra on price. AA-Omniscience v1.1 (max): 47.6% accuracy, 34.5% hallucination, Index 29.6, the best calibration of any OpenAI model on v1.1. At high effort it reads 47.3% / 37.0% / 27.8. Its predecessor, GPT-5.6 Terra, reads 42.3% accuracy and 89.4% hallucination on v1.1, so the step is 54.9 points less hallucination for 5.3 points more accuracy. OpenAI reports the same direction on its own test of error-inducing prompts: at low reasoning effort the error rate fell from 53.6% for GPT-5.6 Terra to 16.7%, and at maximum effort to 4.3%. No Vectara, FACTS or SimpleQA Verified score yet. ^[2][128]

GPT-6 Astra (September 3, 2026): the successor to GPT-5.6 Sol, and the biggest drop against a direct predecessor in the v1 data, from 92.2% to 51.3%. AA-Omniscience v1.1 (max): 61.6% accuracy, 49.8% hallucination, Index 42.4, the top OpenAI model on the v1.1 Index and level with Claude Opus 5.5 (50.3%) on calibration. The high setting does better on v1.1: 61.1% accuracy, 42.5% hallucination, Index 44.6. Vectara new dataset: 8.7%, down from 12.4% for GPT-5.6 Sol. It leads the models scored on all four FACTS slices at 78.1 and leads HalluHard with web search at 27.2% (updated question set), and places second on SimpleQA Verified at 75.8%. ^{[2][1][3][5][101]}

GPT-6 Sol, GPT-6.1 Sol and GPT-6 Luna: GPT-6 Sol (September 22) reads 51.1% accuracy, 60.8% hallucination and Index 21.4 at max on v1.1, with 6.5% on Vectara's new dataset, lower than Astra's 8.7%. GPT-6.1 Sol followed a week later (September 29) and improved on both counts: 61.8% accuracy, the highest of any OpenAI model on v1.1, 51.9% hallucination, Index 42.0. GPT-6 Luna reads 39.6% / 79.3% / -8.3. GPT-5.6 Terra and Luna read 89.4% and 92.3% on v1.1, so every GPT-6 tier with a v1.1 counterpart is better calibrated than the GPT-5.6 tier it replaced. On the November FACTS board GPT-6.1 Sol averages 76.8 across all four slices and GPT-6 Luna 64.8. GPT-6 Sol has only a Parametric score there (76.1). On SimpleQA Verified GPT-6.1 Sol scores 73.8% and GPT-6 Sol 64.4%. ^{[2][1][3][101]}

GPT-5.6 Sol / Terra / Luna (July 9, 2026): the generation before GPT-6, split into three tiers: Sol (flagship), Terra (balanced), Luna (fast, cost-efficient). Sol max posted 59.4% AA-Omniscience accuracy, third-highest on the August 2026 board, behind Claude Fable 5 and Claude Opus 5. Its hallucination rate, withheld at launch, was 92.2% in its final v1 record, Index 22.0, the highest hallucination rate of any OpenAI flagship, 3.2 points worse than GPT-5.5 (89.0%). Artificial Analysis has not re-run Sol on v1.1. Terra and Luna were re-run: on v1.1 Terra reads 42.3% accuracy / 89.4% hallucination / Index -9.3 and Luna 39.1% / 92.3% / -17.1 (46.8% / 87.9% / 0.1 and 42.7% / 92.6% / -10.3 on v1). Sol scores 12.4% on Vectara's new dataset and 69.2% on SimpleQA Verified, where Terra reads 41.5% and Luna 39.1%. One more reliability signal: independent evaluator METR recorded the highest reward-hacking (task-cheating) rate of any public model it had tested on Sol, fabricating results to appear finished. That is distinct from classic hallucination but shares the same root: completing over admitting limits. Use with strong verification for high-stakes factual work. ^{[73][2][1][101]}

GPT-5.5 (April 23, 2026): 57% AA-Omniscience accuracy paired with an 86% hallucination rate at launch, 58.0% and 89.0% in its final v1 record. The clearest illustration of high knowledge with weak calibration until Sol extended the pattern. It scored 66.7 on the FACTS board in September, before the board dropped it, and reads 9.3% on Vectara's new dataset. ^{[63][2][3][1]}

GPT-5.3 Instant (March 2026): reduces hallucination by 26.8% with web search and 19.7% without, relative to prior models, by OpenAI's own measure. ^[10]

GPT-5.2 (December 11, 2025): the professional workhorse. AA-Omniscience accuracy: 43.8% when first published here, 44.3% in its final v1 record (xhigh), with an 81.2% hallucination rate. With web search: 93.9% error-free responses. Without: error rate jumps to 12%. HalluHard: 38.2% with web search. Vectara new dataset: 10.8% (high). FACTS: 48.5 on the 2026 Kaggle board in September, which no longer lists it (GPT-5.2 was not in the December 2025 launch table). ^{[9][2][5][1][3]}

GPT-5 (August 7, 2025): Vectara old dataset: 1.4% (strong). Vectara new dataset: 15.1% at the high setting (weak). HealthBench thinking mode: 1.6%, one of the best medical hallucination scores recorded. SimpleQA without web: 47%. On browsing-enabled ChatGPT production traffic, 9.6% of factual claims from gpt-5-main contained an error (a claim-level production metric, not directly comparable to SimpleQA). AA-Omniscience final v1 record: 40.3% accuracy, 82.2% hallucination (high). FACTS overall: 61.8 in the December 2025 launch table, and 56.5 on the 2026 Kaggle board in September, which no longer lists it. ^{[8][12][100][1][2][3]}

The pattern across the GPT-5 family: web search access is the single biggest variable. With browsing enabled, GPT-5 models compete for the lowest hallucination rates in the industry. Without it, rates jump several times over: GPT-5 thinking's FActScore error rate is 1.0% with browsing and 3.7% without. If you're deploying a GPT-5 variant, keep web access on. The GPT-5.5 and 5.6 generation shows the other half of the pattern: raw accuracy climbed to record highs while calibration got worse, not better. All three GPT-5.6 tiers sat above 87% hallucination on AA-Omniscience v1. GPT-6 reversed the trend: every GPT-6 model hallucinates less than its GPT-5.6 counterpart, GPT-6 Astra gained accuracy at the same time, and GPT-6.1 Terra, at 34.5% on v1.1, is now the best-calibrated OpenAI model.

Claude Family (Anthropic)

Claude Haiku 5.5 (October 7, 2026): Anthropic's small, fast tier. AA-Omniscience v1.1 (max): 34.7% accuracy, 34.2% hallucination, Index 12.3, the lowest hallucination rate of any current Claude on v1.1. Only the deprecated Claude 4.5 Haiku in its reasoning setting reads lower there, 23.4% at 17.5% accuracy. Lower effort settings answer fewer questions correctly and guess slightly more often (38.9% at high effort). Anthropic's own system card is less flattering: on its internal tests Haiku 5.5 "hallucinated more than other recent models, and about as much as Claude Haiku 4.5". The two do not conflict. AA-Omniscience's rate counts wrong answers only among the questions a model gets wrong or declines, and Haiku 5.5 declines often. On the FACTS board it averages 47.3 across all four slices, with 13.1 on Parametric, the stored-knowledge slice, the lowest of any fully scored model. SimpleQA Verified: 23.9%. No Vectara score yet. ^{[2][129][3][101]}

Claude Opus 5.5 (September 22, 2026): AA-Omniscience v1.1 (max run with the default fallback model): 65.5% accuracy, 50.3% hallucination, Index 48.1, second on the v1.1 board behind Gemini 4 Carbon (53.5) and the top public model. The move to v1.1 cut its hallucination rate from 58.6% to 50.3%. Against Opus 5 on v1.1 that is 4.6 points more accuracy and about 9 points less hallucination: a real step toward better calibration, still inside the answer-first profile. On the FACTS board it averages 66.5 across all four slices, the best Anthropic result there. Vectara new dataset: 9.9% (October 22), the first Claude Opus under 10%. Not on the public SimpleQA Verified board yet. ^[2][3][1]

Claude Fable 5.1 (September 1, 2026): the highest AA-Omniscience v1.1 accuracy in this table at 68.4%, and the highest hallucination rate of any current Claude at 70.0% (72.6% on v1). Index 46.3, from a max run with the default fallback model. More knowledge, and less willingness to say "I don't know". No Vectara score yet, and not on the public FACTS or SimpleQA Verified boards yet. ^[2]

Claude Sonnet 5.5 (September 28, 2026): AA-Omniscience v1.1: 51.5% accuracy, 43.3% hallucination, Index 30.5 (max run with the default fallback model). On v1 its accuracy had jumped 13.8 points over Sonnet 5 and its hallucination rate had risen 7.6 points with it. SimpleQA Verified: 44.7%. ^[2][101]

Claude Fable 5 (June 9, 2026): Anthropic's first public Mythos-class model, and the AA-Omniscience Index leader at 43 (re-scored from 40 at launch) until September 2026, when Fable 5.1 (43.5), GPT-6 Astra (43.4) and then Opus 5.5 (46.4) passed it. Accuracy: 65% (a record until September). Hallucination rate: 63.6% on the August 2026 board and on Artificial Analysis's October 4 records, up from 54.9% at launch. The Index lead is accuracy-driven, not calibration-driven: a deliberate strategic shift toward answering more rather than refusing more. It falls back to Opus 4.8 on roughly 8-9% of questions, and the scored configuration includes that fallback. Best for complex reasoning and knowledge breadth where downstream verification exists. For a low-hallucination refusal profile, Opus 4.8 was the safer Claude on v1. Artificial Analysis has not re-run Fable 5 on v1.1. HalluHard, on its updated question set: 38.9% with web search, 50.9% without. No Vectara score yet, and not on the public FACTS board yet. BullshitBench v2 clear-pushback rate: 56%, against 95% for Opus 4.8 (54% until the board corrected 75 historical records on September 10). ^{[72][2][5][94]}

Claude Opus 5 (July 24, 2026): the Opus flagship until Opus 5.5, $5 / $25 per million tokens, 1M-token context, effort levels from low to max. AA-Omniscience at launch: 54% accuracy and a 50% hallucination rate, against 46.6% and 35.9% for Opus 4.8 at its launch. Final v1 record, after Artificial Analysis's August 6 grader change: Index 37 (second on the August board), 61% accuracy, 60.8% hallucination. On v1.1: 60.9% accuracy, 59.4% hallucination, Index 37.6. Its Intelligence Index score of 63 was the highest of any Anthropic model at launch, one point above Fable 5. Read the two numbers together: Anthropic traded the Opus line's refusal profile for a double-digit accuracy gain. Anthropic's launch post reports large accuracy gains in chemistry, biology, financial modeling and legal work and does not report a hallucination figure. SimpleQA Verified: 63.3%, the best Anthropic score on that board. No Vectara score yet, and not on the public FACTS board yet. ^{[83][84][2][101]}

Claude Sonnet 5 (June 30, 2026): the default mid-tier model, built for autonomous execution and multi-step tool use. AA-Omniscience final v1 record: 40.1% accuracy, 39.4% hallucination, Index 16.5. Artificial Analysis has not re-run it on v1.1. Artificial Analysis lists Sonnet 5 (max) at Intelligence Index 55. Anthropic's pre-deployment evals report lower hallucination and sycophancy than Sonnet 4.6 and stronger prompt-injection resistance. BullshitBench v2: 80% clear pushback. HalluHard without web search: 59.7% on the updated question set. SimpleQA Verified: 32.1%. No Vectara score yet. ^{[74][2][99][94][5][101]}

Claude Opus 4.8 (May 28, 2026): the calibration-first flagship, superseding Opus 4.7 after 42 days and itself superseded by Opus 5 after 57. AA-Omniscience Index at launch: 27 (then second-highest overall, behind only Gemini 3.1 Pro's 33). Accuracy at launch: 46.6%. Hallucination rate at launch: 35.9%, essentially flat against Opus 4.7, which is the headline: a full flagship generation passed with calibration held steady rather than traded away for raw knowledge. Its final v1 record reads 48.8% / 39.3% / Index 28.8, the best-calibrated frontier Claude on v1. Artificial Analysis has not re-run it on v1.1. It also took the top spot on the Artificial Analysis Intelligence Index at 61.4 when it launched, with SWE-bench Pro at 69.2% and Terminal-Bench 2.1 at 74.6%. Early testers reported it was more likely to flag uncertainty and less likely to make unsupported claims. BullshitBench v2: 95% clear pushback, the highest score on that board, now tied with Qwen3.8 Max at its minimal reasoning setting. It scored 53.5 on the FACTS board in September, before the board dropped it, and 44.5% on SimpleQA Verified. ^{[64][65][2][94][3][101]}

Claude 4.1 Opus (August 5, 2025): the leader when AA-Omniscience launched in November 2025, with 36% accuracy and an Omniscience Index of 4.8, one of only three models above zero. Its hallucination rate was 48%, third lowest at the time, behind Claude 4.5 Haiku and level with Claude Sonnet 4.5. Artificial Analysis has since deprecated the model and no longer lists it on the AA-Omniscience board. Domain leader in Law, Software Engineering, and Humanities in the launch analysis. FACTS: 46.5 in the December 2025 launch table. Vectara: 4.2% on the original dataset, 11.8% on the new one. Earlier versions of this page gave it a 0% hallucination rate, a figure no source supports. ^{[2][104][3][100][1]}

Claude Opus 4.6 (February 5, 2026): AA-Omniscience final v1 record: 47.0% accuracy, 62.8% hallucination, Index 13.7. Vectara new dataset: 12.2%. It scored 41.5 on the FACTS board in September, before the board dropped it. SimpleQA Verified: 45.2%, second among Anthropic models after Opus 5. ^{[2][1][3][101]}

Claude Opus 4.5 (November 24, 2025): AA-Omniscience at launch: 58% hallucination, 43% accuracy. Final v1 record: 46.6% accuracy, 61.0% hallucination, Index 14.0. HalluHard: 30.2% with web search, the best result on its original question set and second on the board behind GPT-6 Astra's 27.2% on the updated set, and 60.0% without search. FACTS: 51.3 in the December 2025 launch table, and 43.9 on the 2026 Kaggle board in September, which no longer lists it. Vectara new dataset: 10.9%. SimpleQA Verified: 39.0%. ^{[14][2][5][3][1][101]}

Claude Sonnet 4.6 (February 17, 2026): AA-Omniscience hallucination on launch day: about 38%, down from Sonnet 4.5's 48%, at 24% accuracy. By April, after a re-run, Artificial Analysis's record read 46% at 40% accuracy. Final v1 record: 48.4% at the max setting, against 49.1% for Sonnet 4.5 (reasoning), with accuracy at 40.9% against 32.9%. Users preferred Sonnet 4.6 over Opus 4.5 59% of the time, citing fewer hallucinations. Vectara new dataset: 10.6%. It scored 37.0 on the FACTS board in September, before the board dropped it, and reads 30.9% on SimpleQA Verified. ^{[13][50][123][2][1][3][101]}

Claude Opus 4.7 (April 16, 2026): AA-Omniscience at launch: Index 26 (second-highest overall at release, behind only Gemini 3.1 Pro's 33) and a 36% hallucination rate, the strongest calibration profile of any frontier model attempting questions at scale at the time, and 50 percentage points better than GPT-5.5's launch figure. Final v1 record: 48.9% accuracy, 42.3% hallucination, Index 27.3. BenchLM overall: 87 (April 2026). Long-context retrieval dropped to 32.2% (down from Opus 4.6's 78.3%). Anthropic explicitly attributes this to the model now reporting errors when information is missing rather than fabricating an answer. The refusal strategy made measurable. Vectara new dataset: 12.0%, the newest Claude on that board until Opus 5.5 (9.9%) in October. It scored 45.5 on the FACTS board in September, before the board dropped it. HalluHard without web search: 59.6%. ^{[58][63][2][1][3][5]}

The pattern across Claude: by Opus 4.7 and 4.8, Anthropic's models were calibrated to refuse rather than guess, trading raw accuracy for some of the lowest hallucination rates on knowledge benchmarks. On v1 that still described Opus 4.7 and 4.8 (42.3% and 39.3%) and Sonnet 5 (39.4%). Fable 5 and Opus 5 were the deliberate exceptions: they answer more, know more, and hallucinate more (63.6% and 60.8% on v1). Their successors kept the answer-first profile on v1.1, less sharply in Opus 5.5's case: Fable 5.1 at 70.0% and Opus 5.5 at 50.3%. Haiku 5.5 (34.2%) and Sonnet 5.5 (43.3%) are the best-calibrated current Claude models. BullshitBench v2 tells the same story from a different angle: Opus 4.8 pushes back on a nonsense premise 95% of the time, Sonnet 4.6 91%, Fable 5 56%. For applications where a wrong answer is worse than no answer (legal research, medical consultation, compliance work), no single Claude is a safe default: Opus 5.5 still gives a wrong answer on about half of the questions it doesn't answer correctly. ^[2][94]

Gemini Family (Google)

Gemini 4 Carbon (October 28, 2026, not yet publicly available): Google's second Gemini 4 model and the new leader of the AA-Omniscience v1.1 Index. Google is giving access to a subset of cyber defenders first, then to more trusted partners through its Fairwind Program, before a wider release. At high effort: 58.3% accuracy, 11.4% hallucination, Index 53.5, 5.4 points clear of Claude Opus 5.5. At medium effort: 48.9% accuracy and 4.2% hallucination, the lowest rate of any model that answers at least 45% of questions correctly. It leads the Index while giving a wrong answer on only about one in nine of the questions it doesn't answer correctly. Artificial Analysis also puts it first on its Intelligence Index v5. Google says that in its internal testing Carbon cuts unsupported claims by about half against Argon. On FACTS it leads the Search and Multimodal sub-boards (94.4 and 60.2) but has no suite average yet. No Vectara or SimpleQA Verified score yet. ^{[2][125][126][3]}

Gemini 4 Argon (announced September 30, 2026, public since October 13): Google gave an initial group of cyber defenders access first, then opened it on October 13 in the paid tier of the Gemini API and to Google AI Pro and Ultra subscribers in the Gemini app. AA-Omniscience v1.1 (high): 48.3% accuracy, 4.3% hallucination, Index 46.1. The move to v1.1 cut its rate from 15.1% to 4.3%, and it is now the best-calibrated public model that answers at least 45% of questions correctly. On v1 it already read 15.1% against Gemini 3.1 Pro's 50.9%, a clean break from the Gemini pattern below. The other boards have caught up with it since October. FACTS: 72.4, third among models scored on all four slices. SimpleQA Verified: 75.6%, third on the board. Vectara's new dataset is the exception: 12.1%, higher than Gemini 3.1 Pro Preview's 10.4%, so the model that guesses least on knowledge questions still adds unsupported detail to summaries. ^{[2][103][127][3][101][1]}

Gemini 3.8 Flash (September 2, 2026): AA-Omniscience v1.1 (high): 50.5% accuracy, 55.1% hallucination, Index 23.3. On v1 it had cut the Flash line's rate to 55.2% from Gemini 3.7 Flash's 64.5% at nearly the same accuracy. On the FACTS board it averages 71.2 across all four slices, and on SimpleQA Verified it scores 74.6%, fourth on that board. ^[2][3][101]

Gemini 3.5 Flash-Lite (July 21, 2026): 26.5% accuracy, 26.7% hallucination, Index 6.9 on v1.1 (29.5% / 34.4% / 5.2 on v1). Google's lightweight tier answers fewer questions correctly than the Flash models and guesses less often. FACTS: 53.5 on the 2026 board in September. The November board has no Grounding score for it, so it is not ranked here. SimpleQA Verified: 33.1%. ^[2][3][101]

Gemini 3.7 Flash (August 13, 2026): Google's workhorse until 3.8 Flash, shipped three weeks after 3.6 Flash as a refinement rather than a new pretraining run. AA-Omniscience final v1 record: 55.3% accuracy, 64.5% hallucination, Index 26.5. Artificial Analysis has not re-run it on v1.1. That is Gemini-typical: near the top on knowledge, weak on admitting gaps, though far better calibrated than Gemini 3 Flash's 93.0%. Google's launch post reports coding and agent benchmarks and no factuality figure. No Vectara score yet. It scored 71.2 on the FACTS board in September, third at the time behind GPT-6 Astra and GPT-6 Sol, before the board dropped it. SimpleQA Verified: 73.2%, though that board now marks the model deprecated. ^{[87][2][3][101]}

Gemini 3.1 Pro Preview (February 19, 2026): AA-Omniscience Index 30 at launch (33 after a re-run by April), third on the August 2026 board behind Claude Fable 5 (43) and Claude Opus 5 (37), and Google's best public model on that measure until Gemini 4 Argon opened to the public in October. Accuracy at launch: 53%. Hallucination rate: 50%, down from Gemini 3 Pro's 88%. Its final v1 record reads 54.9% accuracy, 50.9% hallucination and Index 31.9, against 91.5% hallucination for Gemini 3 Pro. This was the biggest single-update hallucination improvement until GPT-6 Astra in September 2026. It tops SimpleQA Verified at 77.5% and averages 69.1 on the FACTS board. Vectara new dataset: 10.4%. HalluHard without web search: 57.1%. BullshitBench v2: 37% clear pushback, the lowest of the US frontier flagships. ^{[15][2][101][3][1][5][94]}

Gemini 3 Pro (November 18, 2025): FACTS overall: 68.8 (highest in the December 2025 launch table). FACTS Search: 83.8. FACTS Parametric: 76.4. AA-Omniscience at launch: 53% accuracy, then the highest of any model, with 88% hallucination. Final v1 record: 55.8% and 91.5%. Vectara new dataset: 13.6% (preview). SimpleQA Verified: 72.1% (preview). The Gemini Paradox at launch: the most knowledgeable model, and one of the least self-aware. ^{[3][2][1][101]}

Gemini 3 Flash (December 2025): AA-Omniscience at launch: 55% accuracy, the highest of any model tested at the time, and a 91% hallucination rate. Final v1 record: 53.4% and 93.0%. Speed: 218 tokens/s. The most extreme version of the Gemini Paradox, brilliant and unreliable in equal measure. Suitable only for tasks with external verification. The May 2026 successor, Gemini 3.5 Flash, addresses this directly. ^[16][2]

Gemini 3.5 Flash (May 19, 2026): AA-Omniscience hallucination at launch: 61%, down 31 percentage points from the 92% Artificial Analysis then listed for Gemini 3 Flash. Final v1 record: 51.4% accuracy, 62.2% hallucination, Index 21.2 (high), against 93.0% for Gemini 3 Flash. That single generation cut the most extreme version of the Gemini Paradox by roughly a third and lifted the Omniscience Index 11 points. MMMU-Pro multimodal: 84%, the highest score recorded on that evaluation at launch. Intelligence Index: 55. It scored 63.1 on the FACTS board in September, before the board dropped it, and 70.4% on SimpleQA Verified, which now marks it deprecated. One caveat worth stating plainly: unlike Anthropic and OpenAI, Google has not published first-party hallucination data for this model. The 61% figure is independent benchmarking from Artificial Analysis, not a vendor disclosure. ^{[64][66][67][68][2][3][101]}

The pattern across Gemini: Gemini 3 Pro and 3 Flash attempted nearly every question, which gave them top accuracy scores but catastrophic hallucination rates when they hit their knowledge limits. The 3.1 Pro update showed this is addressable: by guessing less often, it cut the hallucination rate 40.6 percentage points on their final v1 records (91.5% to 50.9%) for under one point of accuracy. Gemini 3.5 Pro, announced at Google I/O in May, was never publicly released. When Google announced Gemini 4 Argon on September 30, a spokesperson told Reuters the company no longer plans to release it. From May to September, Google's public releases were Flash-only: 3.5 Flash in May, 3.6 Flash in July, 3.7 Flash in August and 3.8 Flash in September. The Gemini 4 models break the pattern: Argon, public since October 13, reads 4.3% on v1.1, and Carbon, so far open only to a subset of cyber defenders and trusted partners, 11.4% at high effort and 4.2% at medium. ^[98][103][2]

Grok Family (xAI)

Grok 4.7 (September 21, 2026): AA-Omniscience v1.1 (xhigh): 42.7% accuracy, 20.0% hallucination, Index 31.2 (47.4% / 29.3% / 32.0 on v1). On v1 it was the second release in a row in which xAI cut the rate, 5 points below Grok 4.6. Configurations differ: Artificial Analysis lists Grok 4.7 at xhigh and Grok 4.6 at high. On v1.1 it answers fewer than 45% of questions correctly, so it falls outside the group the headline figures use, but its 20.0% is the lowest rate of any OpenAI, Anthropic or xAI model on the v1.1 board. SimpleQA Verified: 59.4%, the best xAI score on that board. ^[2][101]

Grok 4.6 (August 12, 2026): xAI's flagship until Grok 4.7, $2/$6 per million tokens, 500K context, Intelligence Index 61 (tied with GPT-5.6 Sol max). AA-Omniscience final v1 record (high): 48.2% accuracy, 34.3% hallucination, Index 30.5, fourth on the August 2026 board. On v1.1 only its xhigh setting was re-run: 46.2%, 39.7%, Index 24.9. Against Grok 4.5's final v1 record that is 19.8 points less hallucination for 3.4 points less accuracy. xAI's own model card points the other way: 1.7% on its internal factuality evaluation at high effort versus 0.98% for Grok 4.5, MASK-Rectified dishonesty 1.90% versus 0.67%. Internal and independent measurements disagree on direction, not just size. SimpleQA Verified: 52.9%. Strong candidate for the verification seat in a multi-model setup, still not a sole model for high-stakes factual work. ^{[85][86][2][101]}

Grok 4.5 (July 8, 2026): released under the SpaceXAI brand. AA-Omniscience accuracy jumped to 52% at launch (from Grok 4.3's 35%), but the hallucination rate more than doubled, from 25% to 54%. The Index still rose 8 points to 26 because accuracy climbed faster than the penalty. Final v1 record: 51.6% accuracy, 54.1% hallucination, Index 25.3. It ranked 4th on the Artificial Analysis Intelligence Index at 54 at launch. Pricing $2/$6 per million tokens, 500K context. A textbook case of buying knowledge at the cost of calibration, and, five weeks later, the baseline that Grok 4.6 corrected. xAI's model card for 4.6 gives 4.5 a 0.98% score on the internal factuality evaluation. FACTS: 50.2 on the 2026 board in September. The November board scores it on the Parametric slice only (42.5). SimpleQA Verified: 53.8%. ^{[75][85][2][3][101]}

Grok 4.3 (April 2026): AA-Omniscience in two configurations. High: 34.8% accuracy, 25.8% hallucination, Index 18.0. Medium: 28.8% accuracy and 16.9% hallucination, a lower-attempt setting that edges the older Grok 4.20 Reasoning (17.4%) for the lowest-hallucination Grok. API pricing $1.25/$2.50 per million tokens. ^[69][2]

Grok 4 (July 9, 2025): Vectara old dataset: 4.8%. AA-Omniscience at launch: 39% accuracy, the highest at the time and level with GPT-5 (high), with a 64% hallucination rate, one of only three models with a positive Index. Final v1 record: 40.5%, 64.5%, Index 2.1. FACTS: 53.6 in the December 2025 launch table (Search: 75.3, Multimodal: 25.7). Domain leader in Health and Science in the AA-Omniscience launch analysis. ^{[100][104][2][3]}

Grok 4.1 Fast: xAI claims a 65% hallucination reduction (12.09% to 4.22% on internal benchmarks). AA-Omniscience tells a different story: 72% hallucination at launch and 73.4% in its final v1 record, worse than Grok 4's 64.5%. Vectara new dataset: 19.2% with reasoning, the 9th-highest rate of the 111 models on that board. Sycophancy also increased (MASK benchmark: 0.07 to 0.19-0.23). ^[17][2][1]

Grok-3 (February 17, 2025): Columbia Journalism Review: 94% citation hallucination rate, by far the worst score in that study. Vectara: 2.1% on the original dataset (as Grok-3-Beta) and 5.8% on the new one, the best xAI score on the current board. SimpleQA Verified: 39.3%. ^{[6][100][1][101]}

The pattern across Grok: internal benchmarks and independent benchmarks disagree sharply, and with Grok 4.6 they disagree on direction. AA-Omniscience v1 says 4.6 cut hallucination to 34.3% (down from 4.5's 54.1%), and 4.7 cut it again to 29.3%, which reads 20.0% on v1.1. xAI's own factuality evaluation says 4.6 got slightly worse (1.7% versus 0.98%). Grok 4.5 remains the textbook case of buying knowledge at the cost of calibration: more knowledgeable than Grok 4.3 and hallucinating more than twice as often. The 94% CJR citation hallucination rate comes from Grok-3, tested in March 2025, and no newer Grok has been through that study. Domain-specific value exists in Health and Science, and the calibration of 4.6 and 4.7 makes them stronger second voices than any Grok before them. The gap between vendor and independent numbers still makes Grok risky as a sole model for any high-stakes application.

Perplexity Sonar (Perplexity AI)

Sonar Reasoning Pro (2025): Search Arena score: 1136, statistically tied with Gemini 2.5 Pro for #1. SimpleQA F-score: 0.858, the highest of any model at time of testing. CJR citation accuracy: 37% hallucination (best tested). Response accuracy: >90% for factual queries (94% overall, 95% academic, 94% technical). ^[18][19]

Sonar Pro (2025): Built on Llama 3.3 70B, fine-tuned for search factuality. SimpleQA F-score: 0.858. Outperforms GPT-4o and Claude 3.5 Sonnet on factuality benchmarks. ^[19]

The Perplexity risk: Perplexity introduces a failure mode no other model shares. It cites real URLs with fabricated claims. The sources look legitimate, with real websites and real publication names, but the information attributed to those sources may be invented. This makes Perplexity hallucinations harder to detect than hallucinations from models that don't present external citations. A 37% citation hallucination rate means more than one in three source attributions may contain fabricated content. ^[51]

DeepSeek (DeepSeek AI)

DeepSeek V4.1 Flash 1017 (October 17, 2026): the October build of V4.1 Flash, with open weights under the MIT license. AA-Omniscience v1.1 (max): 39.6% accuracy, 64.1% hallucination, Index 0.9. On v1 this build read 66.8%, against 96.5% for the earlier V4.1 Flash build, so DeepSeek's newest model is far better calibrated than the one it replaced, though still answer-first. On October 27, before Kimi K3.1 Pro's weights came out, Artificial Analysis ranked it level with GLM-5.3 as the top model with released open weights on its Intelligence Index v5. ^{[2][124][138]}

DeepSeek V4.1 Flash: the highest hallucination rate in the v1 data, 96.5% at 46.4% accuracy (max), Index -5.3, in its final v1 record. Artificial Analysis has not re-run this build on v1.1. Against V4 Flash, accuracy rose almost 10 points and the hallucination rate barely moved. Vectara's new dataset: 4.2%, sixth-lowest of the 111 models there. SimpleQA Verified: 42.3%. Both boards list the model as DeepSeek V4.1 Flash with no build date, so either build could be behind those two figures. ^[2][1][101]

DeepSeek V4 Pro and V4 Flash (preview April 24, 2026): a cautionary data point. At launch the V4 Pro preview posted a 94% AA-Omniscience hallucination rate and an Omniscience Index of -10, and V4 Flash hit 96% with an Index of -23, among the highest ever recorded on the benchmark, in the same territory as Gemini 3 Flash (93.0% in its final v1 record) and Llama 4 Maverick (88.9%). Final v1 records: V4 Pro preview 43.0% accuracy, 94.1% hallucination, Index -10.7. V4 Flash 36.8%, 96.1%, Index -23.9. Strong coding performance paired with near-total overconfidence on knowledge questions. Vectara new dataset: 8.6% for V4 Pro. ^[70][2][1]

DeepSeek V4 Pro 0813 (general availability August 13, 2026): the GA build that took V4 Pro out of preview. AA-Omniscience v1.1 (max): 45.2% accuracy, 94.1% hallucination, Index -6.4, the highest hallucination rate of any model in our table on v1.1 (94.8% on v1). On v1, accuracy rose 6.1 points over the preview and the hallucination rate barely moved, which is why the Index climbed from -10.7 to about zero: DeepSeek added knowledge without adding any willingness to say "I don't know". The two builds are scored separately on this page because the checkpoints differ. ^[92][88][2]

DeepSeek-V3 (December 26, 2024): Vectara: 3.9% on the original dataset, 6.1% on the new one. A strong performer on grounded summarization. SimpleQA Verified: 27.4%. ^{[100][1][101]}

DeepSeek-R1 (January 20, 2025): Vectara old dataset: 14.3%, nearly 4x higher than V3, and 11.3% on the new dataset. AA-Omniscience (0528 build): 83.4% hallucination at 30.5% accuracy. Vectara analysis found R1 produces 71.7% "benign hallucinations" (plausible-sounding additions) vs V3's 36.8%. ^{[49][48][100][1][2]}

The pattern: DeepSeek's reasoning model (R1) hallucinates dramatically more than its base model (V3). This is the reasoning tax in its most extreme form. The gap (3.9% vs 14.3%) makes it one of the clearest examples that reasoning capabilities and factual reliability don't move in the same direction. On Vectara's new dataset the gap narrows (6.1% vs 11.3%) but holds.

Open-Weight and Regional Models

Kimi K3.1 Pro (Moonshot AI, October 23, 2026): AA-Omniscience v1.1: 48.9% accuracy, 37.3% hallucination, Index 29.9. Artificial Analysis ranked it the top open-weights model on its Intelligence Index v5 on October 27, and Moonshot released the weights on November 2. On v1 it read 43.9% hallucination, against Kimi K3's 53.2%. Moonshot reports that on its own Kimi Hallu Bench Hard stress test, the claim-level hallucination rate fell from 23.4% for K3 to 8.3%. ^{[2][124][132]}

Muse Wonder and Muse Wonder Full (Meta, October 12, 2026): Meta's new top models. AA-Omniscience v1.1 (max): Muse Wonder Full 41.0% accuracy, 17.6% hallucination, Index 30.6, and Muse Wonder 38.0%, 20.9%, Index 25.1. Both sit with the cautious models: Wonder Full has the lowest hallucination rate in our table outside the Gemini 4 models and Command A+. Muse Wonder is available through Meta's Model API and Muse Code, Wonder Full to Muse Code subscribers, and neither has open weights. Wonder Full also leads Artificial Analysis's Harvey LAB-AA v1.1 legal benchmark, which zeroes any task with a material hallucination, at 13.1%. ^{[2][130][139]}

Muse Spark Open (Meta, October 15, 2026): Meta's open-weights model, released under the Muse Model License with limited commercial terms. AA-Omniscience v1.1 (xhigh): 42.0% accuracy, 28.0% hallucination, Index 25.8. ^[2][131]

Mistral Large 4 (Mistral AI, October 29, 2026, after a preview on October 6): released with open weights under a modified MIT license. AA-Omniscience v1.1 (high): 26.0% accuracy, 28.2% hallucination, Index 5.1. The preview read 23.9%, 41.3% and Index -7.5, so the release answered slightly more questions correctly and guessed much less. ^[2][133]

Inkling 2 Small (Thinking Machines Lab, November 5, 2026): AA-Omniscience v1.1: 33.8% accuracy, 26.7% hallucination, Index 16.1. It is available through Thinking Machines' Tinker platform, and the company plans to release its weights under Apache 2.0, which Artificial Analysis says would make it the highest-scoring open-weights model developed outside China. ^{[2][135][136]}

Tencent Hy4 (Tencent, November 4, 2026): the clearest vendor-versus-independent gap in this update. Tencent's research blog says Hy4 cut its hallucination rate from 15.76% to 5.45%, and its press release gives 12.95% to 7.2% for the same step. Artificial Analysis measures 50.0% accuracy and an 89.1% hallucination rate on AA-Omniscience v1.1. The figures measure different tests, and Tencent's two do not agree with each other. Tencent has promised open weights but not yet released them. ^[2][134]

MiniMax-M3.1-Flash (MiniMax, October 19, 2026, open weights): 45.1% accuracy and a 93.7% hallucination rate on AA-Omniscience v1.1. Like Hy4, it attempts almost every question, about 99% of them, so almost every gap in its knowledge becomes a wrong answer. ^[2][137]

Qwen3.8 Max (0902) (Alibaba, September 2, 2026): 29.5% accuracy, 27.1% hallucination, Index 10.5 on v1.1 (31.7% / 28.8% / 12.0 on v1). On v1 the September build pulled the hallucination rate back down from the preview's 41.7% at the same accuracy, close to the abstention-first profile of Qwen3.7 Max. ^[2]

Muse Spark 1.3 (Meta, September 2, 2026): 38.0% accuracy, 25.2% hallucination, Index 22.4 at max on v1.1 (43.6% / 32.9% / 25.0 on v1). On v1 it sat close to Muse Spark 1.2 (xhigh) on calibration, 1.8 points lower on accuracy. ^[2]

Kimi K3 (Moonshot AI, July 2026, open weights released July 27): a 2.8-trillion-parameter open-weights MoE model with a 1M-token context window, the first open model in the 3T class. AA-Omniscience: 46% accuracy, 51% hallucination, Index 18 at launch (47.6% / 53.2% / 19.7 in its final v1 record). Like Grok 4.5, its accuracy gain over the prior generation came with a calibration regression. Ranks among the top open-weight models on the Artificial Analysis Intelligence Index. ^[76][93][2]

Command A+ (Cohere, open weights, May 20, 2026): 13.9% hallucination on v1.1 (14% on v1), the lowest on Artificial Analysis's board when it launched in May 2026, reached by refusing most questions it can't verify (9.8% accuracy, Index -2.7). Only the Gemini 4 models read lower in our table on v1.1. A "knows-its-limits" archetype similar to Claude 4.5 Haiku, useful as a verification layer in a multi-model pipeline. ^[77][2]

Qwen3.7 Max (Alibaba, May 19, 2026): an abstention-first model that refuses whenever its internal confidence threshold isn't met. At launch: 30% accuracy, 23% hallucination, Index 14. Final v1 record: 31.1%, 25.6%, Index 13.5. It has a 1M-token context window, and its refusal habit makes it another strong verification-layer candidate. ^[78][2]

Qwen3.8 Max Preview (Alibaba, previewed July 19, 2026, launched August 3, the build Artificial Analysis scores): a 2.4-trillion-parameter MoE flagship with open weights promised. AA-Omniscience final v1 record: 31.9% accuracy, 41.7% hallucination, Index 3.4. Accuracy barely moved from Qwen3.7 Max, and the hallucination rate rose from 25.6% to 41.7%: Alibaba loosened the abstention policy that made 3.7 Max a verification-layer candidate, and got almost no knowledge in return. The clearest recent example of calibration being a training choice, not a capability. ^[90][2]

GLM-5.3 (Z.ai, August 14, 2026): a coding and cyber-security flagship built on the GLM-5.2 base without retraining. On v1 it read 33.9% accuracy, 29.6% hallucination and Index 14.3, the best-calibrated new open-weight model of the summer, in the Qwen3.7 Max and Grok 4.3 band. Version 1.1 moved it the other way: 32.3%, 38.9%, Index 6.0, the largest rise of any model in our table that Artificial Analysis re-ran. ^[91][2]

Llama 4 Maverick (Meta, April 5, 2025): Vectara: 4.6% on the original dataset (competitive) and 8.2% on the new one. AA-Omniscience: 88.9% hallucination in its final v1 record (87.6% when first published here), at 24.9% accuracy. The gap between grounded summarization and open-ended knowledge is wider for open-source models than for any proprietary family. ^[100][1][2]

In a 2025 clinical test with planted fake details, a distilled DeepSeek-R1 model built on the fake detail 80.0-82.7% of the time, while Llama-3.3-70B did about as well as GPT-4o. For critical applications, the gap between open and proprietary models depends on the model and the task. ^[105]

Muse Spark 1.1 (Meta, July 9, 2026): carries an AA-Omniscience profile, not just a HealthBench score. July launch configuration: Index 18 (up from Muse Spark 1.0's 4), accuracy 41% (roughly flat), hallucination 38% (down from 73%). The entire gain came from abstention: attempt rate dropped from 95% to 82%. It is the clearest recent example of a model getting more reliable by learning when to stay silent rather than by learning more. The xhigh configuration in our table reads 52.1% accuracy / 50.0% hallucination / Index 28.1: more knowledge, and the abstention discipline gone. HealthBench Professional: 59.3%. Its HealthBench Hard leadership (42.8) belongs to the original Muse Spark. A 1.1-specific Hard score is not yet confirmed. ^[71][88][2]

Muse Spark 1.2 (Meta, August 5, 2026): a coding-focused update to 1.1, co-trained with Meta's Muse Code agent. AA-Omniscience final v1 record (xhigh): 45.4% accuracy, 33.3% hallucination, Index 27.2. Set against the xhigh configuration of 1.1, that is a 16.7-point cut in hallucination for 6.7 points of accuracy. On v1, among models in our table that are now public and answered at least 45% of questions correctly, only Gemini 4 Argon and Grok 4.7 were better calibrated. Meta's launch post reports coding benchmarks only and no factuality figure. ^[89][88][2]

Head-to-Head Model Comparisons

The model profiles in Section 6 show individual performance. This section answers the questions people actually search for: "Is Claude or GPT more accurate?" "Should I use Gemini or Claude?" The answer depends on what you're doing, but the data makes the tradeoffs specific. In the Claude, GPT, Gemini and Grok tables every cell names the model behind the figure, and most rows compare the best result each family posts on a benchmark. Current figures come from Artificial Analysis's AA-Omniscience v1.1 records of November 7, 2026, Vectara's October 22 board, the FACTS Kaggle board of November 5, the SimpleQA Verified board of October 20 and HalluHard's current board. AA-Omniscience rows compare v1.1 records only, so models Artificial Analysis has not re-run, such as Claude Opus 4.8 and Gemini 3.1 Pro, drop out of those rows. Older results are labeled as such: Vectara's original dataset (archived October 2025), the December 2025 FACTS launch table, HealthBench (August 2025), the CJR citation study (March 2025) and the Perplexity comparison, which dates from 2025.

Claude vs GPT

The most searched comparison in AI, and the most context-dependent. On AA-Omniscience v1.1 the newest flagships are close on calibration: Claude Opus 5.5 reads 65.5% accuracy and 50.3% hallucination, GPT-6 Astra 61.6% and 49.8%, and GPT-6.1 Sol 61.8% and 51.9%. Claude Fable 5.1 knows the most, at 68.4%, and guesses the most, at 70.0%. All four give a wrong answer about as often as they decline, or more often. ^[2][124]

The pattern isn't "one is better." It's two different philosophies measured on different scales.

GPT models are stronger when the task has source material to work from. Summarization, document analysis, RAG pipelines, search-grounded Q&A: GPT sticks closer to provided text and scores well on faithfulness benchmarks. OpenAI holds the best frontier-lab score on Vectara's new dataset (GPT-5.4 nano, 3.1%, against 9.8% for the best Claude), and GPT-6 Astra leads the FACTS board's fully scored models at 78.1, 11.6 points above Claude Opus 5.5. GPT also leads on short factual questions answered from memory, without tools: GPT-6 Astra scores 75.8% on SimpleQA Verified, against 63.3% for Claude Opus 5, the best Claude on that board. ^[1][3][101]

Claude used to be the stronger family when the task requires the model to know its own limits. On AA-Omniscience v1, Claude Opus 4.8 gave a wrong answer on 39.3% of the questions it didn't answer correctly, the best calibration of any Claude or GPT model that answered at least 45% correctly. Artificial Analysis has not re-run it on v1.1, and among the models it has re-run, OpenAI now leads: GPT-6.1 Terra reads 34.5% at 47.6% accuracy, against 43.3% for Claude Sonnet 5.5, the best-calibrated Claude that answers at least 45% correctly. When AA-Omniscience launched in November 2025, Anthropic held the three lowest hallucination rates, led by Claude 4.5 Haiku. HalluHard's realistic conversation test splits by question set. On the updated set, GPT-6 Astra with web search (27.2%) beats Claude Fable 5 (38.9%). On the original set, Claude Opus 4.5 (30.2%) edges GPT-5.4 Thinking (31.9%). ^[2][5]

One more variable that often gets overlooked: web search access. OpenAI's GPT-5 numbers show it. Its hallucination rate on SimpleQA without web access was 47%, while 9.6% of its factual claims in browsing-enabled ChatGPT traffic contained an error, a different metric that points the same way. HalluHard shows the effect for both labs on the same test: GPT-6 Astra drops from 43.2% without search to 27.2% with it, and Claude Fable 5 from 50.9% to 38.9%. Whichever model you use, turn search on for factual work. ^[8][12][5]

Claude vs Gemini

For eighteen months the shorthand was "Gemini knows more, Claude is more honest about what it doesn't know." Both halves have flipped. On AA-Omniscience v1.1, Claude leads on knowledge: Fable 5.1 answers 68.4% of questions correctly, against 50.5% for the best public Gemini, Gemini 3.8 Flash, and 58.3% for Gemini 4 Carbon, which is not yet publicly available. SimpleQA Verified points the other way: Gemini 3.1 Pro Preview scores 77.5%, 14.2 points above Claude Opus 5, the best Claude on that board. And Gemini now leads on calibration with a model you can use: Gemini 4 Argon, public since October 13, gives a wrong answer on 4.3% of the questions it doesn't answer correctly, against 50.3% for Claude Opus 5.5. ^[2][101]

Argon changes the trade-off rather than removing it. It answers 48.3% of questions correctly, 17.2 points fewer than Opus 5.5, and declines most of the rest. Claude Sonnet 5.5 sits between them at 51.5% accuracy and 43.3% hallucination. Google's other public model that answers at least 45% correctly on v1.1, Gemini 3.8 Flash, still follows the old Gemini pattern at 55.1%. Before the re-run, Claude Opus 4.8 (39.3%) and Gemini 3.1 Pro (50.9%) were each lab's best-calibrated public model answering at least 45% correctly on v1, and Artificial Analysis has not re-run either on v1.1. ^[2]

Gemini also still leads on grounded summarization, 0.7% on Vectara's original dataset and 3.3% on the new one, though Gemini 4 Argon itself reads 12.1% there, above Claude Opus 5.5's 9.9%. It leads both FACTS tables: Gemini 4 Argon averages 72.4 across all four slices of the 2026 board, 5.9 points above Opus 5.5, and Gemini 3 Pro led the December 2025 launch table at 68.8. ^[100][1][3]

GPT vs Gemini

GPT vs Gemini changed most in October. Gemini 4 Argon opened to the public on October 13, and on AA-Omniscience v1.1 it gives a wrong answer on 4.3% of the questions it doesn't answer correctly, against 49.8% for GPT-6 Astra and 34.5% for GPT-6.1 Terra, OpenAI's best-calibrated model. OpenAI keeps the lead on accuracy, 61.8% for GPT-6.1 Sol against 48.3% for Argon and 50.5% for Gemini 3.8 Flash, and on the FACTS board's fully scored models, 78.1 for GPT-6 Astra against 72.4 for Argon. Gemini keeps a narrow lead on SimpleQA Verified, 77.5% against 75.8%, inside the board's confidence intervals, and still leads the December 2025 FACTS launch table. GPT's other advantages are task-specific: medical applications (1.6% HealthBench), the best frontier-lab score on Vectara's new dataset (GPT-5.4 nano, 3.1%), and the sheer volume of internal evaluation data OpenAI publishes. ^{[2][3][101][1]}

Grok vs the Field

xAI's internal numbers and the independent ones have never agreed on size. With Grok 4.6 they stopped agreeing on direction. AA-Omniscience v1 said 4.6 cut the hallucination rate to 34.3% from Grok 4.5's 54.1%, and Grok 4.7 cut it again to 29.3%. On v1.1, Grok 4.7 reads 20.0%, the lowest of any public model in our table except Gemini 4 Argon, Command A+ and Muse Wonder Full. xAI's own model card says 4.6 got slightly worse on its internal factuality evaluation (1.7% versus 0.98%). Both can be true, because they measure different things on different scales. ^[2][85]

Calibration is where Grok beats most of the field. Knowledge is where it trails: Grok 4.7 answers 42.7% of AA-Omniscience v1.1 questions correctly, 25.7 points behind Claude Fable 5.1, and places 10th of 47 on SimpleQA Verified. The older Grok 4.1 Fast is the cautionary case: 73.4% hallucination in its final AA-Omniscience v1 record and 19.2% on Vectara's new dataset, the 9th-highest rate of 111 models there. The CJR citation study found Grok-3 hallucinated 94% of the time on news source attribution, the worst of the eight tools tested. ^{[2][101][1][6]}

Grok does have genuine domain strengths: it led Health and Science categories in the AA-Omniscience launch analysis. But the gap between xAI's claims and independent measurements is wider than for any other provider.

Perplexity vs ChatGPT vs Claude

In the 2025 tests, Perplexity won on factual search queries. Its RAG-native architecture, built around retrieval rather than parametric knowledge, gives it a structural advantage for questions with verifiable answers. The current search-enabled boards have no Perplexity entry: HalluHard's best search-enabled score is GPT-6 Astra's 27.2%. ^[5]

The catch: Perplexity cites real URLs with fabricated claims. The sources look legitimate, with real websites and real publication names, but the information attributed to those sources may be invented. At a 37% citation hallucination rate, more than one in three source attributions could contain fabricated content. This makes Perplexity hallucinations harder to spot than hallucinations from models that don't present external citations.

Domain-Specific Hallucination Rates

Hallucination rates vary dramatically by subject matter. A model that looks reliable on general-knowledge benchmarks can be dangerously wrong on legal or medical questions. The domain with the strongest primary research is law.

Legal: Purpose-Built Tools Still Fail

Stanford RegLab and Stanford HAI tested the paid legal research tools marketed as "hallucination-free" (Magesh, Surani, Dahl, Suzgun, Manning and Ho, Journal of Empirical Legal Studies, April 2025). Lexis+ AI and Ask Practical Law AI gave incorrect or misgrounded answers on 17% of queries, Westlaw AI-Assisted Research on 33% and GPT-4 on 43%. Stanford HAI's summary puts the Westlaw figure at more than 34%. General-purpose chatbots did worse on specific questions about federal court cases, with hallucination rates from 58% (GPT-4) to 88% (Llama 2) (Dahl, Magesh, Suzgun and Ho, Journal of Legal Analysis, 2024). Both studies tested 2023-2024 product versions, not today's. ^[36][96][97] A newer test by Vals AI (October 14, 2025) put 200 U.S. legal research questions to three legal AI tools and ChatGPT. Where they answered, the legal tools were fully correct on 78-81% of questions and ChatGPT on 80%. Lexis+ AI and Westlaw were not in the test. ^[112] Domain-specific reliability claims need domain-specific checks.

Agentic legal work shows the same gap. Harvey LAB-AA v1.1, which Artificial Analysis launched with Harvey on October 8, 2026, checks every deliverable from 120 private legal tasks against its source documents and credits a task only when it contains no material hallucination. Across the models tested at launch, more than 60% of otherwise passing results contained one. Without the check, Muse Spark 1.3 would have led with a 26.7% all-pass rate, but two thirds of those passes contained a material hallucination. On the live board Meta's Muse Wonder Full now leads at 13.1%, ahead of Muse Wonder (10.0%) and Grok 4.7 (9.4%). ^[139]

Legal: The Courtroom Crisis

Court decisions involving AI-hallucinated material rose sharply through 2025 and have stayed high in 2026.

Damien Charlotin's database lists 2,266 court decisions worldwide in which a party relied on AI-hallucinated material, 1,535 of them in U.S. courts (last updated November 5, 2026). By decision date, it shows 16 decisions in 2023, 61 in 2024 and 865 in 2025, plus 1,324 from January 1 to November 3, 2026. Older years keep growing as past rulings are found: a month earlier the database held 855 decisions for 2025. In May 2025 it showed 10, 37 and 73 for 2023, 2024 and the first five months of 2025. New decisions ran at about 145 a month from January to August 2026, close to five a day, and the database lists 24 decisions dated March 31, 2026 alone. The live count on any later day will be higher than the one printed here. ^[37][38]

Self-represented litigants still account for most cases, 1,305 of the 2,266, but lawyers appear in 898 (database filters, November 5, 2026). In May 2025, Business Insider reported legal professionals at fault in at least 13 of 23 recent cases. In February 2025, Morgan & Morgan, one of the largest U.S. personal injury firms, emailed its more than 1,000 lawyers that AI can invent case law and that filing fake citations could cost them their jobs. Days later a federal court in Wyoming fined the firm's lawyers in Wadsworth v. Walmart. ^{[37][38][114]}

Penalties have grown. The largest U.S. amount in the database is US$110,204 in Couvrette v. Wisnovsky (D. Oregon, March 23, 2026), a US$15,500 sanction plus about US$94,700 in costs. By our tally of the database's U.S. dollar amounts, the highest quarter so far is Q4 2025, at about US$391,000 across 42 decisions, followed by Q1 2026 at about US$361,000 across 35. These totals include costs awards, not only fines. The Fourth Circuit publicly admonished a lawyer on March 11, 2026 (In re Nwaubani). Monthly counts rose through 2025 and held at roughly 125-185 decisions a month from January to August 2026, the months that are close to complete. ^[37]

The filers now include government lawyers. On October 21, 2026, the Federal Labor Relations Authority found that 22 of 29 footnotes and 35 of 42 legal or record citations in exceptions filed by the U.S. Department of Health and Human Services were fictitious, erroneous or irrelevant, including a citation to a decision that does not exist. It said the agency may have used a generative AI tool, denied the exceptions and imposed no sanction (74 FLRA No. 87). Two days later, in U.S. ex rel. Phillips v. Los Angeles Film School (C.D. California), the court sanctioned the relators' lawyer US$5,000 under Rule 11 for AI-fabricated case law, ordered continuing legal education and made a bar referral. ^[140][141]

The benchmark data explains why. On the central holding of a case, the published Dahl study found hallucinations in at least 63% of answers, a lower bound. Stanford HAI's January 2024 summary of the earlier preprint, before GPT-4 was added, gave at least 75%. ^[36][97]

Healthcare: Where Hallucinations Can Kill

ECRI, the patient safety nonprofit, ranked risks from AI-enabled health technologies as the top health technology hazard for 2025. For 2026 it ranked misuse of AI chatbots first, citing chatbots that suggested wrong diagnoses, recommended unnecessary tests and invented body parts while sounding like a trusted expert (January 21, 2026). ^[39][108]

The FDA's public list of AI-enabled medical devices holds 1,614 authorized devices, up from 637 through 2022 (list current as of September 4, 2026). A separate study of 950 of these devices found 60 (6.3%) linked to 182 recall events, with 43.4% of recalls within 12 months of clearance (Lee et al., JAMA Health Forum, August 22, 2025). Most recalls were for diagnostic or measurement errors, not chatbot-style hallucination. ^[107][106]

A 2025 study planted one made-up detail, such as a fake lab value or condition, in each of 300 physician-checked clinical vignettes and asked six models to respond (Omar et al., Communications Medicine, August 2, 2025). The models built on the fake detail in 50-82% of cases. A mitigation prompt cut the average from 66% to 44%. GPT-4o did best, falling from 53% to 23%. A distilled DeepSeek-R1 model did worst, at 80.0-82.7% by default. ^[105]

OpenAI reports that GPT-5 with thinking hallucinated on 1.6% of its HealthBench Hard hallucination test, against 3.6% without thinking, 12.9% for o3 and 15.8% for GPT-4o (August 7, 2025). This is a vendor-run test. ^[109]

HealthBench Professional (April 2026): OpenAI's newer benchmark scores models on 525 physician-written tasks from real clinician use, across care consult, writing and documentation, and medical research. In OpenAI's paper, GPT-5.4 inside ChatGPT for Clinicians scored 59.0, base GPT-5.4 48.1 and specialist physicians with web access 43.7. On consult tasks the gap was 51.0 to 42.7. OpenAI built and graded the benchmark, and picked examples that are hard for its own models. ^[60] Separately, on HealthBench Hard, the hard subset of the original HealthBench, Meta reported Muse Spark at 42.8 at its April 8, 2026 launch. ^[71]

October 2026 update: OpenAI's GPT-6 Astra launch table gives length-adjusted HealthBench Professional scores of 63.4% for GPT-6 Astra, 60.9% for Claude Fable 5, 60.5% for GPT-5.6 Sol, 58.1% for Claude Fable 5.1, 56.4% for Claude Opus 5 and 52.1% for Gemini 3.8 Flash. These are vendor-reported. ^[110]

Finance: Quiet Failures with Loud Consequences

Financial AI hallucinations don't make headlines the way legal ones do. There is no court docket for a wrong number in a model, which is also why the domain has the thinnest primary data on this page.

What can be sourced: EY's 2025 Responsible AI Pulse puts the average loss from AI-related incidents at $4.4 million per affected organization (n=975 C-suite), a figure that covers AI risk broadly, not hallucination alone. ^[79] A Stanford study found that all 24 models tested struggled to flag a false belief once the user stated it as their own. DeepSeek R1's accuracy fell from over 90% to 14.4% (Nature Machine Intelligence, November 3, 2025). That is the failure that matters most when the false premise is a valuation assumption. ^[111]

The strongest financial signal is first-party. In the Suprmind Multi-Model AI Divergence Index, AI models contradicted each other about twice as often on financial questions as on medical ones: 40% against 20% of turns under a blind judge. Legal questions came close behind, at 34.5% against finance's 39.9%, too close to tell apart. Where the stakes are a portfolio or a contract, one model's answer is among the most likely to be contradicted by another. ^[61]

A set of widely circulated finance figures (a 15-25% hallucination rate on financial tasks, 2.3 significant errors per quarter, $50,000 to $2.1 million per incident, a 20.0% versus 76.7% literature-reference comparison, and a $12.7 million SEC total) appeared in earlier versions of this page. None could be traced to a primary study, so they were removed in the August 27, 2026 revision, on the same rule that removed the untraceable business-impact figures on August 8.

The Reasoning Paradox

One of the most counterintuitive findings in 2025-2026 hallucination research: the AI models marketed as most intelligent are often the least reliable on basic factual tasks.

The Core Contradiction

Reasoning models (GPT-5 with thinking, Claude with extended thinking, DeepSeek-R1) use chain-of-thought processes that dramatically improve performance on complex problems. They're measurably better at math, logic, multi-step analysis, and medical diagnosis.

They're also measurably worse at sticking to facts they've been given.

The Evidence

Vectara new dataset: At its November 2025 launch, the thinking models Vectara named, including GPT-5, Claude Sonnet 4.5 and Grok-4, all exceeded 10% hallucination, and Gemini-3-Pro read 13.6%. Grok-4-fast-reasoning reads 20.2% on the current board. Non-reasoning models like Gemini-2.5-Flash-Lite scored 3.3%. OpenAI's GPT-6 models have narrowed the gap since, at 6.5% for GPT-6 Sol and 8.7% for GPT-6 Astra. ^[102][1]

DeepSeek: R1 (reasoning) hallucinates at 14.3% on Vectara vs V3 (base) at 3.9%. Nearly a 4x difference from the same provider. Vectara analysis found R1 produces 71.7% "benign hallucinations" (plausible-sounding additions) compared to V3's 36.8%. ^[48][49]

DeepSeek V4 (April 2026): The pattern held at a greater extreme. V4 Pro and V4 Flash post AA-Omniscience hallucination rates of 94% and 96%, among the highest ever recorded, while ranking near the top of open-weights models for coding and agentic work. Strong capability, almost no calibration. ^[70]

PersonQA regression: OpenAI's o3 hallucinates 33% on questions about real people vs o1's 16%. The o4-mini is worse at 48%. These are newer, more capable models performing worse on a basic factual test. ^[53][54]

GPT-5 thinking mode: HealthBench hallucination drops to 1.6% (excellent). But on Vectara's new dataset, GPT-5 (high) reads 15.1% (poor). Same model, same thinking mode, opposite results depending on the task.

GPT-5.5 (April 2026): AA-Omniscience accuracy of 57% (a record at the time) paired with an 86% hallucination rate. Knowledge expansion appears to have outpaced calibration improvements at the frontier. Claude Opus 4.7 took the opposite trade at its launch: 36% hallucination at lower raw accuracy (42.3% in its final v1 record). ^[2][58][63]

The 2026 flagships: the pattern moved to the very top of the leaderboard. On AA-Omniscience v1, Claude Fable 5 (65% accuracy / 63.6% hallucination), Claude Opus 5 (61% / 60.8%, up from Opus 4.8's 39.3%), GPT-5.6 Sol (59.4% / 92.2%, the highest rate of any OpenAI flagship), Grok 4.5 (51.6% / 54.1%, doubled from Grok 4.3), and Kimi K3 (47.6% / 53.2%) all raised accuracy and raised hallucination in the same release. The most capable models shipped in 2026 are among the most overconfident. Before September, counterexamples included Gemini 3.1 Pro (February 2026), which cut Gemini 3 Pro's rate from 91.5% to 50.9%, and Grok 4.6 (August 2026): 48.2% accuracy and 34.3% hallucination, a 19.8-point calibration gain for a 3.4-point accuracy cost. It proves the trade is a choice, not a law. September's releases split the field on v1: GPT-6 Astra cut OpenAI's rate to 51.3% while gaining accuracy (62.6%), Grok 4.7 cut xAI's again to 29.3%, and Gemini 4 Argon read 15.1%, while Claude Fable 5.1 went the other way, to 72.6%. On AA-Omniscience v1.1, Gemini 4 Carbon (October 2026) shows the trade can be avoided at the top: the highest Index on the board, 53.5, with an 11.4% hallucination rate. ^{[2][72][75][88][125]}

Why This Happens

The mechanism is straightforward. When a reasoning model processes a summarization task, it doesn't just extract. It thinks. It draws inferences, identifies patterns, and generates insights. These additions go beyond the source document. On a benchmark measuring faithfulness to source material, every insight the model adds counts as a hallucination.

It's the difference between "summarize this contract" and "analyze this contract." Reasoning mode adds analysis even when you ask for a summary. That analysis is often useful. On a summarization benchmark, it's scored as a failure.

The Browse Effect Is Bigger Than the Reasoning Effect

OpenAI's system card data reveals something that gets less attention: web access has a larger impact on hallucination rates than reasoning mode does. ^[11][8]

For enterprise deployments, ensuring tool access is more impactful than selecting reasoning vs non-reasoning model variants.

The Decision Framework

This creates a practical matrix for model selection:

Reasoning ON + Web ON: Best for complex analysis, medical diagnosis, multi-step research where both depth and access to current information matter. Lowest hallucination rates on open-ended tasks.

Reasoning OFF + Web ON: Best for document summarization, RAG pipelines, grounded Q&A where you want the model to stay close to source material. Lower risk of "overthinking" additions.

Reasoning ON + Web OFF: Risky combination. The model overthinks and can't verify. Suitable only for closed-world logic problems, math, and code where external facts aren't needed.

Reasoning OFF + Web OFF: Highest hallucination risk across the board. Avoid for any factual task.

What Actually Reduces Hallucination, Ranked by Evidence

Not all hallucination reduction techniques are equal. Some are backed by controlled studies with precise measurements. Others have strong theoretical support but limited production data. This ranking reflects the evidence base, not marketing claims.

Tier 1: Largest Measured Impact

1. Web Search Access

Measured impact: 73-86% hallucination reduction (FActScore, browse-on vs browse-off)

The single highest-impact intervention documented in 2025-2026 research. On OpenAI's FActScore evaluations, o4-mini drops from 37.7% to 5.1% with web access and GPT-5 thinking from 3.7% to 1.0%. GPT-5.3 Instant shows 26.8% reduction when using web vs prior models. ^[8][10]

One scope note. The 73-86% range comes from OpenAI's own FActScore evaluations of its own models. The size of the effect varies by model, benchmark and search system, so treat it as the best-documented case rather than a universal constant, and expect any browse-on number to describe the model plus its search stack, not the model alone.

The mechanism is simple: instead of relying on potentially stale or incorrect training data, the model retrieves current information and grounds its response in external sources. For any enterprise deployment, enabling web or tool access should be the first configuration decision, not an afterthought.

2. RAG (Retrieval Augmented Generation)

Measured impact: Large and setup-dependent. No cross-vendor benchmark isolates it, and the "up to 71%" figure that circulated for enterprise knowledge-base tasks traces to a secondary compilation rather than a controlled study, so this page no longer quotes it as a benchmark result.

RAG connects models to external knowledge bases (company documents, databases, verified sources) and instructs the model to generate responses grounded in retrieved content rather than parametric memory. Hybrid retrievers combining sparse and dense methods achieve the strongest mitigation. Vectara's own leaderboard, which measures exactly this grounded setting, shows the best models under 2% and most frontier models between 5% and 14%. Retrieval narrows the problem, but it does not close it.

RAG is most effective for knowledge-gap hallucinations (the model lacks relevant training data). It's less effective for logic-based hallucinations (the model reasons incorrectly from correct premises). For enterprise document Q&A and knowledge base applications, RAG is the standard of care.

Tier 2: Strong Evidence, Context-Dependent

3. Thinking/Reasoning Mode

Measured impact: 56-59% fewer errors on OpenAI's medical and production tests, but more hallucination on grounded summarization ^[109][8]

GPT-5 with thinking hallucinated on 1.6% of OpenAI's HealthBench Hard hallucination test, against 3.6% without thinking. On production-like ChatGPT traffic with browsing on, 4.8% of GPT-5 thinking responses had at least one major incorrect claim, against 11.6% for GPT-5 main. Both are vendor-run tests, and both are large improvements.

But reasoning mode increases hallucination on Vectara's summarization benchmark (see Section 10). The impact is task-dependent. Enable reasoning for analysis, diagnosis, and complex queries. Disable it for summarization, extraction, and source-faithful tasks.

4. Multi-Model Cross-Validation

Measured impact: 8% higher factual accuracy than the best earlier hallucination-mitigation methods (UAF framework) ^[24]

Amazon's Uncertainty-Aware Fusion framework (WWW Companion 2025) combined multiple LLMs weighted by their accuracy and self-assessment quality. The key finding: different models excel on different question types, so combining them captures complementary strengths.

Cross-model disagreement detection catches hallucinations because models rarely fabricate the same false information. When one model makes an unsupported claim, others typically flag the inconsistency or provide conflicting data. Research on "wisdom of the silicon crowd" shows LLM ensembles can rival human crowd forecasting accuracy through simple aggregation.

The 8% figure understates the practical value. In production, multi-model approaches catch errors that no single-model check would flag, because the checking model has different training data, different biases, and different blind spots.

5. Chain-of-Verification (CoVe)

Measured impact: 28% FActScore improvement ^[23]

A four-step pipeline: generate baseline response, plan verification questions, answer those verification questions independently, then refine the final output. Published at ACL 2024, it outperforms zero-shot, few-shot, and chain-of-thought prompting on long-form generation accuracy.

The cost is latency and compute: four steps instead of one. For applications where accuracy matters more than speed (report generation, research synthesis, compliance documentation), the tradeoff is worth it.

Tier 3: Meaningful but Narrower

6. VeriFY (Training-Time Verification)

Measured impact: 9.7-53.3% lower factual hallucination rates on open models from 2B to 12B ^[25]

VeriFY, described in a February 2026 arXiv preprint, teaches models to assess factual uncertainty during generation rather than relying on post-hoc checking. The model learns to verify its own claims as it produces them. Recall loss is modest: 0.4-5.7%.

This is a training-time intervention, meaning end users don't control it. Its value is in signaling where the field is heading: future model generations will likely internalize verification as a core capability rather than bolting it on after generation.

7. Training Models to Guess Less

Measured impact: 38 percentage points lower AA-Omniscience hallucination rate (Gemini 3 Pro Preview 88%, Gemini 3.1 Pro Preview 50%) at about the same accuracy ^[15]

Artificial Analysis traced the drop to the model guessing less often when it does not know the answer, and its Omniscience Index rose 17 points. Neither Artificial Analysis nor Google's model card names the training method behind it. In their final v1 records the two models read 91.5% and 50.9%.

Like VeriFY, this is a provider-side intervention. Users benefit from it when selecting newer model versions but can't apply it themselves.

8. Domain-Specific Mitigation Prompts

Measured impact: the average across six models fell from 66% to 44% on planted-detail clinical tests, and GPT-4o from 53% to 23% ^[105]

Structured prompts that instruct the model to constrain outputs to verified information, flag uncertainty, and avoid speculation. These work best in narrow domains with clear boundaries and well-defined terminology.

The medical results are encouraging but the absolute rates remain high (44% on average with mitigation is still dangerously wrong for clinical use). Domain prompts are a layer, not a solution.

What Doesn't Work (Or Works Less Than Claimed)

Larger models alone: In Artificial Analysis's launch analysis, accuracy tracked model size and hallucination rate did not. The 2026 flagships confirmed it: on AA-Omniscience v1.1 the three most accurate models in our table, Claude Fable 5.1, Claude Opus 5.5 and GPT-6.1 Sol, all give a wrong answer more often than they decline (70.0%, 50.3% and 51.9%). Bigger models know more but don't necessarily know what they don't know.

Simple temperature reduction: Lowering generation temperature reduces variety but doesn't eliminate hallucination. The model still picks the most probable token. It just does so more consistently, including consistently wrong tokens.

"Be accurate" system prompts: Generic instructions to avoid hallucination show minimal measured effect. Models already "try" to be accurate. The issue is architectural, not motivational.

AI Hallucination Detection Tools

How Detection Tools Work

Most hallucination detectors now ship inside LLM evaluation and monitoring platforms. They use a second model, a trained classifier or formal rules to check an answer against a source. Only three have been tested side by side by an outside party. The rest publish their own numbers or none.

Detection Tools (October 2026)

What the Accuracy Gap Means

In AIMultiple's test of 100 answers (50 correct, 50 hallucinated), W&B Weave and Arize Phoenix were right 91% and 90% of the time overall. Accuracy is not the catch rate. Weave caught 43 of the 50 hallucinations (86%) and Phoenix 42 (84%), so about one in seven got through. Each also flagged 2 of the 50 correct answers. ^[46]

Comet Opik never flagged a correct answer, but it caught only 22 of the 50 hallucinations (44%). Pick the trade-off that fits the cost of a miss. In legal filings, medical decisions and financial reporting a miss costs more than a false alarm, so automated detection is one check among several, not the last one. The test is small, and AIMultiple lists Weights & Biases among its subscribers. ^[46]

Historical Progression

Two Years of Improvement on Vectara's Original Test

Vectara has published its hallucination leaderboard on GitHub since November 2023, and the repository keeps every update. We read all 257 versions of the leaderboard file to find the lowest rate at each update. On the original test, the best score fell from 3.0% (GPT-4, November 2023) to 0.6% (Ant Group's Finix-S1-32B, August 2025). Vectara then moved to a longer, harder dataset in November 2025. There the best score started at 3.3%, and it has stood at 1.8% since December 2025. ^[1]

On the same test, the best score in August 2025 was a fifth of the November 2023 best. The leaders were not always frontier models: Intel's 7B Neural Chat held the top spot in March 2024, Zhipu AI's 9B GLM-4 in August 2024 and Ant Group's 32B Finix-S1 from August 2025. On short-document summarization, the improvement is real.

The Reality Check

These improvements measure the easiest version of the problem: summarizing short documents without adding unsupported facts. When you move to harder, more realistic evaluations, the picture changes:

AA-Omniscience (hard knowledge questions): 52 of the 124 models on Artificial Analysis's v1.1 board (November 7, 2026) give more wrong answers than right ones. 72 have a positive Omniscience Index, and the three most accurate models in our table, Claude Fable 5.1, Claude Opus 5.5 and GPT-6.1 Sol, still give a wrong answer more often than they decline. On the v1 board of October 6, 2026, the split was 141 negative and 61 positive of 202. ^[2][88]

HalluHard (realistic conversations): Even the best entry on the board, GPT-6 Astra with web search, hallucinates 27.2% of the time, and Claude Opus 4.5 with web search 30.2%. Of its 41 entries, 10 are under 50%, 16 fall between 50% and 70%, and 15 are above 70%. ^[5]

Vectara new dataset (enterprise-length documents): Rates are higher than on the original dataset for most models, often several times higher: GPT-4.1 goes from 2.0% to 5.6% and GPT-5 (high) from 1.4% to 15.1%. The best score is 1.8% (finix_s1_32b), not 0.6%, and the best frontier-lab score is 3.1% (GPT-5.4 nano). Most frontier models sit between 5% and 14%. ^[1][100]

Domain-specific tasks: Purpose-built legal AI tools still err on more than 17% of queries (Stanford RegLab, 2025), and clinical tests with planted fake details reach 66% on average without a mitigation prompt. Specialized domains haven't shown the same improvement trajectory as general summarization. ^[36][105]

Improvement is real. But extrapolating from simple benchmarks to enterprise reliability is a mistake the data doesn't support.