AI Hallucination Rates,
Statistics & Benchmarks in 2026

The Benchmark Problem: Why the Numbers Contradict Each Other

Before looking at any hallucination data, you need to understand why different benchmarks give wildly different scores for the same model.

Grok-3 scores 2.1% on the Vectara summarization benchmark. Excellent. That same model scores 94% on the Columbia Journalism Review citation accuracy test. Catastrophic. Same model, same time period, opposite conclusions.

This isn't an error. It's measuring different things. And treating any single benchmark as "the hallucination rate" will mislead you.

The matrix below summarizes what each benchmark actually tests. Click any benchmark name to jump to its dedicated section.

Vectara AI Hallucination Leaderboard (HHEM)

Vectara's leaderboard is the most cited hallucination benchmark in the industry. It measures summarization faithfulness: given a source document, does the model's summary stick to what's actually in the document, or does it add unsupported facts? This makes it a direct proxy for how AI behaves in RAG pipelines, enterprise search tools, and document analysis workflows. The leaderboard exists in two versions, and the gap between them tells an important story. ^[1]

Original Dataset: ~1,000 Documents (April 2025)

This is the dataset most articles reference when they quote hallucination rates. The documents are relatively short and the summarization tasks are straightforward.

On this dataset, the numbers look encouraging. Ant Group's finix_s1_32b and two Gemini 2.0 models take the top three spots, with OpenAI's o3-mini-high tied for third. OpenAI's GPT models in this table sit between 1.2% and 2.0%. Every model in this table stays under 15%, though the full archived board runs up to 29.9% for older small models.

Correction, August 2026: an earlier version of this page listed finix_s1_32b at 1.8% on the original dataset. That is its rate on the new 7,700-article dataset, where it also leads. On the archived original dataset it scores 0.6%, one tenth of a point below Gemini-2.0-Flash. It is the first time a Chinese enterprise model has topped either Vectara table. Vectara froze the original dataset in October 2025, so no model released after that date will appear here. ^[100]

But this dataset is easy. The documents are short, the summarization tasks are clean, and the real world is neither.

New Dataset: 7,700 Articles (November 2025 to September 2026)

Vectara launched a refreshed benchmark in late 2025 with longer documents (up to 32K tokens) spanning law, medicine, finance, technology, and education. This version better reflects what enterprise AI systems actually face.

Rates rose for most models:

September 22, 2026 Update: GPT-6 Joins the Board

Vectara's latest update added three OpenAI models: GPT-6 Sol at 6.5%, GPT-6 Astra at 8.7% and GPT-5.6 Sol at 12.4%. Both GPT-6 models score better than GPT-5.6 Sol, in line with their lower hallucination rates on AA-Omniscience. No Claude, Gemini or Grok model released since May 2026 has a score yet. The board's latest Claude is Opus 4.7 (12.0%), its latest Gemini Pro is 3.1 Pro Preview (10.4%) and its latest Grok is Grok 4.1 Fast (19.2% with reasoning). Earlier snapshots are covered in our February 2026 research report. ^[1]

The Reasoning Tax

When Vectara launched the new dataset in November 2025, the thinking models it named, including GPT-5, Claude Sonnet 4.5 and Grok-4, all came in above 10%, and Gemini-3-Pro read 13.6%. Grok-4-fast-reasoning reads 20.2% on the current board. Several of OpenAI's newer models have since come in under 10% on the same dataset: GPT-6 Sol at 6.5%, GPT-5.4 at 7.0%, GPT-6 Astra at 8.7% and GPT-5.5 at 9.3%, though GPT-5.6 Sol reads 12.4%. The latest Claude and Gemini Pro entries still exceed 10%, at 12.0% for Claude Opus 4.7 and 10.4% for Gemini 3.1 Pro Preview. ^{[102][1][48][49]}

The hypothesis is straightforward. Reasoning models invest computational effort into "thinking through" answers. During summarization, this thinking leads them to add inferences, draw connections, and generate insights that go beyond what's in the source document. That's helpful for analysis. It's hallucination on a summarization benchmark.

This creates a critical decision for enterprise teams: reasoning mode helps on open-ended tasks and hurts on grounded tasks. Knowing when to enable it and when to turn it off is not optional.

AA-Omniscience Benchmark (Artificial Analysis)

AA-Omniscience asks a fundamentally different question than Vectara. Instead of "can you summarize without adding stuff," it asks "when you don't know something, do you admit it or make something up?" ^[2]

The benchmark covers 6,000 questions across 42 topics in six domains. The Omniscience Index (scale: -100 to +100) penalizes wrong answers and doesn't penalize refusal. This makes it the only major benchmark that explicitly rewards models for knowing their own limits.

How to read the three numbers together. Accuracy is the share of questions answered correctly. Hallucination rate is the share of non-correct responses that were wrong answers rather than refusals. The Index is roughly correct minus incorrect. So a model can post the highest accuracy while still fabricating most of the time it doesn't know. That is Claude Fable 5.1's profile: 67.2% accuracy and a 72.6% hallucination rate. On BenchLM's Artificial Analysis-synced board for October 6, 2026, 61 of 202 models have a positive Index. The other 141 give more wrong answers than right ones. ^[2][88]

Snapshot policy. Artificial Analysis re-runs models and adds effort configurations over time, so figures drift after launch, usually by a point or two and sometimes by much more. Both tables below were read from Artificial Analysis's own model records on October 4, 2026, including rows this page first published earlier. The release notes further down quote each model's launch figures, with the current record alongside where it has moved. ^[2]

Top 30 Models by Accuracy, With Hallucination Rates (October 2026)

Hallucination Rates Ranked, Lowest to Highest (October 2026)

All 56 models in the cross-benchmark table that have a current AA-Omniscience hallucination rate, with accuracy alongside. A low rate on its own can mean a model declines most questions, so read the two columns together.

How the Leaders Changed, Newest First

The notes below run from the newest frontier releases back to Gemini 3 Pro in November 2025. Each one quotes the figures Artificial Analysis published at the time. Where a model has since been re-scored, its current record is given as well, and the tables above always carry the current values.

Most of those re-scores happened on one day. On August 6, 2026, Artificial Analysis moved AA-Omniscience to a new grader, GPT-5.6 Luna in place of Gemini 3 Flash Preview, and says all its published results now reflect the change. All 20 models in the chart below read higher today than when their first figure was published, by about 2 points at the median, and about three quarters of that movement came with the grader change. The three largest moves on that day were Anthropic models: Claude Opus 5 (+10.7 points), Claude Fable 5 (+8.7) and Claude Opus 4.7 (+6.1). Compare launch figures with launch figures, and current records with current records. ^{[2][122][123]}

The September 2026 Releases Split the Field

September's frontier releases moved in different directions on AA-Omniscience. GPT-6 Astra (September 3) shows the biggest drop against a direct predecessor in this data: 62.6% accuracy and 51.3% hallucination at max, against 59.4% and 92.2% for GPT-5.6 Sol. OpenAI's flagship went from the highest flagship hallucination rate among the four US frontier labs to below both new Claude flagships, and gained accuracy doing it. Grok 4.7 (September 21) cut xAI's rate for the second release in a row, to 29.3% at 47.4% accuracy. Gemini 4 Argon, announced September 30 and so far limited to an initial group of cyber defenders, posts the best calibration of any high-accuracy model in the table: 15.1% hallucination at 49.9% accuracy, Index 42.4. ^[2]

Anthropic's top models mostly kept their answer-first profile. Claude Opus 5.5 (September 22) took the highest Index in the table, 46.4, on 66.2% accuracy, with a 58.6% hallucination rate, slightly below Opus 5's 60.8%. Claude Fable 5.1 (September 1) went the other way: the highest accuracy in the table, 67.2%, and a 72.6% hallucination rate, 9 points above Fable 5. The Index still rewards Anthropic's knowledge. On calibration alone, OpenAI, Google and xAI each now field a model that hallucinates less than Claude Opus 5.5 or Fable 5.1 while answering at least 47% of questions correctly. All figures in this subsection are Artificial Analysis records read on October 4, 2026. ^[2]

The other September releases mostly followed their labs' direction. GPT-6.1 Sol (September 29) improved on GPT-6 Sol (September 22) on both counts a week later: 62.1% accuracy and 54.3% hallucination, against 54.5% and 60.1%. Claude Sonnet 5.5 (September 28) lifted accuracy to 53.9% from Sonnet 5's 40.1%, and its hallucination rate rose from 39.4% to 47.0%. Gemini 3.8 Flash (September 2) cut the Flash line's rate to 55.2% from Gemini 3.7 Flash's 64.5% at almost the same accuracy, 54.6% against 55.3%. Muse Spark 1.3 and Qwen3.8 Max (0902), also out on September 2, stayed with the cautious models at 32.9% and 28.8%. ^[2]

Grok 4.6 Breaks the 2026 Pattern (August 2026)

Grok 4.6 shipped on August 12, 2026. On AA-Omniscience it posted 48.2% accuracy and a 34.3% hallucination rate, for an Index of 30.5, fourth on the board at the time behind Fable 5, Opus 5 and Gemini 3.1 Pro. Against Grok 4.5 (52% accuracy and 54% hallucination at launch, 51.6% and 54.1% on current records) that was a 20-point drop in hallucination for less than four points of accuracy. It was one of the few 2026 flagship releases to cut hallucination instead of buying accuracy with it, and Grok 4.7 repeated the move in September (29.3%). Artificial Analysis's records still read 48.2%, 34.3% and 30.5. ^[86][88][2]

One oddity worth knowing about. xAI's own model card reports the opposite direction on its internal factuality evaluation: Grok 4.6 hallucinates 1.7% at high effort where Grok 4.5 scored 0.98%, and its MASK-Rectified dishonesty score rose from 0.67% to 1.90%. Internal and independent benchmarks disagree not just on magnitude but on which way the model moved. The internal test is a different task on a different scale, so neither number is wrong. It is a reminder that "hallucination rate" without a benchmark name means nothing. ^[85]

Claude Opus 5 Moves the Opus Line (July 2026)

Claude Opus 5 launched on July 24, 2026 at $5 / $25 per million tokens with a 1M-token context window and four effort levels, and at launch posted 54% accuracy on AA-Omniscience, 7 points above Opus 4.8, for an Index of 31, third behind Fable 5 and Gemini 3.1 Pro. After Artificial Analysis's August 6 grader change its record reads 61% accuracy and Index 37, second only to Fable 5. Its Intelligence Index score of 63 was the highest of any Anthropic model at launch, one point above Fable 5. ^{[83][84][99][123]}

The calibration price was steep. At launch Artificial Analysis put Opus 5's hallucination rate at 50%, 14 points above Opus 4.8's 35.9%. After the August 6 grader change the two read 60.8% and 39.3%. Anthropic's launch post reports accuracy gains in chemistry, biology, finance and law, and says nothing about hallucination. ^[2][122][83]

So the Opus line split into two profiles. Opus 4.7 and 4.8 refuse rather than guess. Opus 5 answers like Fable 5, and Opus 5.5 kept that profile in September at 58.6%. If you are choosing an Anthropic model for the refusal profile, Opus 4.8 (39.3% on current records) and Sonnet 5 (39.4%) have the lowest rates of any current Claude apart from Haiku 4.5, which answers only 14.4% correctly.

The GPT-5.6 Sol Data Point (July 2026, Rate Published August 2026)

GPT-5.6 Sol, the flagship tier of OpenAI's Sol / Terra / Luna family (general availability July 9, 2026), posts an AA-Omniscience accuracy of 59.4% in its max configuration, third at the time behind Fable 5 and Opus 5. At launch Artificial Analysis described it only as "a small accuracy uplift coupled with an increase in hallucination rate" versus GPT-5.5, and this page withheld a number. The AA-synced board added one in August: 92.2% hallucination, Omniscience Index 22.0. When Sol doesn't know, it invents an answer more than nine times out of ten. That was the highest hallucination rate of any flagship from the four US frontier labs, above GPT-5.5's 86% at launch (89.0% on current records), and it left Sol outside the top ten on the Index despite third-place accuracy. GPT-6 Astra succeeded it in September at 51.3%. ^[73][88][2]

The two cheaper tiers followed the same shape. GPT-5.6 Terra: 46.8% accuracy, 87.9% hallucination, Index 0.1. GPT-5.6 Luna: 42.7% accuracy, 92.6% hallucination, Index -10.3. Across all three, OpenAI's mid-2026 models answered almost everything and admitted almost nothing, the pattern GPT-6 Astra broke in September. ^[88]

Claude Fable 5 Resets the Ceiling (June 2026, Re-Scored August 2026)

Claude Fable 5, Anthropic's first public Mythos-class model, launched on June 9, 2026 and immediately took the top of the AA-Omniscience Index at 40, seven points clear of Gemini 3.1 Pro, with a record 61% accuracy. Artificial Analysis tested it with adaptive reasoning, max effort and Opus 4.8 as the fallback model, and its numbers moved up when AA re-graded every model with a new grader on August 6, 2026: Index 43, 65% accuracy, ten points clear of Gemini 3.1 Pro and six clear of the new Claude Opus 5 at the time. Artificial Analysis's records still read 43 and 65% on October 4, 2026. In September, Claude Fable 5.1 (43.5), GPT-6 Astra (43.4) and Claude Opus 5.5 (46.4) passed it on the Index. ^{[72][2][122][88]}

Here is the twist that matters, and the re-score made it sharper. Fable 5's Index lead was driven by accuracy, not by calibration. Its hallucination rate is 63.6%, up from 54.9% at launch, far above Claude Opus 4.8's 39.3% and nowhere near the abstention-first models. It could still top the Index because the Index rewards correct answers and penalizes incorrect ones while ignoring refusals, and Fable 5's 65% correct-answer base is large enough to outweigh the penalty. The 63.6% applies only to its non-correct answers, not to every question. Roughly: it gets 65 of 100 questions right, and of the 35 it gets wrong or skips, it guesses wrong on 22 and admits it doesn't know on 13.

This is a real strategic shift for Anthropic, and it breaks the tidy "Claude refuses rather than guesses" story that held through Claude 4.1 Opus and Opus 4.7/4.8. That story still holds for Opus 4.7 and 4.8. It is not true of Fable 5, and, as the Opus 5 notes above show, it is not true of Opus 5 either. (A note on one stray figure: a 36.18% hallucination rate has circulated for Fable 5. That number belongs to Claude Opus 4.7, misattributed. Artificial Analysis's figure for Fable 5 is 63.6% as of October 2026, 54.9% at launch.)

The GPT-5.5 Data Point (April 2026)

GPT-5.5, released by OpenAI on April 23, 2026, posted the highest accuracy recorded on AA-Omniscience up to then, 57%. It also posted an 86% hallucination rate on the same benchmark, the most extreme accuracy-versus-calibration gap observed at the time: when GPT-5.5 didn't know an answer, it fabricated one 86% of the time. Artificial Analysis's current record reads 58.0% accuracy and 89.0% hallucination. The Gemini 3 Pro pattern (knowledge without self-awareness) intensified with that generation of high-capability models. ^[2][63]

Claude Opus 4.7, released by Anthropic on April 16, 2026, took the opposite tradeoff: a 36% hallucination rate at launch (42.3% on current records), with somewhat lower raw accuracy. The two release decisions, a week apart, were the clearest split up to then between prioritizing what a model knows and what a model knows about its own limits. ^[58][63][2]

Anthropic held that line with its next release. Claude Opus 4.8, which became the new flagship on May 28, 2026, launched with a 35.9% hallucination rate, 46.6% accuracy and an Omniscience Index of 27, at the time second only to Gemini 3.1 Pro. Artificial Analysis's current record reads 39.3%, 48.8% and 28.8. The numbers barely moved from Opus 4.7, which is the point: a full flagship generation passed with calibration held steady rather than traded away for raw knowledge. Early testers described the model as more likely to flag uncertainty and less likely to make unsupported claims. It was the last Opus to make that trade. ^[64][65][2]

The Gemini 3 Pro Paradox (November 2025)

Gemini 3 Pro told the first version of this story. At launch it posted the highest accuracy on the board, 53%, 14 points ahead of the next model. It also showed an 88% hallucination rate: when it didn't know an answer, it fabricated one 88% of the time rather than admitting uncertainty. Artificial Analysis's current record reads 55.8% accuracy and 91.5% hallucination. ^[2]

High knowledge + low self-awareness = a model that's brilliant when it's right and dangerous when it's wrong.

The Gemini 3.1 Pro update partially addressed this. At launch, Artificial Analysis measured its hallucination rate at 50%, down from 88% for Gemini 3 Pro, at about the same accuracy, because the model guessed less often when it did not know. Its Omniscience Index rose 17 points, the highest of any model until Claude Fable 5 arrived at 40 in June, since re-scored to 43. On current records the two read 91.5% and 50.9% hallucination, with Index values of 15.3 and 31.9. This proved that dramatic hallucination reduction is possible without meaningful accuracy sacrifice. ^[15][2]

Domain-Specific Leaders

No single model dominated all knowledge areas in Artificial Analysis's November 2025 launch analysis:

Source: Artificial Analysis AA-Omniscience, launch analysis, November 2025. Domain leaders have not been re-published for the 2026 flagships. ^[2]

Claude models led in domains where precise reasoning and citation accuracy matter. Grok led in domains where broad knowledge coverage matters. GPT led in business applications. This fragmentation is itself data. It means no single model is the safest choice for every professional use case.

One Statistic That Matters More Than the Rest

In Artificial Analysis's launch analysis, accuracy tracked model size. Hallucination rate did not. The 2026 releases have not broken that finding. The three most accurate models in our table, Claude Fable 5.1 (67.2%), Claude Opus 5.5 (66.2%) and Claude Fable 5 (65%), have hallucination rates of 72.6%, 58.6% and 63.6%: each still gives a wrong answer more often than it declines.

Throwing more parameters at the problem increases knowledge without increasing self-awareness. This is why the hallucination problem won't simply disappear with the next model generation.

FACTS Benchmark (Google DeepMind)

Google DeepMind's FACTS benchmark suite, published in December 2025, takes a different approach to most evaluations: instead of producing one hallucination score, it breaks factuality into four distinct dimensions and reports the overall as an average across them. This multi-dimensional view exposes that models have dramatically different strengths depending on task type. Grok 4 scores 75.3 on Search but just 25.7 on Multimodal, a 50-point gap within the same model. The figures here are the December 2025 Suite launch snapshot. The overall is the four-slice average (Gemini 3 Pro's 68.8 is the mean of its four slice scores, GPT-5's 61.8 the mean of its four). ^[3]

What the Four Slices Measure

Grounding: Can the model faithfully use information from provided documents? Tested through summarization and extraction tasks with source material.

Multimodal: Can the model accurately describe and reason about visual content alongside text?

Parametric: Does the model's internal knowledge (stored in its weights from training) produce correct answers without external tools?

Search: How accurate is the model when it has access to web search and retrieval tools?

Model Scores Across All Four Slices

The 2026 Kaggle Board (Updated September 28, 2026)

Google DeepMind now runs FACTS as a public leaderboard on Kaggle, and that is where the 2026 models are scored. GPT-6 Astra leads at 75.7, followed by GPT-6 Sol at 72.5 and Gemini 3.7 Flash at 71.2. Claude Opus 5.5 is the only Anthropic model in the top 10, at 65.3. Every launch-table model on the 2026 board scores lower there: GPT-5 reads 56.5 against 61.8 in the launch table, and Claude Opus 4.5 43.9 against 51.3. The board scores the search slice with a revised component, FACTS Search V2, which lists 20 models so far, so treat it as a separate scale. ^[3]

What This Data Reveals

No model broke 70% at launch. The best score in the December 2025 table was Gemini 3 Pro's 68.8, so every model was wrong more than 30% of the time on this multi-dimensional factuality evaluation. On the 2026 board GPT-6 Astra reaches 75.7, on a scale that is not directly comparable.

Search is the strongest slice for everyone. Gemini 3 Pro hits 83.8 and GPT-5 hits 77.7 on search-enabled factuality. When models can look things up, they're materially more accurate. When they rely on stored knowledge alone, accuracy drops. This matches the browse-on vs browse-off findings from OpenAI's system cards.

Grok 4 has a 50-point internal gap. It scores 75.3 on Search but 25.7 on Multimodal, a massive inconsistency that means it can find facts well but struggles with visual content. Any evaluation that averages these into a single score hides this gap.

Gemini 3 Pro's improvement is real. Compared to Gemini 2.5 Pro, Gemini 3 Pro reduced error rates by 55% on the Search slice and 35% on the Parametric slice. That's a large generation-over-generation improvement in factual accuracy, driven primarily by better search and grounding capabilities.

Frontier Model Hallucination Profiles

Every model below is profiled across multiple benchmarks. Single-benchmark comparisons mislead. The profiles show where each model is reliable and where it isn't.

Models added in the October 2026 update come first in each family below. Every AA-Omniscience figure below is the current record, read from Artificial Analysis on October 4, 2026, unless it is marked as a launch figure or an earlier board. Launch figures stay where the story depends on them, so you can see how a model's record moved after release.

To see any two of these models side by side on every benchmark, use the head-to-head comparison further down the page.

GPT Family (OpenAI)

GPT-6 Astra (September 3, 2026): the successor to GPT-5.6 Sol, and the biggest drop against a direct predecessor in this data. AA-Omniscience (max): 62.6% accuracy, 51.3% hallucination, Index 43.4, effectively tied with Claude Fable 5.1 for second in the cross-benchmark table. Against its predecessor, GPT-5.6 Sol, that is 40.9 points less hallucination and 3.2 points more accuracy. The high setting is better calibrated still: 61.1% accuracy, 44.8% hallucination, Index 43.7. Vectara new dataset: 8.7%, down from 12.4% for GPT-5.6 Sol. It leads the 2026 FACTS board at 75.7 and HalluHard with web search at 27.2% (updated question set), and places second on SimpleQA Verified at 75.8%. ^{[2][1][3][5][101]}

GPT-6 Sol, GPT-6.1 Sol and GPT-6 Luna: GPT-6 Sol (September 22) reads 54.5% accuracy, 60.1% hallucination and Index 27.1 at max, with 6.5% on Vectara's new dataset, lower than Astra's 8.7%. GPT-6.1 Sol followed a week later (September 29) and improved on both counts: 62.1% accuracy, 54.3% hallucination, Index 41.5. GPT-6 Luna reads 43.8% / 76.7% / 0.7. Each is better calibrated than its GPT-5.6 counterpart, all three of which sat above 87%. On the 2026 FACTS board GPT-6 Sol scores 72.5 and Luna 63.3. On SimpleQA Verified GPT-6.1 Sol scores 73.8% and GPT-6 Sol 64.4%. ^{[2][1][3][101]}

GPT-5.6 Sol / Terra / Luna (July 9, 2026): the generation before GPT-6, split into three tiers: Sol (flagship), Terra (balanced), Luna (fast, cost-efficient). Sol max posted 59.4% AA-Omniscience accuracy, third-highest on the August 2026 board, behind Claude Fable 5 and Claude Opus 5. Its hallucination rate, withheld at launch, is 92.2% on Artificial Analysis's current records, Index 22.0. That is the highest hallucination rate of any OpenAI flagship, 3.2 points worse than GPT-5.5 (89.0%). Terra reads 46.8% accuracy / 87.9% hallucination / Index 0.1, Luna 42.7% / 92.6% / -10.3. Sol scores 12.4% on Vectara's new dataset and 69.2% on SimpleQA Verified. One more reliability signal: independent evaluator METR recorded the highest reward-hacking (task-cheating) rate of any public model it had tested on Sol, fabricating results to appear finished. That is distinct from classic hallucination but shares the same root: completing over admitting limits. Use with strong verification for high-stakes factual work. ^{[73][2][1][101]}

GPT-5.5 (April 23, 2026): 57% AA-Omniscience accuracy paired with an 86% hallucination rate at launch, 58.0% and 89.0% on Artificial Analysis's current records. The clearest illustration of high knowledge with weak calibration until Sol extended the pattern. On the 2026 FACTS board it scores 66.7, and on Vectara's new dataset 9.3%. ^{[63][2][3][1]}

GPT-5.3 Instant (March 2026): reduces hallucination by 26.8% with web search and 19.7% without, relative to prior models, by OpenAI's own measure. ^[10]

GPT-5.2 (December 11, 2025): the professional workhorse. AA-Omniscience accuracy: 43.8% when first published here, 44.3% on current records (xhigh), with an 81.2% hallucination rate. With web search: 93.9% error-free responses. Without: error rate jumps to 12%. HalluHard: 38.2% with web search. Vectara new dataset: 10.8% (high). FACTS: 48.5 on the 2026 Kaggle board (GPT-5.2 was not in the December 2025 launch table). ^{[9][2][5][1][3]}

GPT-5 (August 7, 2025): Vectara old dataset: 1.4% (strong). Vectara new dataset: 15.1% at the high setting (weak). HealthBench thinking mode: 1.6%, one of the best medical hallucination scores recorded. SimpleQA without web: 47%. On browsing-enabled ChatGPT production traffic, 9.6% of factual claims from gpt-5-main contained an error (a claim-level production metric, not directly comparable to SimpleQA). AA-Omniscience on current records: 40.3% accuracy, 82.2% hallucination (high). FACTS overall: 61.8 in the December 2025 launch table, 56.5 on the 2026 Kaggle board. ^{[8][12][100][1][2][3]}

The pattern across the GPT-5 family: web search access is the single biggest variable. With browsing enabled, GPT-5 models compete for the lowest hallucination rates in the industry. Without it, rates jump several times over: GPT-5 thinking's FActScore error rate is 1.0% with browsing and 3.7% without. If you're deploying a GPT-5 variant, keep web access on. The GPT-5.5 and 5.6 generation shows the other half of the pattern: raw accuracy climbed to record highs while calibration got worse, not better. All three GPT-5.6 tiers sit above 87% hallucination on AA-Omniscience. GPT-6 reversed the trend: every GPT-6 model hallucinates less than its GPT-5.6 counterpart, and GPT-6 Astra gained accuracy at the same time.

Claude Family (Anthropic)

Claude Opus 5.5 (September 22, 2026): the highest AA-Omniscience Index in this table, 46.4, with 66.2% accuracy and 58.6% hallucination (Artificial Analysis's max run with fallback). Against Opus 5 that is 5.2 points more accuracy and 2.2 points less hallucination: a small step in the right direction, still inside the answer-first profile of Opus 5. On the 2026 FACTS board it scores 65.3, the best Anthropic result there. No Vectara score yet, and not on the public SimpleQA Verified board yet. ^[2][3]

Claude Fable 5.1 (September 1, 2026): the highest AA-Omniscience accuracy in this table at 67.2%, and the highest hallucination rate of any 2026 Claude at 72.6%, 9 points above Fable 5. Index 43.5, effectively tied with GPT-6 Astra for second in the table (max run with fallback). More knowledge, and less willingness to say "I don't know". No Vectara score yet, and not on the public FACTS or SimpleQA Verified boards yet. ^[2]

Claude Sonnet 5.5 (September 28, 2026): 53.9% accuracy, 47.0% hallucination, Index 32.3 (max run with fallback). Accuracy jumped 13.8 points over Sonnet 5, and the hallucination rate rose 7.6 points with it. SimpleQA Verified: 44.7%. ^[2][101]

Claude Fable 5 (June 9, 2026): Anthropic's first public Mythos-class model, and the AA-Omniscience Index leader at 43 (re-scored from 40 at launch) until September 2026, when Fable 5.1 (43.5), GPT-6 Astra (43.4) and then Opus 5.5 (46.4) passed it. Accuracy: 65% (a record until September). Hallucination rate: 63.6% on the August 2026 board and on Artificial Analysis's October 4 records, up from 54.9% at launch. The Index lead is accuracy-driven, not calibration-driven: a deliberate strategic shift toward answering more rather than refusing more. It falls back to Opus 4.8 on roughly 8-9% of questions, and the scored configuration includes that fallback. Best for complex reasoning and knowledge breadth where downstream verification exists. For a low-hallucination refusal profile, Opus 4.8 remains the safer Claude. HalluHard, on its updated question set: 38.9% with web search, 50.9% without. No Vectara score yet, and not on the public FACTS board yet. BullshitBench v2 clear-pushback rate: 54%, against 95% for Opus 4.8. ^{[72][2][5][94]}

Claude Opus 5 (July 24, 2026): the Opus flagship until Opus 5.5, $5 / $25 per million tokens, 1M-token context, effort levels from low to max. AA-Omniscience at launch: 54% accuracy and a 50% hallucination rate, against 46.6% and 35.9% for Opus 4.8 at its launch. Current records, after Artificial Analysis's August 6 grader change: Index 37 (second on the August board), 61% accuracy, 60.8% hallucination. Its Intelligence Index score of 63 was the highest of any Anthropic model at launch, one point above Fable 5. Read the two numbers together: Anthropic traded the Opus line's refusal profile for a double-digit accuracy gain. Anthropic's launch post reports large accuracy gains in chemistry, biology, financial modeling and legal work and does not report a hallucination figure. SimpleQA Verified: 63.3%, the best Anthropic score on that board. No Vectara score yet, and not on the public FACTS board yet. ^{[83][84][2][101]}

Claude Sonnet 5 (June 30, 2026): the default mid-tier model, built for autonomous execution and multi-step tool use. AA-Omniscience: 40.1% accuracy, 39.4% hallucination, Index 16.5, unchanged since the August 2026 board. Artificial Analysis lists Sonnet 5 (max) at Intelligence Index 55. Anthropic's pre-deployment evals report lower hallucination and sycophancy than Sonnet 4.6 and stronger prompt-injection resistance. BullshitBench v2: 80% clear pushback. HalluHard without web search: 59.7% on the updated question set. SimpleQA Verified: 32.1%. No Vectara score yet. ^{[74][2][99][94][5][101]}

Claude Opus 4.8 (May 28, 2026): the calibration-first flagship, superseding Opus 4.7 after 42 days and itself superseded by Opus 5 after 57. AA-Omniscience Index at launch: 27 (then second-highest overall, behind only Gemini 3.1 Pro's 33). Accuracy at launch: 46.6%. Hallucination rate at launch: 35.9%, essentially flat against Opus 4.7, which is the headline: a full flagship generation passed with calibration held steady rather than traded away for raw knowledge. On current records it reads 48.8% / 39.3% / Index 28.8, unchanged since the August re-pull and still the best-calibrated frontier Claude. It also took the top spot on the Artificial Analysis Intelligence Index at 61.4 when it launched, with SWE-bench Pro at 69.2% and Terminal-Bench 2.1 at 74.6%. Early testers reported it was more likely to flag uncertainty and less likely to make unsupported claims. BullshitBench v2: 95% clear pushback, the highest score on that board. On the 2026 FACTS board it scores 53.5, and on SimpleQA Verified 44.5%. ^{[64][65][2][94][3][101]}

Claude 4.1 Opus (August 5, 2025): the leader when AA-Omniscience launched in November 2025, with 36% accuracy and an Omniscience Index of 4.8, one of only three models above zero. Its hallucination rate was 48%, third lowest at the time, behind Claude 4.5 Haiku and level with Claude Sonnet 4.5. Artificial Analysis has since deprecated the model and no longer lists it on the AA-Omniscience board. Domain leader in Law, Software Engineering, and Humanities in the launch analysis. FACTS: 46.5 in the December 2025 launch table. Vectara: 4.2% on the original dataset, 11.8% on the new one. Earlier versions of this page gave it a 0% hallucination rate, a figure no source supports. ^{[2][104][3][100][1]}

Claude Opus 4.6 (February 5, 2026): AA-Omniscience on current records: 47.0% accuracy, 62.8% hallucination, Index 13.7. Vectara new dataset: 12.2%. On the 2026 FACTS board it scores 41.5, and on SimpleQA Verified 45.2%, second among Anthropic models after Opus 5. ^{[2][1][3][101]}

Claude Opus 4.5 (November 24, 2025): AA-Omniscience at launch: 58% hallucination, 43% accuracy. Current records: 46.6% accuracy, 61.0% hallucination, Index 14.0. HalluHard: 30.2% with web search, the best result on its original question set and second on the board behind GPT-6 Astra's 27.2% on the updated set, and 60.0% without search. FACTS: 51.3 in the December 2025 launch table, 43.9 on the 2026 Kaggle board. Vectara new dataset: 10.9%. ^{[14][2][5][3][1]}

Claude Sonnet 4.6 (February 17, 2026): AA-Omniscience hallucination on launch day: about 38%, down from Sonnet 4.5's 48%, at 24% accuracy. By April, after a re-run, Artificial Analysis's record read 46% at 40% accuracy. Current records: 48.4% at the max setting, against 49.1% for Sonnet 4.5 (reasoning), with accuracy at 40.9% against 32.9%. Users preferred Sonnet 4.6 over Opus 4.5 59% of the time, citing fewer hallucinations. Vectara new dataset: 10.6%. On the 2026 FACTS board it scores 37.0, and on SimpleQA Verified 30.9%. ^{[13][50][123][2][1][3][101]}

Claude Opus 4.7 (April 16, 2026): AA-Omniscience at launch: Index 26 (second-highest overall at release, behind only Gemini 3.1 Pro's 33) and a 36% hallucination rate, the strongest calibration profile of any frontier model attempting questions at scale at the time, and 50 percentage points better than GPT-5.5's launch figure. Current records: 48.9% accuracy, 42.3% hallucination, Index 27.3. BenchLM overall: 87 (April 2026). Long-context retrieval dropped to 32.2% (down from Opus 4.6's 78.3%). Anthropic explicitly attributes this to the model now reporting errors when information is missing rather than fabricating an answer. The refusal strategy made measurable. Vectara new dataset: 12.0%, the newest Claude on that board. On the 2026 FACTS board it scores 45.5. HalluHard without web search: 59.6%. ^{[58][63][2][1][3][5]}

The pattern across Claude: by Opus 4.7 and 4.8, Anthropic's models were calibrated to refuse rather than guess, trading raw accuracy for some of the lowest hallucination rates on knowledge benchmarks. That still describes Opus 4.7 and 4.8 (42.3% and 39.3% on current records) and Sonnet 5 (39.4%). Fable 5 and Opus 5 are the deliberate exceptions: they answer more, know more, and hallucinate more (63.6% and 60.8%), and their September successors kept that profile (Fable 5.1 at 72.6%, Opus 5.5 at 58.6%). Sonnet 5.5 sits between the two groups at 47.0%. BullshitBench v2 tells the same story from a different angle: Opus 4.8 pushes back on a nonsense premise 95% of the time, Sonnet 4.6 91%, Fable 5 54%. For applications where a wrong answer is worse than no answer (legal research, medical consultation, compliance work), the Opus 4.8 refusal profile is structurally safer than any of the 5.x flagships. ^[2][94]

Gemini Family (Google)

Gemini 4 Argon (announced September 30, 2026, not yet publicly available): Google is giving access to an initial group of cyber defenders first. AA-Omniscience (high): 49.9% accuracy, 15.1% hallucination, Index 42.4. That is the best calibration of any high-accuracy model in this table, and a clean break from the Gemini pattern below: against Gemini 3.1 Pro, 35.8 points less hallucination for 5 points less accuracy. No Vectara score yet, and not on the public FACTS or SimpleQA Verified boards yet. ^[2][103]

Gemini 3.8 Flash (September 2, 2026): AA-Omniscience (high) 54.6% accuracy, 55.2% hallucination, Index 29.6. Against Gemini 3.7 Flash, 9.3 points less hallucination at nearly the same accuracy. On the 2026 FACTS board it scores 69.4, and on SimpleQA Verified 74.6%, third on that board. ^[2][3][101]

Gemini 3.5 Flash-Lite: 29.5% accuracy, 34.4% hallucination, Index 5.2. Google's lightweight tier answers fewer questions correctly than the Flash models and guesses less often. FACTS 2026 board: 53.5. SimpleQA Verified: 33.1%. ^[2][3][101]

Gemini 3.7 Flash (August 13, 2026): Google's workhorse until 3.8 Flash, shipped three weeks after 3.6 Flash as a refinement rather than a new pretraining run. AA-Omniscience: 55.3% accuracy, 64.5% hallucination, Index 26.5, unchanged since the August 2026 board. That is Gemini-typical: near the top on knowledge, weak on admitting gaps, though far better calibrated than Gemini 3 Flash's 93.0%. Google's launch post reports coding and agent benchmarks and no factuality figure. No Vectara score yet. On the 2026 FACTS board on Kaggle it scores 71.2, third behind GPT-6 Astra and GPT-6 Sol, and on SimpleQA Verified 73.2%. ^{[87][2][3][101]}

Gemini 3.1 Pro Preview (February 19, 2026): AA-Omniscience Index 30 at launch (33 after a re-run by April), third on the August 2026 board behind Claude Fable 5 (43) and Claude Opus 5 (37), and still Google's best public model on that measure. Accuracy at launch: 53%. Hallucination rate: 50%, down from Gemini 3 Pro's 88%. Current records read 54.9% accuracy, 50.9% hallucination and Index 31.9, against 91.5% hallucination for Gemini 3 Pro. This was the biggest single-update hallucination improvement until GPT-6 Astra in September 2026. It tops SimpleQA Verified at 77.5% and scores 67.4 on the 2026 FACTS board. Vectara new dataset: 10.4%. HalluHard without web search: 57.1%. BullshitBench v2: 37% clear pushback, the lowest of the frontier flagships. ^{[15][2][101][3][1][5][94]}

Gemini 3 Pro (November 18, 2025): FACTS overall: 68.8 (highest in the December 2025 launch table). FACTS Search: 83.8. FACTS Parametric: 76.4. AA-Omniscience at launch: 53% accuracy, then the highest of any model, with 88% hallucination. Current records: 55.8% and 91.5%. Vectara new dataset: 13.6% (preview). SimpleQA Verified: 72.1% (preview). The Gemini Paradox at launch: the most knowledgeable model, and one of the least self-aware. ^{[3][2][1][101]}

Gemini 3 Flash (December 2025): AA-Omniscience at launch: 55% accuracy, the highest of any model tested at the time, and a 91% hallucination rate. Current records: 53.4% and 93.0%. Speed: 218 tokens/s. The most extreme version of the Gemini Paradox, brilliant and unreliable in equal measure. Suitable only for tasks with external verification. The May 2026 successor, Gemini 3.5 Flash, addresses this directly. ^[16][2]

Gemini 3.5 Flash (May 19, 2026): AA-Omniscience hallucination at launch: 61%, down 31 percentage points from the 92% Artificial Analysis then listed for Gemini 3 Flash. Current records: 51.4% accuracy, 62.2% hallucination, Index 21.2 (high), against 93.0% for Gemini 3 Flash. That single generation cut the most extreme version of the Gemini Paradox by roughly a third and lifted the Omniscience Index 11 points. MMMU-Pro multimodal: 84%, the highest score recorded on that evaluation at launch. Intelligence Index: 55. On the 2026 FACTS board it scores 63.1, and on SimpleQA Verified 70.4%. One caveat worth stating plainly: unlike Anthropic and OpenAI, Google has not published first-party hallucination data for this model. The 61% figure is independent benchmarking from Artificial Analysis, not a vendor disclosure. ^{[64][66][67][68][2][3][101]}

The pattern across Gemini: Gemini 3 Pro and 3 Flash attempted nearly every question, which gave them top accuracy scores but catastrophic hallucination rates when they hit their knowledge limits. The 3.1 Pro update showed this is addressable: by guessing less often, it cut the hallucination rate 40.6 percentage points on current records (91.5% to 50.9%) for under one point of accuracy. Gemini 3.5 Pro, announced at Google I/O in May, was never publicly released. When Google announced Gemini 4 Argon on September 30, a spokesperson told Reuters the company no longer plans to release it. Since May, Google's public releases have been Flash-only: 3.5 Flash in May, 3.6 Flash in July, 3.7 Flash in August and 3.8 Flash in September. Gemini 4 Argon, so far limited to an initial group of cyber defenders, breaks the pattern at 15.1% hallucination. ^[98][103][2]

Grok Family (xAI)

Grok 4.7 (September 21, 2026): AA-Omniscience (xhigh) 47.4% accuracy, 29.3% hallucination, Index 32.0. The second release in a row in which xAI cut the rate: 5 points below Grok 4.6, for under one point of accuracy. Configurations differ: Artificial Analysis lists Grok 4.7 at xhigh and Grok 4.6 at high. Among public models that answer at least 45% of questions correctly, it is now the best calibrated in this table. SimpleQA Verified: 59.4%, the best xAI score on that board. ^[2][101]

Grok 4.6 (August 12, 2026): xAI's flagship until Grok 4.7, $2/$6 per million tokens, 500K context, Intelligence Index 61 (tied with GPT-5.6 Sol max). AA-Omniscience: 48.2% accuracy, 34.3% hallucination, Index 30.5, unchanged since the August 2026 board, where it ranked fourth. Against Grok 4.5's current record that is 19.8 points less hallucination for 3.4 points less accuracy. xAI's own model card points the other way: 1.7% on its internal factuality evaluation at high effort versus 0.98% for Grok 4.5, MASK-Rectified dishonesty 1.90% versus 0.67%. Internal and independent measurements disagree on direction, not just size. SimpleQA Verified: 52.9%. Strong candidate for the verification seat in a multi-model setup, still not a sole model for high-stakes factual work. ^{[85][86][2][101]}

Grok 4.5 (July 8, 2026): released under the SpaceXAI brand. AA-Omniscience accuracy jumped to 52% at launch (from Grok 4.3's 35%), but the hallucination rate more than doubled, from 25% to 54%. The Index still rose 8 points to 26 because accuracy climbed faster than the penalty. Current records: 51.6% accuracy, 54.1% hallucination, Index 25.3. It ranked 4th on the Artificial Analysis Intelligence Index at 54 at launch. Pricing $2/$6 per million tokens, 500K context. A textbook case of buying knowledge at the cost of calibration, and, five weeks later, the baseline that Grok 4.6 corrected. xAI's model card for 4.6 gives 4.5 a 0.98% score on the internal factuality evaluation. FACTS 2026 board: 50.2. SimpleQA Verified: 53.8%. ^{[75][85][2][3][101]}

Grok 4.3 (April 2026): AA-Omniscience in two configurations. High: 34.8% accuracy, 25.8% hallucination, Index 18.0. Medium: 28.8% accuracy and 16.9% hallucination, a lower-attempt setting that edges the older Grok 4.20 Reasoning (17.4%) for the lowest-hallucination Grok. API pricing $1.25/$2.50 per million tokens. ^[69][2]

Grok 4 (July 9, 2025): Vectara old dataset: 4.8%. AA-Omniscience at launch: 39% accuracy, the highest at the time and level with GPT-5 (high), with a 64% hallucination rate, one of only three models with a positive Index. Current records: 40.5%, 64.5%, Index 2.1. FACTS: 53.6 in the December 2025 launch table (Search: 75.3, Multimodal: 25.7). Domain leader in Health and Science in the AA-Omniscience launch analysis. ^{[100][104][2][3]}

Grok 4.1 Fast: xAI claims a 65% hallucination reduction (12.09% to 4.22% on internal benchmarks). AA-Omniscience tells a different story: 72% hallucination at launch and 73.4% on current records, worse than Grok 4's 64.5%. Vectara new dataset: 19.2% with reasoning, the 9th-highest rate of the 108 models on that board. Sycophancy also increased (MASK benchmark: 0.07 to 0.19-0.23). ^[17][2][1]

Grok-3 (February 17, 2025): Columbia Journalism Review: 94% citation hallucination rate, by far the worst score in that study. Vectara: 2.1% on the original dataset (as Grok-3-Beta) and 5.8% on the new one, the best xAI score on the current board. SimpleQA Verified: 39.3%. ^{[6][100][1][101]}

The pattern across Grok: internal benchmarks and independent benchmarks disagree sharply, and with Grok 4.6 they disagree on direction. AA-Omniscience says 4.6 cut hallucination to 34.3% (down from 4.5's 54.1%), and 4.7 cut it again to 29.3%. xAI's own factuality evaluation says 4.6 got slightly worse (1.7% versus 0.98%). Grok 4.5 remains the textbook case of buying knowledge at the cost of calibration: more knowledgeable than Grok 4.3 and hallucinating more than twice as often. The 94% CJR citation hallucination rate comes from Grok-3, tested in March 2025, and no newer Grok has been through that study. Domain-specific value exists in Health and Science, and the calibration of 4.6 and 4.7 makes them stronger second voices than any Grok before them. The gap between vendor and independent numbers still makes Grok risky as a sole model for any high-stakes application.

Perplexity Sonar (Perplexity AI)

Sonar Reasoning Pro (2025): Search Arena score: 1136, statistically tied with Gemini 2.5 Pro for #1. SimpleQA F-score: 0.858, the highest of any model at time of testing. CJR citation accuracy: 37% hallucination (best tested). Response accuracy: >90% for factual queries (94% overall, 95% academic, 94% technical). ^[18][19]

Sonar Pro (2025): Built on Llama 3.3 70B, fine-tuned for search factuality. SimpleQA F-score: 0.858. Outperforms GPT-4o and Claude 3.5 Sonnet on factuality benchmarks. ^[19]

The Perplexity risk: Perplexity introduces a failure mode no other model shares. It cites real URLs with fabricated claims. The sources look legitimate, with real websites and real publication names, but the information attributed to those sources may be invented. This makes Perplexity hallucinations harder to detect than hallucinations from models that don't present external citations. A 37% citation hallucination rate means more than one in three source attributions may contain fabricated content. ^[51]

DeepSeek (DeepSeek AI)

DeepSeek V4.1 Flash: the highest hallucination rate in this table, 96.5% at 46.4% accuracy (max), Index -5.3. Against V4 Flash, accuracy rose almost 10 points and the hallucination rate barely moved. ^[2]

DeepSeek V4 Pro and V4 Flash (preview April 24, 2026): a cautionary data point. At launch the V4 Pro preview posted a 94% AA-Omniscience hallucination rate and an Omniscience Index of -10, and V4 Flash hit 96% with an Index of -23, among the highest ever recorded on the benchmark, in the same territory as Gemini 3 Flash (93.0% on current records) and Llama 4 Maverick (88.9%). Current records: V4 Pro preview 43.0% accuracy, 94.1% hallucination, Index -10.7. V4 Flash 36.8%, 96.1%, Index -23.9. Strong coding performance paired with near-total overconfidence on knowledge questions. Vectara new dataset: 8.6% for V4 Pro. ^[70][2][1]

DeepSeek V4 Pro 0813 (general availability August 13, 2026): the GA build that took V4 Pro out of preview. AA-Omniscience on current records: 49.1% accuracy, 94.8% hallucination, Index 0.8. Accuracy rose 6.1 points over the preview and the hallucination rate barely moved, which is why the Index climbed from -10.7 to about zero: DeepSeek added knowledge without adding any willingness to say "I don't know". The two builds are scored separately on this page because the checkpoints differ. ^[92][88][2]

DeepSeek-V3 (December 26, 2024): Vectara: 3.9% on the original dataset, 6.1% on the new one. A strong performer on grounded summarization. ^[100][1]

DeepSeek-R1 (January 20, 2025): Vectara old dataset: 14.3%, nearly 4x higher than V3, and 11.3% on the new dataset. AA-Omniscience (0528 build): 83.4% hallucination at 30.5% accuracy. Vectara analysis found R1 produces 71.7% "benign hallucinations" (plausible-sounding additions) vs V3's 36.8%. ^{[49][48][100][1][2]}

The pattern: DeepSeek's reasoning model (R1) hallucinates dramatically more than its base model (V3). This is the reasoning tax in its most extreme form. The gap (3.9% vs 14.3%) makes it one of the clearest examples that reasoning capabilities and factual reliability don't move in the same direction. On Vectara's new dataset the gap narrows (6.1% vs 11.3%) but holds.

Open-Weight and Regional Models

Qwen3.8 Max (0902) (Alibaba, September 2, 2026): 31.7% accuracy, 28.8% hallucination, Index 12.0. The September build pulled the hallucination rate back down from the preview's 41.7% at the same accuracy, close to the abstention-first profile of Qwen3.7 Max. ^[2]

Muse Spark 1.3 (Meta, September 2, 2026): 43.6% accuracy, 32.9% hallucination, Index 25.0 at max. Close to Muse Spark 1.2 (xhigh) on calibration, 1.8 points lower on accuracy. ^[2]

Kimi K3 (Moonshot AI, July 2026, open weights released July 27): a 2.8-trillion-parameter open-weights MoE model with a 1M-token context window, the first open model in the 3T class. AA-Omniscience: 46% accuracy, 51% hallucination, Index 18 at launch (47.6% / 53.2% / 19.7 on current records, unchanged since the August re-pull). Like Grok 4.5, its accuracy gain over the prior generation came with a calibration regression. Ranks among the top open-weight models on the Artificial Analysis Intelligence Index. ^[76][93][2]

Command A+ (Cohere, open weights, May 20, 2026): the lowest hallucination rate in our table at 14% (and the lowest on Artificial Analysis's board when it launched in May 2026), reached by refusing most questions it can't verify (9% accuracy, Index -4). A "knows-its-limits" archetype similar to Claude 4.5 Haiku, useful as a verification layer in a multi-model pipeline. ^[77][2]

Qwen3.7 Max (Alibaba, May 19, 2026): an abstention-first model that refuses whenever its internal confidence threshold isn't met. At launch: 30% accuracy, 23% hallucination, Index 14. Current records: 31.1%, 25.6%, Index 13.5. It has a 1M-token context window, and its refusal habit makes it another strong verification-layer candidate. ^[78][2]

Qwen3.8 Max Preview (Alibaba, previewed July 19, 2026, launched August 3, the build Artificial Analysis scores): a 2.4-trillion-parameter MoE flagship with open weights promised. AA-Omniscience: 31.9% accuracy, 41.7% hallucination, Index 3.4, unchanged since the August 2026 board. Accuracy barely moved from Qwen3.7 Max, and the hallucination rate rose from 25.6% to 41.7%: Alibaba loosened the abstention policy that made 3.7 Max a verification-layer candidate, and got almost no knowledge in return. The clearest recent example of calibration being a training choice, not a capability. ^[90][2]

GLM-5.3 (Z.ai, August 14, 2026): a coding and cyber-security flagship built on the GLM-5.2 base without retraining. AA-Omniscience: 33.9% accuracy, 29.6% hallucination, Index 14.3, unchanged since the August 2026 board. That makes it the best-calibrated new open-weight model of the summer, in the Qwen3.7 Max and Grok 4.3 band. ^[91][2]

Llama 4 Maverick (Meta, April 5, 2025): Vectara: 4.6% on the original dataset (competitive) and 8.2% on the new one. AA-Omniscience: 88.9% hallucination on current records (87.6% when first published here), at 24.9% accuracy. The gap between grounded summarization and open-ended knowledge is wider for open-source models than for any proprietary family. ^[100][1][2]

In a 2025 clinical test with planted fake details, a distilled DeepSeek-R1 model built on the fake detail 80.0-82.7% of the time, while Llama-3.3-70B did about as well as GPT-4o. For critical applications, the gap between open and proprietary models depends on the model and the task. ^[105]

Muse Spark 1.1 (Meta, July 9, 2026): carries an AA-Omniscience profile, not just a HealthBench score. July launch configuration: Index 18 (up from Muse Spark 1.0's 4), accuracy 41% (roughly flat), hallucination 38% (down from 73%). The entire gain came from abstention: attempt rate dropped from 95% to 82%. It is the clearest recent example of a model getting more reliable by learning when to stay silent rather than by learning more. The xhigh configuration in our table reads 52.1% accuracy / 50.0% hallucination / Index 28.1: more knowledge, and the abstention discipline gone. HealthBench Professional: 59.3%. Its HealthBench Hard leadership (42.8) belongs to the original Muse Spark. A 1.1-specific Hard score is not yet confirmed. ^[71][88][2]

Muse Spark 1.2 (Meta, August 5, 2026): a coding-focused update to 1.1, co-trained with Meta's Muse Code agent. AA-Omniscience (xhigh): 45.4% accuracy, 33.3% hallucination, Index 27.2, unchanged since the August 2026 board. Set against the xhigh configuration of 1.1, that is a 16.7-point cut in hallucination for 6.7 points of accuracy. Among public models that answer at least 45% of questions correctly, only Grok 4.7 is better calibrated. Meta's launch post reports coding benchmarks only and no factuality figure. ^[89][88][2]

Head-to-Head Model Comparisons

The model profiles in Section 6 show individual performance. This section answers the questions people actually search for: "Is Claude or GPT more accurate?" "Should I use Gemini or Claude?" The answer depends on what you're doing, but the data makes the tradeoffs specific. In the Claude, GPT, Gemini and Grok tables every cell names the model behind the figure, and most rows compare the best result each family posts on a benchmark. Current figures come from Artificial Analysis's October 4, 2026 records, Vectara's September 22 board, the FACTS Kaggle board of September 28, the SimpleQA Verified board of October 5 and HalluHard's October board. Older results are labeled as such: Vectara's original dataset (archived October 2025), the December 2025 FACTS launch table, HealthBench (August 2025), the CJR citation study (March 2025) and the Perplexity comparison, which dates from 2025.

Claude vs GPT

The most searched comparison in AI, and the most context-dependent. Both labs shipped new flagships in September. GPT-6 Astra reads 62.6% AA-Omniscience accuracy and 51.3% hallucination. Claude Opus 5.5 reads 66.2% and 58.6%, and Claude Fable 5.1 67.2% and 72.6%. Claude's newest models know more. GPT-6 Astra is better at declining what it doesn't know. All three still give a wrong answer more often than they decline. ^[2]

The pattern isn't "one is better." It's two different philosophies measured on different scales.

GPT models are stronger when the task has source material to work from. Summarization, document analysis, RAG pipelines, search-grounded Q&A: GPT sticks closer to provided text and scores well on faithfulness benchmarks. OpenAI holds the best frontier-lab score on Vectara's new dataset (GPT-5.4 nano, 3.1%, against 9.8% for the best Claude), and GPT-6 Astra leads the 2026 FACTS board at 75.7, 10.4 points above Claude Opus 5.5. GPT also leads on short factual questions answered from memory, without tools: GPT-6 Astra scores 75.8% on SimpleQA Verified, against 63.3% for Claude Opus 5, the best Claude on that board. ^[1][3][101]

Claude models are stronger when the task requires the model to know its own limits, but only some of them. Claude Opus 4.8 gives a wrong answer on 39.3% of the questions it doesn't answer correctly, the best calibration of any Claude or GPT model that answers at least 45% of questions correctly. Anthropic's newest flagships went the other way: Opus 5.5 reads 58.6% and Fable 5.1 72.6%, both above GPT-6 Astra's 51.3%. When AA-Omniscience launched in November 2025, Anthropic held the three lowest hallucination rates, led by Claude 4.5 Haiku. HalluHard's realistic conversation test splits by question set. On the updated set, GPT-6 Astra with web search (27.2%) beats Claude Fable 5 (38.9%). On the original set, Claude Opus 4.5 (30.2%) edges GPT-5.4 Thinking (31.9%). ^[2][5]

One more variable that often gets overlooked: web search access. OpenAI's GPT-5 numbers show it. Its hallucination rate on SimpleQA without web access was 47%, while 9.6% of its factual claims in browsing-enabled ChatGPT traffic contained an error, a different metric that points the same way. HalluHard shows the effect for both labs on the same test: GPT-6 Astra drops from 43.2% without search to 27.2% with it, and Claude Fable 5 from 50.9% to 38.9%. Whichever model you use, turn search on for factual work. ^[8][12][5]

Claude vs Gemini

For eighteen months the shorthand was "Gemini knows more, Claude is more honest about what it doesn't know." Both halves have moved. On AA-Omniscience, Claude now leads on knowledge: Fable 5.1 answers 67.2% of questions correctly, against 55.8% for the best Gemini, Gemini 3 Pro. SimpleQA Verified points the other way: Gemini 3.1 Pro Preview scores 77.5%, 14.2 points above Claude Opus 5, the best Claude on that board. Gemini now leads on calibration, but only with a model you can't use yet. Gemini 4 Argon's hallucination rate is 15.1%, about a quarter of Claude Opus 5.5's 58.6%. ^[2][101]

Among models you can use today, the old split still holds. Claude Opus 4.8 gives a wrong answer on 39.3% of the questions it doesn't answer correctly, Gemini 3.1 Pro on 50.9% and Gemini 3.8 Flash, Google's newest public model, on 55.2%. Anthropic's newest flagships sit above all three: Opus 5.5 at 58.6% and Fable 5.1 at 72.6%. The "more honest" half of the shorthand now holds for Opus 4.7, Opus 4.8 and Sonnet 5.5, the Claude models that answer at least 45% of questions correctly and still hallucinate less than Gemini 3.1 Pro, not for Claude as a whole. ^[2]

Gemini also still leads on grounded summarization, 0.7% on Vectara's original dataset and 3.3% on the new one, and on both FACTS tables, where Gemini 3.7 Flash scores 71.2 on the 2026 board and Gemini 3 Pro led the December 2025 launch table at 68.8. ^[100][1][3]

GPT vs Gemini

GPT vs Gemini changed most in September. GPT-6 Astra widened OpenAI's lead on accuracy (62.6% against Gemini 3 Pro's 55.8%) and took the lead on the 2026 FACTS board (75.7 against Gemini 3.7 Flash's 71.2). Among public models that answer at least 45% of questions correctly, GPT-6 Astra at max, the setting in our table, is level with Gemini 3.1 Pro on calibration, 51.3% against 50.9%, and its high setting does better at 44.8%. Gemini 4 Argon, at 15.1%, is ahead of both, but it is not yet publicly available. Gemini keeps a narrow lead on SimpleQA Verified, 77.5% against 75.8%, inside the board's confidence intervals, and still leads the December 2025 FACTS launch table. GPT's other advantages are task-specific: medical applications (1.6% HealthBench), the best frontier-lab score on Vectara's new dataset (GPT-5.4 nano, 3.1%), and the sheer volume of internal evaluation data OpenAI publishes. ^{[2][3][101][1]}

Grok vs the Field

xAI's internal numbers and the independent ones have never agreed on size. With Grok 4.6 they stopped agreeing on direction. AA-Omniscience says 4.6 cut the hallucination rate to 34.3% from Grok 4.5's 54.1%, and Grok 4.7 cut it again to 29.3%, the lowest of any public model that answers at least 45% of questions correctly. xAI's own model card says 4.6 got slightly worse on its internal factuality evaluation (1.7% versus 0.98%). Both can be true, because they measure different things on different scales. ^[2][85]

Calibration is where Grok now leads. Knowledge is where it trails: Grok 4.7 answers 47.4% of AA-Omniscience questions correctly, 19.8 points behind Claude Fable 5.1, and places 11th of 50 on SimpleQA Verified. The older Grok 4.1 Fast is the cautionary case: 73.4% hallucination on AA-Omniscience and 19.2% on Vectara's new dataset, the 9th-highest rate of 108 models there. The CJR citation study found Grok-3 hallucinated 94% of the time on news source attribution, the worst of the eight tools tested. ^{[2][101][1][6]}

Grok does have genuine domain strengths: it led Health and Science categories in the AA-Omniscience launch analysis. But the gap between xAI's claims and independent measurements is wider than for any other provider.

Perplexity vs ChatGPT vs Claude

In the 2025 tests, Perplexity won on factual search queries. Its RAG-native architecture, built around retrieval rather than parametric knowledge, gives it a structural advantage for questions with verifiable answers. The current search-enabled boards have no Perplexity entry: HalluHard's best search-enabled score is GPT-6 Astra's 27.2%. ^[5]

The catch: Perplexity cites real URLs with fabricated claims. The sources look legitimate, with real websites and real publication names, but the information attributed to those sources may be invented. At a 37% citation hallucination rate, more than one in three source attributions could contain fabricated content. This makes Perplexity hallucinations harder to spot than hallucinations from models that don't present external citations.

Domain-Specific Hallucination Rates

Hallucination rates vary dramatically by subject matter. A model that looks reliable on general-knowledge benchmarks can be dangerously wrong on legal or medical questions. The domain with the strongest primary research is law.

Legal: Purpose-Built Tools Still Fail

Stanford RegLab and Stanford HAI tested the paid legal research tools marketed as "hallucination-free" (Magesh, Surani, Dahl, Suzgun, Manning and Ho, Journal of Empirical Legal Studies, April 2025). Lexis+ AI and Ask Practical Law AI gave incorrect or misgrounded answers on 17% of queries, Westlaw AI-Assisted Research on 33% and GPT-4 on 43%. Stanford HAI's summary puts the Westlaw figure at more than 34%. General-purpose chatbots did worse on specific questions about federal court cases, with hallucination rates from 58% (GPT-4) to 88% (Llama 2) (Dahl, Magesh, Suzgun and Ho, Journal of Legal Analysis, 2024). Both studies tested 2023-2024 product versions, not today's. ^[36][96][97] A newer test by Vals AI (October 14, 2025) put 200 U.S. legal research questions to three legal AI tools and ChatGPT. Where they answered, the legal tools were fully correct on 78-81% of questions and ChatGPT on 80%. Lexis+ AI and Westlaw were not in the test. ^[112] Domain-specific reliability claims need domain-specific checks.

Legal: The Courtroom Crisis

Court decisions involving AI-hallucinated material rose sharply through 2025 and have stayed high in 2026.

Damien Charlotin's database lists 2,149 court decisions worldwide in which a party relied on AI-hallucinated material, 1,473 of them in U.S. courts (last updated October 5, 2026). By decision date, it shows 16 decisions in 2023, 61 in 2024 and 855 in 2025, plus 1,217 from January 1 to October 1, 2026. Older years keep growing as past rulings are found. In May 2025 the same database showed 10, 37 and 73 for 2023, 2024 and the first five months of 2025. New decisions have run at about 140 a month in 2026, close to five a day, and the database lists 24 decisions dated March 31, 2026 alone. The live count on any later day will be higher than the one printed here. ^[37][38]

Self-represented litigants still account for most cases, 1,233 of the 2,149, but lawyers appear in 854 (database filters, October 5, 2026). In May 2025, Business Insider reported legal professionals at fault in at least 13 of 23 recent cases. In February 2025, Morgan & Morgan, one of the largest U.S. personal injury firms, emailed its more than 1,000 lawyers that AI can invent case law and that filing fake citations could cost them their jobs. Days later a federal court in Wyoming fined the firm's lawyers in Wadsworth v. Walmart. ^{[37][38][114]}

Penalties have grown. The largest U.S. amount in the database is US$110,204 in Couvrette v. Wisnovsky (D. Oregon, March 23, 2026), a US$15,500 sanction plus about US$94,700 in costs. By our tally of the database's U.S. dollar amounts, the highest quarter so far is Q4 2025, at about US$391,000 across 42 decisions, followed by Q1 2026 at about US$325,000 across 35. These totals include costs awards, not only fines. The Fourth Circuit publicly admonished a lawyer on March 11, 2026 (In re Nwaubani). Monthly counts rose through 2025 and have held at roughly 115-180 decisions a month in 2026. ^[37]

The benchmark data explains why. On the central holding of a case, the published Dahl study found hallucinations in at least 63% of answers, a lower bound. Stanford HAI's January 2024 summary of the earlier preprint, before GPT-4 was added, gave at least 75%. ^[36][97]

Healthcare: Where Hallucinations Can Kill

ECRI, the patient safety nonprofit, ranked risks from AI-enabled health technologies as the top health technology hazard for 2025. For 2026 it ranked misuse of AI chatbots first, citing chatbots that suggested wrong diagnoses, recommended unnecessary tests and invented body parts while sounding like a trusted expert (January 21, 2026). ^[39][108]

The FDA's public list of AI-enabled medical devices holds 1,614 authorized devices, up from 637 through 2022 (list current as of September 4, 2026). A separate study of 950 of these devices found 60 (6.3%) linked to 182 recall events, with 43.4% of recalls within 12 months of clearance (Lee et al., JAMA Health Forum, August 22, 2025). Most recalls were for diagnostic or measurement errors, not chatbot-style hallucination. ^[107][106]

A 2025 study planted one made-up detail, such as a fake lab value or condition, in each of 300 physician-checked clinical vignettes and asked six models to respond (Omar et al., Communications Medicine, August 2, 2025). The models built on the fake detail in 50-82% of cases. A mitigation prompt cut the average from 66% to 44%. GPT-4o did best, falling from 53% to 23%. A distilled DeepSeek-R1 model did worst, at 80.0-82.7% by default. ^[105]

OpenAI reports that GPT-5 with thinking hallucinated on 1.6% of its HealthBench Hard hallucination test, against 3.6% without thinking, 12.9% for o3 and 15.8% for GPT-4o (August 7, 2025). This is a vendor-run test. ^[109]

HealthBench Professional (April 2026): OpenAI's newer benchmark scores models on 525 physician-written tasks from real clinician use, across care consult, writing and documentation, and medical research. In OpenAI's paper, GPT-5.4 inside ChatGPT for Clinicians scored 59.0, base GPT-5.4 48.1 and specialist physicians with web access 43.7. On consult tasks the gap was 51.0 to 42.7. OpenAI built and graded the benchmark, and picked examples that are hard for its own models. ^[60] Separately, on HealthBench Hard, the hard subset of the original HealthBench, Meta reported Muse Spark at 42.8 at its April 8, 2026 launch. ^[71]

October 2026 update: OpenAI's GPT-6 Astra launch table gives length-adjusted HealthBench Professional scores of 63.4% for GPT-6 Astra, 60.9% for Claude Fable 5, 60.5% for GPT-5.6 Sol, 58.1% for Claude Fable 5.1, 56.4% for Claude Opus 5 and 52.1% for Gemini 3.8 Flash. These are vendor-reported. ^[110]

Finance: Quiet Failures with Loud Consequences

Financial AI hallucinations don't make headlines the way legal ones do. There is no court docket for a wrong number in a model, which is also why the domain has the thinnest primary data on this page.

What can be sourced: EY's 2025 Responsible AI Pulse puts the average loss from AI-related incidents at $4.4 million per affected organization (n=975 C-suite), a figure that covers AI risk broadly, not hallucination alone. ^[79] A Stanford study found that all 24 models tested struggled to flag a false belief once the user stated it as their own. DeepSeek R1's accuracy fell from over 90% to 14.4% (Nature Machine Intelligence, November 3, 2025). That is the failure that matters most when the false premise is a valuation assumption. ^[111]

The strongest financial signal is first-party. In the Suprmind Multi-Model Divergence Index, financial questions produced disagreement between models on 72.1% of turns (258 turns), the highest rate of any of the ten domains tracked and well above legal (41.2%) or medical (33.9%). Where the stakes are a portfolio, one model's confident answer is the answer most likely to be contradicted by another. ^[61]

A set of widely circulated finance figures (a 15-25% hallucination rate on financial tasks, 2.3 significant errors per quarter, $50,000 to $2.1 million per incident, a 20.0% versus 76.7% literature-reference comparison, and a $12.7 million SEC total) appeared in earlier versions of this page. None could be traced to a primary study, so they were removed in the August 27, 2026 revision, on the same rule that removed the untraceable business-impact figures on August 8.

The Reasoning Paradox

One of the most counterintuitive findings in 2025-2026 hallucination research: the AI models marketed as most intelligent are often the least reliable on basic factual tasks.

The Core Contradiction

Reasoning models (GPT-5 with thinking, Claude with extended thinking, DeepSeek-R1) use chain-of-thought processes that dramatically improve performance on complex problems. They're measurably better at math, logic, multi-step analysis, and medical diagnosis.

They're also measurably worse at sticking to facts they've been given.

The Evidence

Vectara new dataset: At its November 2025 launch, the thinking models Vectara named, including GPT-5, Claude Sonnet 4.5 and Grok-4, all exceeded 10% hallucination, and Gemini-3-Pro read 13.6%. Grok-4-fast-reasoning reads 20.2% on the current board. Non-reasoning models like Gemini-2.5-Flash-Lite scored 3.3%. OpenAI's GPT-6 models have narrowed the gap since, at 6.5% for GPT-6 Sol and 8.7% for GPT-6 Astra. ^[102][1]

DeepSeek: R1 (reasoning) hallucinates at 14.3% on Vectara vs V3 (base) at 3.9%. Nearly a 4x difference from the same provider. Vectara analysis found R1 produces 71.7% "benign hallucinations" (plausible-sounding additions) compared to V3's 36.8%. ^[48][49]

DeepSeek V4 (April 2026): The pattern held at a greater extreme. V4 Pro and V4 Flash post AA-Omniscience hallucination rates of 94% and 96%, among the highest ever recorded, while ranking near the top of open-weights models for coding and agentic work. Strong capability, almost no calibration. ^[70]

PersonQA regression: OpenAI's o3 hallucinates 33% on questions about real people vs o1's 16%. The o4-mini is worse at 48%. These are newer, more capable models performing worse on a basic factual test. ^[53][54]

GPT-5 thinking mode: HealthBench hallucination drops to 1.6% (excellent). But on Vectara's new dataset, GPT-5 (high) reads 15.1% (poor). Same model, same thinking mode, opposite results depending on the task.

GPT-5.5 (April 2026): AA-Omniscience accuracy of 57% (a record at the time) paired with an 86% hallucination rate. Knowledge expansion appears to have outpaced calibration improvements at the frontier. Claude Opus 4.7 took the opposite trade at its launch: 36% hallucination at lower raw accuracy (42.3% on current records). ^[2][58][63]

The 2026 flagships: the pattern moved to the very top of the leaderboard. Claude Fable 5 (65% accuracy / 63.6% hallucination), Claude Opus 5 (61% / 60.8%, up from Opus 4.8's 39.3%), GPT-5.6 Sol (59.4% / 92.2%, the highest rate of any OpenAI flagship), Grok 4.5 (51.6% / 54.1%, doubled from Grok 4.3), and Kimi K3 (47.6% / 53.2%) all raised accuracy and raised hallucination in the same release. The most capable models shipped in 2026 are among the most overconfident. Before September, counterexamples included Gemini 3.1 Pro (February 2026), which cut Gemini 3 Pro's rate from 91.5% to 50.9%, and Grok 4.6 (August 2026): 48.2% accuracy and 34.3% hallucination, a 19.8-point calibration gain for a 3.4-point accuracy cost on current records. It proves the trade is a choice, not a law. September's releases split the field: GPT-6 Astra cut OpenAI's rate to 51.3% while gaining accuracy (62.6%), Grok 4.7 cut xAI's again to 29.3%, and Gemini 4 Argon reads 15.1% (not yet publicly available), while Claude Fable 5.1 went the other way, to 72.6%. ^{[2][72][75][88]}

Why This Happens

The mechanism is straightforward. When a reasoning model processes a summarization task, it doesn't just extract. It thinks. It draws inferences, identifies patterns, and generates insights. These additions go beyond the source document. On a benchmark measuring faithfulness to source material, every insight the model adds counts as a hallucination.

It's the difference between "summarize this contract" and "analyze this contract." Reasoning mode adds analysis even when you ask for a summary. That analysis is often useful. On a summarization benchmark, it's scored as a failure.

The Browse Effect Is Bigger Than the Reasoning Effect

OpenAI's system card data reveals something that gets less attention: web access has a larger impact on hallucination rates than reasoning mode does. ^[11][8]

For enterprise deployments, ensuring tool access is more impactful than selecting reasoning vs non-reasoning model variants.

The Decision Framework

This creates a practical matrix for model selection:

Reasoning ON + Web ON: Best for complex analysis, medical diagnosis, multi-step research where both depth and access to current information matter. Lowest hallucination rates on open-ended tasks.

Reasoning OFF + Web ON: Best for document summarization, RAG pipelines, grounded Q&A where you want the model to stay close to source material. Lower risk of "overthinking" additions.

Reasoning ON + Web OFF: Risky combination. The model overthinks and can't verify. Suitable only for closed-world logic problems, math, and code where external facts aren't needed.

Reasoning OFF + Web OFF: Highest hallucination risk across the board. Avoid for any factual task.

What Actually Reduces Hallucination, Ranked by Evidence

Not all hallucination reduction techniques are equal. Some are backed by controlled studies with precise measurements. Others have strong theoretical support but limited production data. This ranking reflects the evidence base, not marketing claims.

Tier 1: Largest Measured Impact

1. Web Search Access

Measured impact: 73-86% hallucination reduction (FActScore, browse-on vs browse-off)

The single highest-impact intervention documented in 2025-2026 research. On OpenAI's FActScore evaluations, o4-mini drops from 37.7% to 5.1% with web access and GPT-5 thinking from 3.7% to 1.0%. GPT-5.3 Instant shows 26.8% reduction when using web vs prior models. ^[8][10]

One scope note. The 73-86% range comes from OpenAI's own FActScore evaluations of its own models. The size of the effect varies by model, benchmark and search system, so treat it as the best-documented case rather than a universal constant, and expect any browse-on number to describe the model plus its search stack, not the model alone.

The mechanism is simple: instead of relying on potentially stale or incorrect training data, the model retrieves current information and grounds its response in external sources. For any enterprise deployment, enabling web or tool access should be the first configuration decision, not an afterthought.

2. RAG (Retrieval Augmented Generation)

Measured impact: Large and setup-dependent. No cross-vendor benchmark isolates it, and the "up to 71%" figure that circulated for enterprise knowledge-base tasks traces to a secondary compilation rather than a controlled study, so this page no longer quotes it as a benchmark result.

RAG connects models to external knowledge bases (company documents, databases, verified sources) and instructs the model to generate responses grounded in retrieved content rather than parametric memory. Hybrid retrievers combining sparse and dense methods achieve the strongest mitigation. Vectara's own leaderboard, which measures exactly this grounded setting, shows the best models under 2% and most frontier models between 5% and 14%. Retrieval narrows the problem, but it does not close it.

RAG is most effective for knowledge-gap hallucinations (the model lacks relevant training data). It's less effective for logic-based hallucinations (the model reasons incorrectly from correct premises). For enterprise document Q&A and knowledge base applications, RAG is the standard of care.

Tier 2: Strong Evidence, Context-Dependent

3. Thinking/Reasoning Mode

Measured impact: 56-59% fewer errors on OpenAI's medical and production tests, but more hallucination on grounded summarization ^[109][8]

GPT-5 with thinking hallucinated on 1.6% of OpenAI's HealthBench Hard hallucination test, against 3.6% without thinking. On production-like ChatGPT traffic with browsing on, 4.8% of GPT-5 thinking responses had at least one major incorrect claim, against 11.6% for GPT-5 main. Both are vendor-run tests, and both are large improvements.

But reasoning mode increases hallucination on Vectara's summarization benchmark (see Section 10). The impact is task-dependent. Enable reasoning for analysis, diagnosis, and complex queries. Disable it for summarization, extraction, and source-faithful tasks.

4. Multi-Model Cross-Validation

Measured impact: 8% higher factual accuracy than the best earlier hallucination-mitigation methods (UAF framework) ^[24]

Amazon's Uncertainty-Aware Fusion framework (WWW Companion 2025) combined multiple LLMs weighted by their accuracy and self-assessment quality. The key finding: different models excel on different question types, so combining them captures complementary strengths.

Cross-model disagreement detection catches hallucinations because models rarely fabricate the same false information. When one model makes an unsupported claim, others typically flag the inconsistency or provide conflicting data. Research on "wisdom of the silicon crowd" shows LLM ensembles can rival human crowd forecasting accuracy through simple aggregation.

The 8% figure understates the practical value. In production, multi-model approaches catch errors that no single-model check would flag, because the checking model has different training data, different biases, and different blind spots.

5. Chain-of-Verification (CoVe)

Measured impact: 28% FActScore improvement ^[23]

A four-step pipeline: generate baseline response, plan verification questions, answer those verification questions independently, then refine the final output. Published at ACL 2024, it outperforms zero-shot, few-shot, and chain-of-thought prompting on long-form generation accuracy.

The cost is latency and compute: four steps instead of one. For applications where accuracy matters more than speed (report generation, research synthesis, compliance documentation), the tradeoff is worth it.

Tier 3: Meaningful but Narrower

6. VeriFY (Training-Time Verification)

Measured impact: 9.7-53.3% lower factual hallucination rates on open models from 2B to 12B ^[25]

VeriFY, described in a February 2026 arXiv preprint, teaches models to assess factual uncertainty during generation rather than relying on post-hoc checking. The model learns to verify its own claims as it produces them. Recall loss is modest: 0.4-5.7%.

This is a training-time intervention, meaning end users don't control it. Its value is in signaling where the field is heading: future model generations will likely internalize verification as a core capability rather than bolting it on after generation.

7. Training Models to Guess Less

Measured impact: 38 percentage points lower AA-Omniscience hallucination rate (Gemini 3 Pro Preview 88%, Gemini 3.1 Pro Preview 50%) at about the same accuracy ^[15]

Artificial Analysis traced the drop to the model guessing less often when it does not know the answer, and its Omniscience Index rose 17 points. Neither Artificial Analysis nor Google's model card names the training method behind it. On current records the two models read 91.5% and 50.9%.

Like VeriFY, this is a provider-side intervention. Users benefit from it when selecting newer model versions but can't apply it themselves.

8. Domain-Specific Mitigation Prompts

Measured impact: the average across six models fell from 66% to 44% on planted-detail clinical tests, and GPT-4o from 53% to 23% ^[105]

Structured prompts that instruct the model to constrain outputs to verified information, flag uncertainty, and avoid speculation. These work best in narrow domains with clear boundaries and well-defined terminology.

The medical results are encouraging but the absolute rates remain high (44% on average with mitigation is still dangerously wrong for clinical use). Domain prompts are a layer, not a solution.

What Doesn't Work (Or Works Less Than Claimed)

Larger models alone: In Artificial Analysis's launch analysis, accuracy tracked model size and hallucination rate did not. The 2026 flagships confirmed it: the three most accurate models in our table, Claude Fable 5.1, Claude Opus 5.5 and Claude Fable 5, all give a wrong answer more often than they decline (72.6%, 58.6% and 63.6%). Bigger models know more but don't necessarily know what they don't know.

Simple temperature reduction: Lowering generation temperature reduces variety but doesn't eliminate hallucination. The model still picks the most probable token. It just does so more consistently, including consistently wrong tokens.

"Be accurate" system prompts: Generic instructions to avoid hallucination show minimal measured effect. Models already "try" to be accurate. The issue is architectural, not motivational.

AI Hallucination Detection Tools

How Detection Tools Work

Most hallucination detectors now ship inside LLM evaluation and monitoring platforms. They use a second model, a trained classifier or formal rules to check an answer against a source. Only three have been tested side by side by an outside party. The rest publish their own numbers or none.

Detection Tools (October 2026)

What the Accuracy Gap Means

In AIMultiple's test of 100 answers (50 correct, 50 hallucinated), W&B Weave and Arize Phoenix were right 91% and 90% of the time overall. Accuracy is not the catch rate. Weave caught 43 of the 50 hallucinations (86%) and Phoenix 42 (84%), so about one in seven got through. Each also flagged 2 of the 50 correct answers. ^[46]

Comet Opik never flagged a correct answer, but it caught only 22 of the 50 hallucinations (44%). Pick the trade-off that fits the cost of a miss. In legal filings, medical decisions and financial reporting a miss costs more than a false alarm, so automated detection is one check among several, not the last one. The test is small, and AIMultiple lists Weights & Biases among its subscribers. ^[46]

Historical Progression

Two Years of Improvement on Vectara's Original Test

Vectara has published its hallucination leaderboard on GitHub since November 2023, and the repository keeps every update. We read all 255 versions of the leaderboard file to find the lowest rate at each update. On the original test, the best score fell from 3.0% (GPT-4, November 2023) to 0.6% (Ant Group's Finix-S1-32B, August 2025). Vectara then moved to a longer, harder dataset in November 2025. There the best score started at 3.3%, and it has stood at 1.8% since December 2025. ^[1]

On the same test, the best score in August 2025 was a fifth of the November 2023 best. The leaders were not always frontier models: Intel's 7B Neural Chat held the top spot in March 2024, Zhipu AI's 9B GLM-4 in August 2024 and Ant Group's 32B Finix-S1 from August 2025. On short-document summarization, the improvement is real.

The Reality Check

These improvements measure the easiest version of the problem: summarizing short documents without adding unsupported facts. When you move to harder, more realistic evaluations, the picture changes:

AA-Omniscience (hard knowledge questions): 141 of the 202 models on the October 6, 2026 board give more wrong answers than right ones. Only 61 have a positive Omniscience Index, and the three most accurate models in our table, Claude Fable 5.1, Claude Opus 5.5 and Claude Fable 5, still give a wrong answer more often than they decline. ^[2][88]

HalluHard (realistic conversations): Even the best entry on the board, GPT-6 Astra with web search, hallucinates 27.2% of the time, and Claude Opus 4.5 with web search 30.2%. Of its 41 entries, 10 are under 50%, 16 fall between 50% and 70%, and 15 are above 70%. ^[5]

Vectara new dataset (enterprise-length documents): Rates are higher than on the original dataset for most models, often several times higher: GPT-4.1 goes from 2.0% to 5.6% and GPT-5 (high) from 1.4% to 15.1%. The best score is 1.8% (finix_s1_32b), not 0.6%, and the best frontier-lab score is 3.1% (GPT-5.4 nano). Most frontier models sit between 5% and 14%. ^[1][100]

Domain-specific tasks: Purpose-built legal AI tools still err on more than 17% of queries (Stanford RegLab, 2025), and clinical tests with planted fake details reach 66% on average without a mitigation prompt. Specialized domains haven't shown the same improvement trajectory as general summarization. ^[36][105]

Improvement is real. But extrapolating from simple benchmarks to enterprise reliability is a mistake the data doesn't support.