Suprmind Data Lab · Q3 2026 Edition
We re-checked 17,885 real questions answered side by side by GPT, Claude, Gemini, Grok and Perplexity, with model names hidden and answers in random order. Judges from Anthropic and Google then read 3,262 of them, to test which findings hold.
Published October 2026 · Data from July 30 to September 10, 2026 · This edition replaces the April 2026 edition, corrected below.
First time here? See how this index works.
Suprmind is a chat platform where up to five AI models answer the same question in one conversation: GPT from OpenAI, Claude from Anthropic, Gemini from Google and Grok from xAI, plus Perplexity. Each is built and tuned differently, so their answers often differ, and one can catch what another missed. This index measures how often that happens, between which models and on which questions.
A real question asked on Suprmind. We publish counts, never the questions themselves.
In Sequential mode the models answer in turn, and each reads the answers before it. In Super Mind they answer at once, without seeing each other.
A separate AI model reads all the answers, with model names hidden and in random order. It notes each contradiction, correction and addition it finds.
Invented example
QuestionHow long do shoppers in the EU have to cancel something they bought online?
Answer 1Seven working days.
Answer 2Fourteen days. Seven working days was the old minimum. Since 2014, EU law has given shoppers 14 days.
Answer 3The clock starts when the item arrives, not when it is ordered. Made-to-order goods are an exception.
What the judge records
A judge's call is a reading, not proof, and judges from different companies read the same answers differently. So judges from OpenAI, Anthropic and Google all read the same 3,262 questions, and we say which results hold under all three.
Suprmind sorts its models into three teams: the A-team (top models), Operators (mid-tier) and Daily Drivers (budget). On this page, a line-up is the exact set of models that answered together, and Daily Drivers ran as two line-ups with different GPT models. In live traffic, different line-ups answered different questions, so model-by-model comparisons stay inside one line-up.
For this edition, a blind judge read all 17,885 real questions answered between July 30 and September 10, 2026. It found at least one contradiction in 29.9% of them and at least one correction in 35.8%.
This is not a benchmark or a hallucination leaderboard. It measures how AI models behave next to each other, in our product, on our users' questions. For accuracy against public benchmarks, see AI Hallucination Rates and Benchmarks.
GPT, Perplexity and Claude. When at least two of three AI judges, from OpenAI, Anthropic and Google, had to agree, GPT, Perplexity and Claude answers were judged corrected by another model on 3 to 4% of answers. Gemini and Grok answers: 6.5% and 7.4%. Tested pair by pair, every gap between the two groups is larger than chance, and no gap inside a group is.
Flip the judge switch on the chart: the levels move a lot, while GPT and Perplexity stay ahead of Gemini and Grok under every judge.
Share of each family's answers that a judge found corrected by another model. Super Mind questions, where every model answers on its own. Lines show 95% intervals.
At least two of three judges agree (OpenAI, Anthropic, Google), 1,796 questions. Same scale in every view, so you can see how much the level depends on the judge.
One blind judge, all 6,536 Super Mind questions. Same scale in every view, so you can see how much the level depends on the judge.
OpenAI judge (gpt-5-mini), same 1,796 questions. Same scale in every view, so you can see how much the level depends on the judge.
Anthropic judge (Claude Sonnet 5.5), same 1,796 questions. Same scale in every view, so you can see how much the level depends on the judge.
Google judge (Gemini 3.8 Flash), same 1,796 questions. Same scale in every view, so you can see how much the level depends on the judge.
Perplexity means answers served through Perplexity's API. It joined only five-model questions, so it has fewer answers. A family mixes versions: most Claude answers came from Claude Haiku 4.5, most Gemini answers from Gemini 3.5 Flash-Lite. July 30 to September 10, 2026.
The comparison uses Super Mind questions, where every model answers on its own and no model sees another's answer. Within a line-up the models faced the same questions, except that Perplexity joined only the five-model ones. So a gap between them is a gap in how their answers held up when a judge read them side by side.
One judge reading every question tells the same story at a different level. Across 6,536 real questions that several AI models answered independently, a blind judge found Claude and GPT answers corrected by another model least often (8.9% and 9.2% of answers), Gemini and Grok most often (13.0% and 12.7%).
Switch the judge and the levels move a lot: the Anthropic judge flags the most corrections and the Google judge the fewest. The order barely moves. GPT and Perplexity answers are judged corrected less often than Gemini and Grok answers under each of the three judges.
Claude's place is the one that depends on the judge. Under the OpenAI and Google judges, Claude ties GPT at the top. Under Anthropic's own judge, Claude sits behind GPT. No judge went easy on its own maker's models. Anthropic's judge was harder on Claude than the OpenAI and Google judges were.
A family here mixes several models. Most Claude answers in this window came from Claude Haiku 4.5, Anthropic's budget model, and most Gemini answers from Gemini 3.5 Flash-Lite. The model-by-model view further down separates the models.
Scan the headlines. Each card jumps to its chart and the evidence.
3.4%, 3.7% and 4.3% of answers judged corrected by another model, with two of three judges required to agree.
6.5% and 7.4% under the same rule. GPT and Perplexity stay ahead of both under each of the three judges.
Judged corrected on 8.7% of its answers, against 21.6% for Gemini 3.1 Pro and 21.0% for Grok 4.3. Same 310 questions.
In our top line-up's replay, Opus 5.5 corrected 24 of every 100 earlier answers it followed. Grok 4.3 corrected 3.
Speaking last, budget models were judged corrected on 3.4% to 10.2% of answers. Speaking first, 16.1% to 29.7%.
45% of high-stakes questions drew a correction under a blind judge, against 28% of low-stakes ones, at the same point in the conversation.
Contradictions on 40% of financial questions and 20% of medical ones, at the same point in the conversation. Legal sits close to finance.
An answer a blind judge read as assertive was contradicted more often, but not judged corrected more often.
Above its teammates' average under all three judges. In the A-team, two of three judges rated Claude Opus 5 even more sure, and it was the least corrected.
57.5% to 67.2% of disagreements were judgment calls, depending on the judge. Factual disputes were 17.0% to 30.3%.
189 of 411 disputes over facts and numbers (46%). Software versions led with 101, then prices, current events, statistics and schedules.
When two fact-check judges from different companies agreed, they sided with the correcting model 18 times out of 23, checked against the web of late September 2026.
Claude Opus 5 in the top line-up, a three-way tie in the mid-tier line-up, and GPT-5.6 Luna in one of our two budget line-ups. Here every model is compared only with the models it answered next to, on the same questions. In live traffic, models in different line-ups never answered the same questions, and harder questions go to the top line-up, so a rate in one line-up says nothing about a rate in another.
Share of each model's answers that a blind judge found corrected by another model, compared only inside its own line-up. Super Mind questions. Lines show 95% intervals.
| Model | Judged corrected | In a contradiction | Judge sided with it, per 100 answers |
|---|---|---|---|
| Claude Opus 5 | 8.7% | 45.2% | 61.9 |
| GPT-5.4 | 13.9% | 31.6% | 21.9 |
| Gemini 3.1 Pro | 21.6% | 43.5% | 17.1 |
| Grok 4.3 | 21.0% | 33.5% | 14.2 |
| Perplexity | 12.3% | 36.2% | 35.5 |
Swipe the table sideways to see every column.
OpenAI judge (gpt-5-mini), names hidden, answers in random order. 310 questions. Perplexity: 138 answers, from August 8, when its seat moved to Perplexity's Agent API.
Judged corrected by at least two of three judges (OpenAI, Anthropic, Google), 310 questions. Perplexity here covers the whole window.
| Model | Judged corrected | In a contradiction | Judge sided with it, per 100 answers |
|---|---|---|---|
| Claude Sonnet 5 | 7.8% | 28.1% | 30.1 |
| GPT-5.2 | 9.0% | 21.9% | 15.8 |
| Gemini 3.6 Flash | 11.4% | 25.2% | 10.2 |
| Grok 4.3 | 14.8% | 23.0% | 7.1 |
| Perplexity | 8.8% | 23.4% | 20.5 |
Swipe the table sideways to see every column.
OpenAI judge (gpt-5-mini), names hidden, answers in random order. 778 questions. Perplexity: 645 answers, from August 8, when its seat moved to Perplexity's Agent API.
Judged corrected by at least two of three judges (OpenAI, Anthropic, Google), 778 questions. Perplexity here covers the whole window.
| Model | Judged corrected | In a contradiction | Judge sided with it, per 100 answers |
|---|---|---|---|
| Claude Haiku 4.5 | 7.7% | 22.2% | 19.1 |
| GPT-5.4-mini | 7.4% | 21.1% | 13.5 |
| Gemini 3.5 Flash-Lite | 10.0% | 22.1% | 8.3 |
| Grok 4.3 | 9.7% | 22.1% | 9.7 |
Swipe the table sideways to see every column.
OpenAI judge (gpt-5-mini), names hidden, answers in random order. 1,735 questions.
Judged corrected by at least two of three judges (OpenAI, Anthropic, Google), 316 questions. On this smaller sample no gap between the budget models separates.
| Model | Judged corrected | In a contradiction | Judge sided with it, per 100 answers |
|---|---|---|---|
| Claude Haiku 4.5 | 11.1% | 27.2% | 18.9 |
| GPT-5.6 Luna | 8.5% | 26.7% | 21.6 |
| Gemini 3.5 Flash-Lite | 11.9% | 25.1% | 8.1 |
| Grok 4.3 | 10.9% | 24.8% | 13.3 |
Swipe the table sideways to see every column.
OpenAI judge (gpt-5-mini), names hidden, answers in random order. 1,158 questions.
Judged corrected by at least two of three judges (OpenAI, Anthropic, Google), 219 questions. On this smaller sample no gap between the budget models separates.
In live traffic, models in different line-ups never answered the same questions, so compare inside a tab, not across tabs. "Judge sided with it": how often, per 100 answers, the judge credited this model's answer with correcting another. Model versions as measured, July 30 to September 10, 2026.
In our top line-up, a blind judge found Claude Opus 5 corrected by another model on 8.7% of its answers. Gemini 3.1 Pro: 21.6%. Grok 4.3: 21.0%. Same 310 questions. GPT-5.4 sat in between at 13.9%. When two of three judges had to agree, Opus 5's rate fell to 2.3%, against 17.4% for Gemini 3.1 Pro and 18.4% for Grok 4.3.
Opus 5 was also part of the most contradictions: 45.2% of its answers clashed with another model's. When the judge found one answer correcting another, it sided with Opus 5 62 times per 100 answers, almost three times as often as with GPT-5.4. Under all three judges, Opus 5 was both the least corrected model and the one the judge sided with most in the A-team.
In our mid-tier line-up, Grok 4.3 was judged corrected on 14.8% of its answers. Claude Sonnet 5: 7.8%. GPT-5.2: 9.0%. Same 778 questions. Under all three judges, Grok 4.3 is the most corrected model in this line-up, and Claude Sonnet 5, GPT-5.2 and Perplexity tie at the top. Gemini 3.6 Flash moves with the judge, and the three-judge consensus puts it between the two groups.
Among budget models, a blind judge found GPT-5.6 Luna corrected by another model on 8.5% of its answers and Gemini 3.5 Flash-Lite on 11.9%, over the same 1,158 questions. Judges from Anthropic and Google, reading 219 of those questions, also found Luna corrected less often than Gemini 3.5 Flash-Lite.
In the budget line-up with GPT-5.4-mini, the OpenAI judge found GPT-5.4-mini corrected on 7.4% of its answers, against 10.0% for Gemini 3.5 Flash-Lite and 9.7% for Grok 4.3. Claude Haiku 4.5 came out at 7.7% under that judge, but its place moves with the judge, so we do not rank it.
In three of our four main line-ups, the judge sided most often with the Claude model when it found one answer correcting another. In the fourth, it sided most often with GPT-5.6 Luna. And no GPT model was the most corrected in any of our line-ups.
When a judge reads two answers to the same question and finds that one corrects the other, it takes a side. Count those calls for every pair of models and a clear order appears. In every line-up, the side a blind judge took ran one way down a single ranking. Of 62 model pairs in seven line-ups, 27 leaned one way beyond chance, and all 27 fit one ranking per line-up. Gemini and Grok sat at the bottom of all four main line-ups.
When two answers to the same question clashed and the judge found one corrected the other, whose side did it take? Each cell: per 100 questions both models answered, how often it sided with the row model over the column model.
| Sided with, over | Claude Opus 5 | Perplexity | GPT-5.4 | Grok 4.3 | Gemini 3.1 Pro |
|---|---|---|---|---|---|
| Claude Opus 5 | · | 7.2 | 11.6 | 16.8 | 13.5 |
| Perplexity | 3.6 | · | 2.2 | 7.2 | 11.6 |
| GPT-5.4 | 3.2 | 5.1 | · | 2.9 | 6.1 |
| Grok 4.3 | 1.9 | 2.9 | 1.3 | · | 5.5 |
| Gemini 3.1 Pro | 2.6 | 2.2 | 2.9 | 4.2 | · |
Swipe the table sideways to see every column.
4 of 10 pairs in this line-up lean one way beyond chance, and every one of them fits the order above.
| Sided with, over | Claude Sonnet 5 | Perplexity | GPT-5.2 | Gemini 3.6 Flash | Grok 4.3 |
|---|---|---|---|---|---|
| Claude Sonnet 5 | · | 5.6 | 5.0 | 7.1 | 9.5 |
| Perplexity | 3.9 | · | 3.6 | 4.8 | 5.4 |
| GPT-5.2 | 2.3 | 2.5 | · | 3.9 | 4.4 |
| Gemini 3.6 Flash | 1.9 | 1.1 | 1.5 | · | 3.1 |
| Grok 4.3 | 1.9 | 1.2 | 0.9 | 1.8 | · |
Swipe the table sideways to see every column.
7 of 10 pairs in this line-up lean one way beyond chance, and every one of them fits the order above.
| Sided with, over | Claude Haiku 4.5 | GPT-5.4-mini | Grok 4.3 | Gemini 3.5 Flash-Lite |
|---|---|---|---|---|
| Claude Haiku 4.5 | · | 4.5 | 6.1 | 6.3 |
| GPT-5.4-mini | 4.7 | · | 3.8 | 3.3 |
| Grok 4.3 | 2.9 | 2.2 | · | 3.1 |
| Gemini 3.5 Flash-Lite | 2.5 | 2.4 | 2.5 | · |
Swipe the table sideways to see every column.
3 of 6 pairs in this line-up lean one way beyond chance, and every one of them fits the order above.
| Sided with, over | GPT-5.6 Luna | Claude Haiku 4.5 | Grok 4.3 | Gemini 3.5 Flash-Lite |
|---|---|---|---|---|
| GPT-5.6 Luna | · | 7.0 | 5.9 | 6.4 |
| Claude Haiku 4.5 | 5.5 | · | 5.3 | 5.7 |
| Grok 4.3 | 2.8 | 4.7 | · | 4.1 |
| Gemini 3.5 Flash-Lite | 1.6 | 3.1 | 2.1 | · |
Swipe the table sideways to see every column.
4 of 6 pairs in this line-up lean one way beyond chance, and every one of them fits the order above.
Super Mind: no model saw another's answer, so "sided with" is the judge's reading of two independent answers. OpenAI judge, names hidden, answers in random order. Perplexity: answers served through Perplexity's API from August 8.
In the A-team, a blind judge sided with Claude Opus 5 over Grok 4.3 on 16.8% of the questions both answered, and with Grok over Opus on 1.9%. On the same questions, it sided with Opus 5 over Gemini 3.1 Pro more than twice as often as with GPT-5.4 over Gemini 3.1 Pro: 13.5% against 6.1%. In Operators, it sided with Claude Sonnet 5 over Grok 4.3 on 9.5% of shared questions, and the other way round on 1.9%.
In the A-team, the judge sided with Claude Opus 5 over each of its teammates more often than the other way round.
In Super Mind no model sees another's answer, so "sided with" is the judge's reading of two independent answers. It is not one model correcting another. For that, we need Sequential, and a fair way to measure it.
In Sequential mode each model reads the answers before it, so a later model can correct an earlier one. Our own traffic cannot rank models fairly here, because Suprmind's speaking order is nearly fixed: in four-model chains, Claude spoke last 97.7% of the time. So we replayed real questions in every speaking order. We sent 118 real first questions through our budget line-up four times each, so every model spoke once in every position, and the first 60 of them through the A-team five times each.
Real questions replayed in every speaking order, so every model spoke once in every position. Corrections a model made per 100 earlier answers it could have corrected.
| Critic, on | Claude Opus 5.5 | GPT-5.4 | Perplexity | Gemini 3.1 Pro | Grok 4.3 |
|---|---|---|---|---|---|
| Claude Opus 5.5 | · | 20.0 | 14.7 | 32.2 | 28.0 |
| GPT-5.4 | 9.0 | · | 9.9 | 16.7 | 13.3 |
| Perplexity | 8.3 | 7.5 | · | 17.3 | 15.0 |
| Gemini 3.1 Pro | 4.5 | 9.0 | 7.3 | · | 11.8 |
| Grok 4.3 | 3.2 | 2.8 | 2.0 | 5.4 | · |
Swipe the table sideways to see every column.
60 real questions, sent in every speaking order. The judge counts a correction only when the critic spoke after the answer it corrected.
| Critic, on | Claude Haiku 4.5 | GPT-5.6 Luna | Gemini 3.5 Flash-Lite | Grok 4.3 |
|---|---|---|---|---|
| Claude Haiku 4.5 | · | 14.9 | 16.3 | 16.2 |
| GPT-5.6 Luna | 12.7 | · | 14.0 | 12.8 |
| Gemini 3.5 Flash-Lite | 4.4 | 7.0 | · | 6.3 |
| Grok 4.3 | 6.3 | 5.1 | 4.7 | · |
Swipe the table sideways to see every column.
118 real questions, sent in every speaking order. The judge counts a correction only when the critic spoke after the answer it corrected.
Replay of real first questions from this edition, run in October 2026 on one research account: concise answers, web search on. By then Claude Opus 5 had been replaced by Claude Opus 5.5. Perplexity: answers served through Perplexity's API.
We sent 60 real questions through our top AI line-up in every speaking order. Claude Opus 5.5 corrected 24 of every 100 earlier answers it followed: about twice GPT-5.4 (12), three times Gemini 3.1 Pro (8), seven times Grok 4.3 (3). The order holds whoever it follows. On Gemini 3.1 Pro's answers, Opus 5.5 corrected 32.2 per 100, Perplexity 17.3, GPT-5.4 16.7 and Grok 4.3 5.4.
We sent 118 real questions through four budget AI models in every speaking order. Two of them did most of the correcting: Claude Haiku 4.5 and GPT-5.6 Luna corrected 13 to 16 of every 100 earlier answers they followed. Gemini 3.5 Flash-Lite and Grok 4.3, about 5 to 6.
Grok 4.3 sat in both our budget and our top line-up. On the same 50 questions it found less to correct in the top line-up's answers.
Share of each model's answers that the judge found corrected by another model, by where it spoke in the chain. Every model spoke once in every position on every question.
| Speaking | 1stin the chain | 2ndin the chain | 3rdin the chain | 4thin the chain |
|---|---|---|---|---|
| Claude Haiku 4.5 | 16.1% | 11.0% | 5.9% | 3.4% |
| GPT-5.6 Luna | 20.3% | 13.6% | 5.1% | 10.2% |
| Grok 4.3 | 29.7% | 15.3% | 9.3% | 5.9% |
| Gemini 3.5 Flash-Lite | 24.6% | 17.8% | 9.3% | 9.3% |
Swipe the table sideways to see every column.
118 questions, 4 orders each: 118 answers per cell.
| Speaking | 1stin the chain | 2ndin the chain | 3rdin the chain | 4thin the chain | 5thin the chain |
|---|---|---|---|---|---|
| Claude Opus 5.5 | 18.3% | 11.7% | 11.7% | 11.7% | 1.7% |
| GPT-5.4 | 31.7% | 18.3% | 16.7% | 16.7% | 13.3% |
| Perplexity | 28.3% | 20.0% | 13.3% | 6.7% | 8.3% |
| Gemini 3.1 Pro | 31.7% | 33.3% | 41.7% | 25.0% | 8.3% |
| Grok 4.3 | 40.0% | 35.0% | 26.7% | 23.3% | 15.0% |
Swipe the table sideways to see every column.
60 questions, 5 orders each: 60 answers per cell, so single cells are noisy.
The first answer has every later model as a possible critic, and no one speaks after the last. Counted per chance to be corrected, the gap shrinks. A last answer can still be judged corrected when the judge reads an earlier answer as correcting it. Production's speaking order is nearly fixed, so this replay is the fair comparison.
In a chain of AI answers, the last word is rarely challenged. We sent 118 real questions through four budget AI models (Claude Haiku 4.5, GPT-5.6 Luna, Gemini 3.5 Flash-Lite and Grok 4.3) in every speaking order: each was judged corrected on 3.4% to 10.2% of its answers when it spoke last, and on 16.1% to 29.7% when it spoke first. In the A-team, Claude Opus 5.5 speaking last was judged corrected on 1 of its 60 answers (1.7%), the others on 8.3% to 15.0%.
Most of that is arithmetic. The first answer has every later model as a possible critic, and no one speaks after the last. Counted per chance to be corrected, the gap shrinks: in the A-team the first and fourth answers fare about the same (12.2% and 12.3%), and in the budget line-up the first answer still fares worse, 11.7% against 5.9% for the third. The practical point stands: whatever the last model says goes unchallenged unless you ask again.
The replay used real first questions from this edition, sent again in October 2026 through our production system on one research account, with concise answers and web search on. It is not the everyday mix, which is mostly follow-up questions. By then Claude Opus 5 had been replaced by Claude Opus 5.5.
They draw more corrections, and their answers read about as sure. A blind judge found a correction in 45% of high-stakes turns, against 28% of low-stakes ones, at the same point in the conversation.
Each conversation was labelled low, medium or high stakes by one pass of an AI model, not by a person. Rates are compared at the same depth in the conversation, because early turns carry more disagreement.
Share of questions with at least one correction or contradiction, by how much was at stake, at the same point in the conversation. Lines show 95% intervals.
Questions with a correction
Questions with a contradiction
OpenAI judge, names hidden, answers in random order, all 17,885 questions. High minus low stakes: corrections +17.1 points, contradictions +7.0. Rates standardised to the same conversation depth.
OpenAI judge · high minus low +12.7 points
Anthropic judge · high minus low +15.0 points
Google judge · high minus low +6.1 points
Questions with a correction, under each judge, on the same 1,796 Super Mind questions, weighted to the line-up mix. The judges flag corrections at very different levels, and all three see the rise.
Stakes labels come from one labelling pass by an AI model per conversation, not from people. July 30 to September 10, 2026.
Judges from OpenAI, Anthropic and Google all see the rise. On the same 1,796 questions, high-stakes questions drew more corrections than low-stakes ones by 12.7, 15.0 and 6.1 points. Contradictions rise less with stakes than corrections do, and not under every judge.
The tone barely moves. On the blind judge's 1 to 10 scale of how assertive an answer reads, no model family moved more than 0.09 points between low-stakes and high-stakes questions with both modes pooled, or 0.13 points within one mode. GPT's was the only drop larger than chance, from 7.27 to 7.18. The questions where a mistake costs the most are the ones where the answer you read is most often judged corrected by another model, and its tone gives no sign of it.
Application
High-stakes, financial and legal questions drew the most corrections in this index. In Suprmind, up to five AI models answer your question in the same conversation, and in Sequential mode each one reads the answers before it. A correction reaches you before you act on an answer, not after.
See how the multi-AI workflow worksNo. Sounding sure starts arguments. On the same question, an AI answer a blind judge read as confident was contradicted by the other models more often, but was not judged corrected any more often.
With the question, the model and the answer's reading position held fixed, answers the judge rated 8 or more out of 10 for assertiveness were contradicted 4.0 points more often than answers rated 7, and judged corrected 1.6 points less often. Every pairing of our three judges agrees that assertive answers draw more pushback. Whether they also get judged corrected more depends on which judge decides: not under the OpenAI or Google judges, but clearly under Anthropic's.
How much more or less assertive each model's answers read than its teammates' answers to the same question, on a 1 to 10 scale. Lines show 95% intervals.
OpenAI judge, 310 questions. Perplexity: 138 answers, from August 8, when its seat moved to Perplexity's Agent API. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
OpenAI judge, 775 questions. Perplexity: 642 answers, from August 8, when its seat moved to Perplexity's Agent API. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
OpenAI judge, 314 questions. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
OpenAI judge, 218 questions. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Anthropic judge, 310 questions. Perplexity: 138 answers, from August 8, when its seat moved to Perplexity's Agent API. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Anthropic judge, 778 questions. Perplexity: 645 answers, from August 8, when its seat moved to Perplexity's Agent API. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Anthropic judge, 315 questions. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Anthropic judge, 219 questions. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Google judge, 310 questions. Perplexity: 138 answers, from August 8, when its seat moved to Perplexity's Agent API. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Google judge, 778 questions. Perplexity: 645 answers, from August 8, when its seat moved to Perplexity's Agent API. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Google judge, 316 questions. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Google judge, 219 questions. Points on the judge's 1 to 10 assertiveness scale, against the other models' answers to the same question.
Tone is each judge's reading of the language, not a measure of accuracy. The three judges rated 34%, 46% and 73% of the same answers 8 or more out of 10. Super Mind, 1,796 questions.
Gemini models rated above their teammates' average in every line-up, under judges from all three companies. Under the OpenAI judge, Gemini's answers rated 8 or more were not judged corrected more often than its answers rated 7. GPT-5.4, GPT-5.2 and Perplexity rated below their teammates' average under all three judges.
The clearest case is the A-team. Under the Anthropic and Google judges, Claude Opus 5 sounded the most sure of all five models, and it was also the least corrected model in that line-up. Tone tells you how an answer is written, not how it will hold up.
Tone is each judge's reading of the language. The three judges rated 34%, 46% and 73% of the same answers 8 or more out of 10, so compare models inside one judge.
Finance and law, which come out statistically tied. AI models contradict each other about twice as often on financial questions as on medical ones: 40% against 20% of turns under a blind judge, at the same point in the conversation. Judges from OpenAI, Anthropic and Google all found financial questions more contested than medical ones, by different margins.
Share of questions with at least one contradiction or correction, at the same point in the conversation. Lines show 95% intervals.
OpenAI judge, names hidden, answers in random order, every eligible question. Topics come from one labelling pass by an AI model per conversation, and questions labelled "other" are not shown. Rates standardised to each mode's mix of conversation depths. Small topics have wide intervals.
Legal questions come second, close enough to finance that the two cannot be told apart: 34.5% against 39.9%. Legal questions also drew corrections as often as financial ones, on 51.2% and 49.2% of turns. Every other topic sat at 40% or lower. Legal traffic is concentrated, though: in Super Mind, all of it came from 62 accounts.
Medical questions drew among the fewest contradictions under all three judges. Topic labels come from one pass of an AI model per conversation, not from people, and the smaller topics carry wide intervals.
Partly. When the models read each other's answers (Sequential), the first turn is where they disagree most, under all three judges. After that it depends on the mode.
When models read each other's answers (Sequential), a blind judge found contradictions in 36.7% of first turns and about half that later on: 21.2% at turns 4 to 7 and 19.1% at turns 8 to 15. When they answer on their own (Super Mind), disagreement drops after the first turn and then climbs back: 42.2% on the first turn, 25.8% on turns 2 and 3, and 32.2% to 35.4% after that.
Share of turns with at least one contradiction, by how far into the conversation the question came.
OpenAI judge, names hidden, answers in random order, all 17,885 questions.
The same 1,466 Sequential questions, about 300 from each depth, read by all three judges. Levels differ by judge. All three find the most disagreement on the first turn.
July 30 to September 10, 2026. The 50 most active accounts wrote 58.6% of Sequential turns and 48.9% of Super Mind turns from the 16th turn on. Leaving them out does not change the pattern.
Whether Sequential disagreement keeps falling after the first few turns depends on the judge. Under Anthropic's and Google's judges it drops after the first turn and then climbs back part of the way, and under Anthropic's judge turns 8 to 15 come close to the first. All three judges still put the first turn highest.
Long conversations come mostly from heavy users: the 50 most active accounts wrote 58.6% of Sequential turns from the 16th turn on. Leaving those accounts out does not change the pattern.
Mostly judgment calls, not facts. Under all three judges, judgment calls were the largest share of disagreements: 67.2%, 64.1% and 57.5%. Factual disputes made up 17.0% to 30.3% and numbers 12.2% to 14.1%.
Every disagreement each judge found, by type. Then a closer look at the factual disputes.
OpenAI judge 1,532 disagreements
Anthropic judge 4,060 disagreements
Google judge 1,290 disagreements
The same 1,796 Super Mind questions read by each judge.
Factual disputes: 411 factual and numerical disputes we could trace word for word to both answers. Categories come from an AI classifier and are approximate.
Even the factual disputes are mostly not facts you can look up. Of 411 disputes over facts and numbers that we could trace word for word to both answers, 36% were about world facts. The rest were advice or opinion (32%), something only the conversation shows (16%), an assistant describing itself (15%) and facts about the user (1.5%).
Almost half of these disputes (189 of 411, or 46%) were about facts that change: software versions, prices, current events, statistics and schedules. An answer can be out of date without being made up.
When a correction can be fact-checked, the correcting model is usually right. When two fact-check judges from different companies agreed on the outcome, they sided with the correcting model 18 times out of 23. On points that do not change over time, 10 out of 10. Both judges read the same evidence, gathered from the web as it stood at the end of September 2026, not on the day of the conversation.
These numbers describe our setup: our fact-checker, our line-ups and our users' questions between July 30 and September 10, 2026. They are not a ranking of which AI is right in general. The fact-checker is True North, which we are building into Suprmind. It pulls checkable claims from an answer, researches each one and decides whether the evidence supports it, contradicts it or cannot tell.
We started from 1,198 factual and numerical disputes between models. 843 could be traced to both answers, and in 411 we found both claims word for word. The fact-checker settled 90 of them (22%) and left 254 (62%) unresolved. When it could decide whether a correcting model was right, the corrector was right 26 times and wrong 11 times. A second judge from Anthropic, reading the same evidence, made it 22 right and 7 wrong. On the 130 disputes we could check again as of each conversation's date, the fact-checker's record went from 20 right and 11 wrong to 10 right and 3 wrong. It decided fewer disputes and reversed none of its earlier calls.
Corrections are not always right either. The correcting model's own claim was contradicted in 16.7% of the disputes we could link, and in 7.7% when both judges had to agree.
We asked research tools from five companies (four in deep-research mode, Grok in its standard mode) to settle 22 of the disputes on their own, as of the date of each conversation. They backed our fact-checker on 14 of the 22, and a fixed version on 16. Before the test we had set a bar of 20, later lowered to 19, and neither version reached it. Most misses were facts that change over time: 7 of 8 for the version as it ran, which graded them against the web of late September instead of the day of the conversation, and 5 of 6 for the fixed version. On stable facts both versions matched the tools 9 times out of 10.
Share of each model's checkable claims that our fact-checker found contradicted by the evidence, out of the claims it could decide. Lines show 95% intervals.
A-team, 60 questions from the speaking-order replay. Fact-checker as it ran: OpenAI models do the research and the judging.
Budget line-up, 118 questions from the speaking-order replay. Fact-checker as it ran: OpenAI models do the research and the judging.
True North as it ran: OpenAI models research and judge gap 38.8 points (95% interval 24.0 to 52.7)
A Claude judge reads the same evidence gap 20.5 points (95% interval 7.6 to 33.7)
Claude does the research and the judging gap 12.0 points (95% interval −1.5 to 25.8)
The same 347 checked values from the replay, under three fact-checkers. The gap shrinks as the fact-checker moves away from OpenAI models, and with Claude doing both steps it can no longer be told apart from zero.
Fact-checker: True North, which we are building into Suprmind. It pulls checkable claims from each answer, researches them and decides each one: supported, contradicted or unverifiable. A contradicted claim is our fact-checker's call on the evidence it found, in October 2026.
We also fact-checked every checkable claim in the replay answers. In the A-team, the fact-checker found 14.5% of GPT-5.4's decided claims contradicted by the evidence and 56.8% of Gemini 3.1 Pro's. In the budget line-up, 18.4% of GPT-5.6 Luna's and 51.4% of Claude Haiku 4.5's.
Our fact-checker runs on OpenAI models, and GPT models are among those it checks. To test whether that tilts the results, we re-checked a sample of 300 of those facts, first with a Claude judge reading the same evidence, then with Claude doing both the research and the judging. The gap between the models it contradicted least, GPT and Perplexity, and the other three went from 39 points to 21, then 12. At 12 points it can no longer be told apart from zero on this sample. Under every fact-checker we tried, GPT and Perplexity claims were contradicted less often, but how much less depends on who checks.
One pattern holds under all three fact-checkers: numbers are where claims break. 49.6% of decided claims containing a number were contradicted, against 33.5% of claims without one.
In Suprmind, several AI models answer your question in the same thread. When one states a number, a fact or a recommendation the others would challenge, you see the disagreement instead of one confident answer.
Test It on a Real Question7 days free. No credit card. Grok, GPT, Claude and Gemini in the trial.
Three things people often read as signs of a better answer turned out to say little about whether a judge found it corrected.
On the same question, the longest answer was judged corrected 11.1% of the time and the shortest 10.4%. Across three judges, length shows no consistent effect.
The slowest answer to a question was judged corrected 10.6% of the time and the fastest 10.5%, over 4,325 questions.
When a model failed and another stood in, the answer arrived about twice as late. The judge found no reliable change in how often it was corrected.
An AI judge reads all the answers to a question and records each contradiction and correction it finds between them. Our judge is OpenAI's gpt-5-mini, the same judge Suprmind runs in production. For this edition it re-read every eligible question from the window, 17,885 in all, with model names hidden and the answers in a fresh random order for each question. Two more judges, Anthropic's Claude Sonnet 5.5 and Google's Gemini 3.8 Flash, read 1,796 Super Mind questions and 1,466 Sequential questions on exactly the same input.
AI judges are swayed by position. Our judge marked whichever answer it read first as corrected by another model two to four times as often as the others, whoever wrote it, and judges from Anthropic and Google did the same. It is not our prompt: reversing its worked examples or relabelling the answers barely moved it. So since September 30 our judge reads every line-up in a fresh random order, which spreads the effect evenly over the models.
Share of answers judged corrected by another model, by the position in which the judge read them. Five-answer Super Mind questions, answers shuffled into a random order for every question.
OpenAI judge, the same 1,109 questions with five answers.
Anthropic judge, the same 1,109 questions with five answers.
Google judge, the same 1,109 questions with five answers.
Because the order is random for every question, the effect lands on every model equally often, which keeps model rates fair. Rates and the size of the first-read spike differ by judge, but every judge shows it.
Researchers have found the same pull in other AI judges [1] [2]. It matters a lot. Reading the same Super Mind answers, the old production judge, which saw model names and read the answers in a fixed order, and the blind judge with a random order ranked the five families almost in reverse:
| Family | Old judge: names visible, fixed order | Blind judge, random order |
|---|---|---|
| GPT | 10.4% | 7.4% |
| Perplexity | 0.5% | 8.9% |
| Claude | 3.8% | 7.5% |
| Gemini | 1.0% | 10.9% |
| Grok | 1.3% | 10.1% |
Share of answers judged corrected by another model, the same 3,358 Super Mind questions read by both judges. The April edition's provider comparisons came from judges set up the same way as the old one, which is why they are withdrawn.
Our judge skipped 1 in 4 turns, and those were the most contested: re-judging all 17,885, the skipped turns had contradictions in 44% of cases, the rest in 25%. That is why the main rates in this edition come from a full re-judge of all 17,885 questions.
On the same 1,796 questions, the three judges found a contradiction in 36.5%, 55.7% and 91.4% of them. A rate is always one judge's reading, so we compare models inside one judge, and we say so whenever a finding holds under one judge but not the others.
90.7% of this edition's questions came from accounts on a free trial of Spark, our entry plan, and 47.3% from people who signed up through our search landing pages in August. The search-page accounts had shorter conversations, but the same rate of disagreement once mode, depth, line-up, answer count, topic and stakes were held fixed. This is a cross-section of Suprmind's traffic, not of AI use in general.
Every model is named by the exact version that answered, between July 30 and September 10, 2026. Claude Opus 5 and Claude Sonnet 5 have since been replaced by version 5.5. The October replay used Claude Opus 5.5. Perplexity means answers served through Perplexity's API.
We do not publish per-question data, because it contains our users' questions. The aggregate tables behind every chart are available to researchers and journalists on request.
All 12 charts in light and dark, a fact sheet with the key numbers and how each one is measured, the method on one page, logos and a founder quote. The link appears as soon as you submit, and a copy goes to your inbox.
Your press kit is ready. We also sent the link to your inbox.
Open the press kitEvery chart on this page has Download, Embed and Cite buttons, and the charts are free to use under CC BY 4.0 with a link to this page. Cite it as: Suprmind Data Lab, Multi-Model AI Divergence Index, Q3 2026 Edition. For the aggregate tables, a custom cut of the data or an interview, email [email protected]. We handle your details under our privacy policy.
Each edition has its own window, judge and population, so we never compare rates across editions. The April edition stays on this page, with the corrections below.
Every eligible question re-checked by a blind judge with model names hidden and answers in random order. Checks by judges from three companies, a speaking-order replay and a fact-check of factual disputes.
A second edition was planned for July 2026. We stopped it on July 17. From June 24, a prompt bug had removed the instructions that make AI models challenge each other after the first turn, which made conversations look as if the models converged. This edition's window starts after the fix and the related changes.
Correction, October 2026. We re-checked the April 2026 edition with the stricter method behind this Q3 2026 Edition. Four things change.
What holds: finance still draws the most disagreement, and legal and medical questions much less. Re-judged, 67.8% of financial turns had a contradiction, against 36.0% of legal ones.
| April 2026 measure | As published (1,324 turns) | Re-checked, blind judge (1,042 turns) |
|---|---|---|
| Turns with a contradiction | 54.4% | 50.1% (April's judge on the same turns: 51.1%) |
| Turns with a correction | 72.1% | 55.9% (April's judge on the same turns: 69.6%) |
| Financial turns with a contradiction | 72.1% | 67.8% |
| Legal turns with a contradiction | 41.2% | 36.0% |
| Confident answers challenged: Claude | 33.8% (lowest) | 52.2% |
| Confident answers challenged: Gemini | 51.3% (highest) | 55.1% |
| Catch asymmetry, Perplexity over Gemini | 9.77x | Withdrawn |
| Silent turns | 0.9% | An upper bound (see point 4) |
The original April datasets are archived. The re-checked April aggregates are available on request at [email protected].
Charts
Here is every chart on this page as an image. They are free to use under CC BY 4.0 with credit to Suprmind Data Lab and a link to this page. Open a chart for the interactive version, then use its Download, Embed or Cite button.
Which AI models get corrected least
AI models judged corrected, model by model
Who a blind judge sided with, by line-up
Who corrects whom in a chain of AI answers
The last word: judged corrected by speaking position
AI model corrections rise with the stakes
How sure each AI model sounds
Where AI models disagree most, by topic
Do AI models converge over a conversation?
What AI models disagree about
Fact-checked AI claims, model by model
AI judges mark the answer they read first as correctedA recurring study by Suprmind Data Lab of how AI models disagree when they answer the same question. It counts contradictions and corrections between models on real questions from Suprmind, where up to five AI models answer in one conversation. This Q3 2026 Edition covers 17,885 questions from July 30 to September 10, 2026, re-checked by a blind judge, with judges from three companies checking the main findings.
When at least two of three judges had to agree, GPT, Perplexity and Claude answers were judged corrected by another model on 3 to 4% of answers, against 6.5% for Gemini and 7.4% for Grok. In our top line-up, a blind judge found Claude Opus 5 corrected on 8.7% of its answers, against 21.6% for Gemini 3.1 Pro and 21.0% for Grok 4.3. With two of three judges required to agree, the rates were 2.3%, 17.4% and 18.4%. "Judged corrected" is a judge's reading, not proof that an answer was wrong.
They draw more corrections. A blind judge found a correction in 45% of high-stakes questions and 28% of low-stakes ones, at the same point in the conversation, and judges from OpenAI, Anthropic and Google all saw the rise. The blind judge read the answers as about as sure on the high-stakes questions.
Finance and law. AI models contradicted each other on about 40% of financial questions and 20% of medical ones, at the same point in the conversation. Legal questions came second, close enough to finance that the two cannot be separated, and drew corrections as often as financial ones.
Not in our data. An answer a blind judge read as confident was contradicted by other models more often, but it was not judged corrected any more often. Gemini models rated above their teammates' average for sounding sure in every line-up, and under the OpenAI judge their most assertive answers were not judged corrected more often.
Most of the time a fact-check cannot settle it: of 411 factual disputes we could trace word for word, our fact-checker settled 22%. When a correction could be checked, the correcting model was right 26 times and wrong 11 times. When two fact-check judges from different companies agreed, they sided with the correcting model 18 times out of 23. Both read the same evidence, gathered from the web of late September 2026. These numbers come from our setup and a small sample.
The judge reads every question with model names hidden and the answers in a fresh random order. AI judges mark the answer they read first as corrected two to four times as often as the others, so a fixed order would penalise whichever model is read first. A random order spreads that effect evenly across the models. Judges from Anthropic and Google also re-read 3,262 of the same questions, and we say so whenever a finding depends on the judge.
The method. April's judges saw model names and read the answers in a nearly fixed order, and two different judges covered its window. Re-checked with this edition's blind judge, April's disagreement rate holds (50.1% against 51.1% on the same turns), its correction rate falls from 69.6% to 55.9%, and its provider rankings, including the Confidence Trap by provider and the 9.77x Catch Asymmetry, are withdrawn. The full correction is in the Edition History section.
Yes. Quote any number or chart with a link to this page. Charts are free to use under CC BY 4.0, and every chart has Download and Cite buttons. Cite it as: Suprmind Data Lab, Multi-Model AI Divergence Index, Q3 2026 Edition. We do not publish per-question data, because it contains users' questions. Aggregate tables are available to researchers and journalists at [email protected].
Run your next important question past several AI models in one conversation. Where they disagree is where to look twice.
Compare Plans
