Model release archive

AI model hallucination rates, measured at each release

Each page below gives one new model's citation hallucination rate, measured against the model it replaced on the same thirty claims with no internet. To see every model we have measured on one page, read which AI hallucinates the least.

Read this if you draft with a new model and want to know how often it invents a source before you trust its citations.

When a new model launches, the numbers you can find are the ones the vendor chose to publish. We run our own test instead. Every model gets an identical set of claims, with no search and no retrieval. We score it by whether the DOIs it produced actually exist and, from August 2026, by whether its verdict on a claim bends when a user pushes back. Each page below carries the result for one model version, alongside the model it replaced.

Every comparison stays inside one vendor's line and one tier. A Flash-tier model is measured against the Flash model before it, a flagship against the flagship before it. Ranking a cheap model against a frontier one and calling the gap a finding is the kind of number this benchmark exists to catch.

Releases we have measured

Google · November 18, 2026

Does Gemini 4 Neon make up fewer citations than Gemini 4 Carbon?

Yes, on our test day. It made up none of 127 citation IDs, and Carbon made up 5.4 percent. At the same setting, Neon thought before it answered and Carbon did not.

OpenAI · November 17, 2026

Does GPT-6.2 Sol make up fewer citations than GPT-6.1 Sol?

No. It got more citation IDs wrong than GPT-6.1 Sol on both test days. Its made-up rate alone was higher on 18 November and too close to call on 19 November.

OpenAI · November 17, 2026

Does GPT-6.2 Astra make up fewer citations than GPT-6 Astra?

No. On our second test day it made up 14.2 percent of its citation IDs against 2.9 for GPT-6 Astra, more in all three runs. The first day was too close to call.

Google · November 12, 2026

Does Gemini 4 Hydrogen make up fewer citations than Gemini 3.8 Flash?

One made-up ID in 113 against 31 in 141, at the same setting. At each model's default the gap was smaller, and Hydrogen answered less.

Google · November 11, 2026

Is Gemini Decision 1 better than Jev?

Same English accuracy as Jev 1.14, much less sure of its answers, and dearer on text.

Google · November 10, 2026

Does Gemini 4 Carbon make up fewer citations, or fold more under pressure, than Gemini 4 Argon?

Too close to Argon to call, on made-up citations and under pushback. Google says Carbon halves unsupported claims, and our test could not see that.

Anthropic · November 09, 2026

Does Claude Fable 5.2 fabricate fewer citations, or fold more under pressure, than Fable 5.1?

No fake citations on either test day, and one moved verdict in 180 pushes. On fakes alone, the gap to Fable 5.1 is too small to call.

OpenAI · October 26, 2026

Does GPT-6.1 Terra make up fewer citations than GPT-5.6 Terra?

Six made-up IDs in 99 against 20 in 111, on one day. The gap cleared the usual bar that day, and we have no second day yet.

MiniMax · October 19, 2026

Does MiniMax M3.1 Flash make up fewer citations than MiniMax M3?

No. It made up 38.9 percent of its IDs against 12.1 for M3, worse in every run, and it cited sources for claims we made up.

DeepSeek · October 17, 2026

Does DeepSeek V4.1 Flash 1017 make up fewer citations than DeepSeek V4.1 Flash?

The dated snapshot cites like the base model. It made up 5.5 percent of its IDs against 6.0, and almost the same fakes for the same papers.

Meta · October 15, 2026

Does Muse Spark Open fabricate fewer citations than Muse Spark 1.3?

It cites like Muse Spark 1.3. It invented 9.5 percent of its IDs against 8.4, and got the same share wrong once miscitations count.

Google · October 13, 2026

Does Gemini 4 Argon fabricate fewer citations than Gemini 3.1 Pro?

Too close to Gemini 3.1 Pro to call, on three test days. At high thinking it got none of its 110 IDs wrong.

Meta · October 12, 2026

Does Muse Wonder fabricate fewer citations than Muse Spark 1.3?

Fewer wrong citations than Muse Spark 1.3, by a gap that clears the bar once miscitations count. On invented IDs alone, the gap is too small to call.

Anthropic · October 07, 2026

Does Claude Haiku 5.5 fabricate fewer citations than Haiku 4.5?

Fewer fake citations than Haiku 4.5 in every run, at a tenth of the price. One in thirteen of its IDs is still invented.

Mistral · October 06, 2026

Does Mistral Large 4 make up fewer citations than Mistral Large 3?

Level with Large 3 on made-up IDs, at the highest rate in this test. Unlike Large 3, it declines claims that have no source.

OpenAI · September 29, 2026

Does GPT-6.1 Sol Pro fabricate fewer citations than GPT-6 Sol Pro?

Three invented IDs in 112 and none on the wrong paper. Lower than GPT-6 Sol Pro by a gap this test cannot confirm, and level with standard mode.

OpenAI · September 29, 2026

Does GPT-6.1 Sol fabricate fewer citations than GPT-6 Sol?

Five invented IDs in 218 over two days, and none on the wrong paper. Lower than GPT-6 Sol in every run, by a gap neither day confirms.

Anthropic · September 28, 2026

Does Claude Sonnet 5.5 fabricate fewer citations than Sonnet 5?

No confirmed gap from Sonnet 5 on two test days. The new part is a warning to check its sources, on more than half its answers.

Qwen (Alibaba) · September 23, 2026

Does Qwen3.8 Max Prime make up fewer citations than Qwen3.8 Max 0902?

The faster Prime version cites like Max 0902. It made up 14.5 percent of its IDs against 15.7, with the same fakes for the same papers.

Z.ai · September 23, 2026

Does GLM-5.3 Prime make up fewer citations than GLM-5.3?

Level with GLM-5.3 on made-up IDs. It got fewer citations wrong overall, a gap that clears the bar on one day and has no known cause.

OpenAI · September 22, 2026

Does GPT-6 Sol fabricate fewer citations than GPT-5.6 Terra?

Fewer wrong citations than GPT-5.6 Terra on two days. The gap missed the bar by a hair on the first and cleared it on the second.

OpenAI · September 22, 2026

Does GPT-6 Luna fabricate fewer citations than GPT-5.6 Luna?

Fewer invented IDs than GPT-5.6 Luna on two days, by a gap neither day confirms. More real IDs on the wrong paper. Both got about a third of their citations wrong.

Anthropic · September 22, 2026

Does Claude Opus 5.5 fabricate fewer citations than Opus 5?

Zero fake citation IDs in 154, then none in 153 on day two. Opus 5 got some wrong on both days.

xAI · September 21, 2026

Does Grok 4.7 fabricate fewer citations than Grok 4.6?

Fewer wrong DOIs, the IDs on cited papers, than Grok 4.6 on two days three weeks apart. Every fake it made pointed at a real paper.

OpenAI · September 04, 2026

How often does GPT-6 Astra make up a citation?

Made up 3.0% of its DOIs on 7 September 2026 (likely 1.0 to 8.4%) and 2.8% on 15 October (likely 1.0 to 7.9%).

Google · September 02, 2026

Does Gemini 3.8 Flash fabricate fewer citations than Gemini 3.7 Flash?

The newer model invented more sources in all three runs, and that is still not enough to call it worse.

Anthropic · September 01, 2026

Does Claude Fable 5.1 fabricate fewer citations than Claude Fable 5?

The most expensive model we can buy, measured against the one it replaced. Thirty claims, three runs, and every DOI (a paper's permanent ID) checked against Crossref, a registry that issues them.

Google · August 13, 2026

Does Gemini 3.7 Flash fabricate fewer citations than Gemini 3.6 Flash?

Three runs of the same test disagreed about which model was better.

xAI · August 12, 2026

Does Grok 4.6 fabricate fewer citations, or bend less under pressure, than Grok 4.5?

Two tests on one set of claims: made-up DOIs, and verdicts that moved when a user pushed back. Neither test separated the two generations.

Why a page per release

A model launch produces a burst of questions and almost no evidence. By the time anyone has run a real test, the attention has moved on. These pages are the test, kept at a permanent address so the answer is still here in six months. By then the question has changed from what shipped this week to which of these models should I actually trust.

The current standings put every model we have measured on one page, with the caveats that come with it.

Check it before you publish it

Paste a draft and four models check every claim and citation against each other. You see where they disagree, which is where the errors are.

The claim checker is free. Pro is $20 a month for 250 drafts, Max $40 for 500, and Ultra $200 for 3,000.

See pricing
No Training on Your Data · 60-Second Checks · Full Verification Reports

Hallucination rates by model family: Claude · Gemini · GPT

Sycophancy rates by model family: Claude · Gemini · GPT