AI model hallucination rates, measured at each release
Each page below gives one new model's citation hallucination rate, measured against the model it replaced on the same thirty claims with no internet. To see every model we have measured on one page, read which AI hallucinates the least.
Read this if you draft with a new model and want to know how often it invents a source before you trust its citations.
When a new model launches, the numbers you can find are the ones the vendor chose to publish. We run our own test instead. Every model gets an identical set of claims, with no search and no retrieval. We score it by whether the DOIs it produced actually exist and, from August 2026, by whether its verdict on a claim bends when a user pushes back. Each page below carries the result for one model version, alongside the model it replaced.
Every comparison stays inside one vendor's line and one tier. A Flash-tier model is measured against the Flash model before it, a flagship against the flagship before it. Ranking a cheap model against a frontier one and calling the gap a finding is the kind of number this benchmark exists to catch.
Releases we have measured
Does Gemini 4 Neon make up fewer citations than Gemini 4 Carbon?
Yes, on our test day. It made up none of 127 citation IDs, and Carbon made up 5.4 percent. At the same setting, Neon thought before it answered and Carbon did not.
Does GPT-6.2 Sol make up fewer citations than GPT-6.1 Sol?
No. It got more citation IDs wrong than GPT-6.1 Sol on both test days. Its made-up rate alone was higher on 18 November and too close to call on 19 November.
Does GPT-6.2 Astra make up fewer citations than GPT-6 Astra?
No. On our second test day it made up 14.2 percent of its citation IDs against 2.9 for GPT-6 Astra, more in all three runs. The first day was too close to call.
Does Gemini 4 Hydrogen make up fewer citations than Gemini 3.8 Flash?
One made-up ID in 113 against 31 in 141, at the same setting. At each model's default the gap was smaller, and Hydrogen answered less.
Is Gemini Decision 1 better than Jev?
Same English accuracy as Jev 1.14, much less sure of its answers, and dearer on text.
Does Gemini 4 Carbon make up fewer citations, or fold more under pressure, than Gemini 4 Argon?
Too close to Argon to call, on made-up citations and under pushback. Google says Carbon halves unsupported claims, and our test could not see that.
Does Claude Fable 5.2 fabricate fewer citations, or fold more under pressure, than Fable 5.1?
No fake citations on either test day, and one moved verdict in 180 pushes. On fakes alone, the gap to Fable 5.1 is too small to call.
Does GPT-6.1 Terra make up fewer citations than GPT-5.6 Terra?
Six made-up IDs in 99 against 20 in 111, on one day. The gap cleared the usual bar that day, and we have no second day yet.
Does MiniMax M3.1 Flash make up fewer citations than MiniMax M3?
No. It made up 38.9 percent of its IDs against 12.1 for M3, worse in every run, and it cited sources for claims we made up.
Does DeepSeek V4.1 Flash 1017 make up fewer citations than DeepSeek V4.1 Flash?
The dated snapshot cites like the base model. It made up 5.5 percent of its IDs against 6.0, and almost the same fakes for the same papers.
Does Muse Spark Open fabricate fewer citations than Muse Spark 1.3?
It cites like Muse Spark 1.3. It invented 9.5 percent of its IDs against 8.4, and got the same share wrong once miscitations count.
Does Gemini 4 Argon fabricate fewer citations than Gemini 3.1 Pro?
Too close to Gemini 3.1 Pro to call, on three test days. At high thinking it got none of its 110 IDs wrong.
Does Muse Wonder fabricate fewer citations than Muse Spark 1.3?
Fewer wrong citations than Muse Spark 1.3, by a gap that clears the bar once miscitations count. On invented IDs alone, the gap is too small to call.
Does Claude Haiku 5.5 fabricate fewer citations than Haiku 4.5?
Fewer fake citations than Haiku 4.5 in every run, at a tenth of the price. One in thirteen of its IDs is still invented.
Does Mistral Large 4 make up fewer citations than Mistral Large 3?
Level with Large 3 on made-up IDs, at the highest rate in this test. Unlike Large 3, it declines claims that have no source.
Does GPT-6.1 Sol Pro fabricate fewer citations than GPT-6 Sol Pro?
Three invented IDs in 112 and none on the wrong paper. Lower than GPT-6 Sol Pro by a gap this test cannot confirm, and level with standard mode.
Does GPT-6.1 Sol fabricate fewer citations than GPT-6 Sol?
Five invented IDs in 218 over two days, and none on the wrong paper. Lower than GPT-6 Sol in every run, by a gap neither day confirms.
Does Claude Sonnet 5.5 fabricate fewer citations than Sonnet 5?
No confirmed gap from Sonnet 5 on two test days. The new part is a warning to check its sources, on more than half its answers.
Does Qwen3.8 Max Prime make up fewer citations than Qwen3.8 Max 0902?
The faster Prime version cites like Max 0902. It made up 14.5 percent of its IDs against 15.7, with the same fakes for the same papers.
Does GLM-5.3 Prime make up fewer citations than GLM-5.3?
Level with GLM-5.3 on made-up IDs. It got fewer citations wrong overall, a gap that clears the bar on one day and has no known cause.
Does GPT-6 Sol fabricate fewer citations than GPT-5.6 Terra?
Fewer wrong citations than GPT-5.6 Terra on two days. The gap missed the bar by a hair on the first and cleared it on the second.
Does GPT-6 Luna fabricate fewer citations than GPT-5.6 Luna?
Fewer invented IDs than GPT-5.6 Luna on two days, by a gap neither day confirms. More real IDs on the wrong paper. Both got about a third of their citations wrong.
Does Claude Opus 5.5 fabricate fewer citations than Opus 5?
Zero fake citation IDs in 154, then none in 153 on day two. Opus 5 got some wrong on both days.
Does Grok 4.7 fabricate fewer citations than Grok 4.6?
Fewer wrong DOIs, the IDs on cited papers, than Grok 4.6 on two days three weeks apart. Every fake it made pointed at a real paper.
How often does GPT-6 Astra make up a citation?
Made up 3.0% of its DOIs on 7 September 2026 (likely 1.0 to 8.4%) and 2.8% on 15 October (likely 1.0 to 7.9%).
Does Gemini 3.8 Flash fabricate fewer citations than Gemini 3.7 Flash?
The newer model invented more sources in all three runs, and that is still not enough to call it worse.
Does Claude Fable 5.1 fabricate fewer citations than Claude Fable 5?
The most expensive model we can buy, measured against the one it replaced. Thirty claims, three runs, and every DOI (a paper's permanent ID) checked against Crossref, a registry that issues them.
Does Gemini 3.7 Flash fabricate fewer citations than Gemini 3.6 Flash?
Three runs of the same test disagreed about which model was better.
Does Grok 4.6 fabricate fewer citations, or bend less under pressure, than Grok 4.5?
Two tests on one set of claims: made-up DOIs, and verdicts that moved when a user pushed back. Neither test separated the two generations.
Why a page per release
A model launch produces a burst of questions and almost no evidence. By the time anyone has run a real test, the attention has moved on. These pages are the test, kept at a permanent address so the answer is still here in six months. By then the question has changed from what shipped this week to which of these models should I actually trust.
The current standings put every model we have measured on one page, with the caveats that come with it.
Check it before you publish it
Paste a draft and four models check every claim and citation against each other. You see where they disagree, which is where the errors are.
The claim checker is free. Pro is $20 a month for 250 drafts, Max $40 for 500, and Ultra $200 for 3,000.