Claude Hallucination Rate by Version
By Arun Agrahri, Founder of TrueStandard. Published .
In our tests of 29 September and 23 October 2026, Claude Opus 5.5 made up none of the 307 citation IDs it gave. Claude Opus 5 ran beside it and made up 5.2%, then 1.4%. We test every new Claude next to the version it replaced, so each gap comes from one day and one test. Only Haiku's gap was too wide to call luck on both of its test days.
Key takeaways
- Claude Opus 5.5 made up 0 of 154 citation IDs in our test of 29 September 2026, and 0 of 153 on 23 October.
- Claude Haiku 5.5 made up 7.8% against 19.0% for Claude Haiku 4.5 in our test of 8 October 2026. A week later it was 8.1% against 25.6%.
- Claude Sonnet 5.5 and Claude Fable 5.1 stayed too close to the old versions to call, in our tests of September and October 2026.
- On 100 harder questions in our October 2026 tests, Claude Opus 5.5 made up 3.4%, then 2.4%. Claude Haiku 5.5 made up 17.9%, then 19.0%.
Each new Claude against the version it replaced
We gave each pair the same thirty claims on the same day and asked for sources with DOIs. A DOI is the permanent ID a journal gives a paper, so a made-up one leads nowhere. Half the claims have real papers behind them, and half are ones we invented, where the right answer is to decline. We then looked up every ID.
Made-up citation IDs for each pair of Claude versions, 2026
| Old to new | Tested | New | New likely range | Old | Old likely range | Luck | Result |
|---|---|---|---|---|---|---|---|
| Opus 5 to Opus 5.5 | 29 Sep 2026 | 0.0% | 0.0 to 2.4% | 5.2% | 2.5 to 10.3% | 0.0045 | Fewer made up |
| Opus 5 to Opus 5.5 | 23 Oct 2026 | 0.0% | 0.0 to 2.4% | 1.4% | 0.4 to 5.1% | 0.22 | Too close to call |
| Sonnet 5 to Sonnet 5.5 | 29 Sep 2026 | 7.4% | 4.0 to 13.5% | 8.9% | 5.2 to 14.9% | 0.82 | Too close to call |
| Sonnet 5 to Sonnet 5.5 | 23 Oct 2026 | 4.3% | 1.9 to 9.7% | 11.5% | 7.1 to 18.0% | 0.060 | Too close to call |
| Haiku 4.5 to Haiku 5.5 | 8 Oct 2026 | 7.8% | 4.3 to 13.7% | 19.0% | 12.7 to 27.6% | 0.017 | Fewer made up |
| Haiku 4.5 to Haiku 5.5 | 15 Oct 2026 | 8.1% | 4.4 to 14.2% | 25.6% | 18.7 to 34.1% | 0.00027 | Fewer made up |
| Fable 5 to Fable 5.1 | 3 Sep 2026 | 1.4% | 0.4 to 4.9% | 2.2% | 0.7 to 6.2% | 0.68 | Too close to call |
New and Old show how many citation IDs no registry has issued, over three runs that day. Likely range is where the true rate most likely sits, 95 times in 100. Luck is the chance of a gap this wide if the two were equal, written as p in the text below. Under 0.05 is the usual bar. Compare inside one row only, because a rate can move between days.
Citation IDs Claude Opus 5.5 made up in our tests of 29 September and 23 October 2026. Claude Opus 5 made up 9 of 273 on the same days. We looked up every ID in Crossref, DataCite and doi.org, the registries that issue them.
Opus 5.5 made up no IDs on either day
In our test of 29 September 2026, Opus 5.5 made up 0 of 154 IDs and Opus 5 made up 7 of 135 (p = 0.0045). In our test of 23 October, Opus 5.5 made up 0 of 153 and Opus 5 made up 2 of 138 (p = 0.22), which is too close to call. Opus 5 also put real IDs on the wrong paper, and Opus 5.5 did not. Counting both errors, the pair stood at 0% against 7.4%, then 0% against 4.3% (p = 0.0004, then 0.011). That wider count is the gap both days support.
Read the Claude Opus 5.5 measurement
On harder questions in our test of 16 October 2026, Claude Opus 5.5 still made up 3.4% of its IDs. TrueStandard checks each claim and citation across four models from different labs, and it flags what none of them can back up.
Try the free claim checkerHaiku 5.5 cut its rate by more than half, while Sonnet and Fable stayed close
In our test of 8 October 2026, Haiku 5.5 made up 7.8% against Haiku 4.5's 19.0% (p = 0.017). On 15 October it was 8.1% against 25.6% (p = 0.00027). Sonnet 5.5 made up 7.4% against Sonnet 5's 8.9% on 29 September 2026 (p = 0.82), then 4.3% against 11.5% on 23 October (p = 0.060). Fable 5.1 made up 1.4% against Fable 5's 2.2% on 3 September 2026 (p = 0.68). Luck alone could explain the Sonnet and Fable gaps.
Harder questions raise every Claude rate
The thirty claims above lean on famous papers. We also asked 100 hard health and science questions from AA-Omniscience, a public question set from Artificial Analysis. Opus 5.5 made up 3.4% on 16 October 2026 and 2.4% on 23 October, and it gave an ID on about 63% of the questions. A model can lower its rate by answering less, so read the two columns together.
Made-up citation IDs on 100 hard questions, October 2026
| Model | Tested | Answered | Made up | Likely range |
|---|---|---|---|---|
| Claude Opus 5.5 | 16 Oct 2026 | 63% | 3.4% | 2.0 to 5.7% |
| Claude Opus 5.5 | 23 Oct 2026 | 64% | 2.4% | 1.3 to 4.6% |
| Claude Haiku 5.5 | 15 Oct 2026 | 51% | 17.9% | 13.9 to 22.8% |
| Claude Haiku 5.5 | 16 Oct 2026 | 53% | 19.0% | 15.0 to 23.7% |
| Claude Sonnet 5.5 | 15 Oct 2026 | 45% | 23.1% | 18.5 to 28.4% |
Answered is the share of questions with at least one citation ID. Made up is the share of those IDs that no registry has issued, over three runs that day. Likely range shows where the true rate most likely sits, 95 times in 100.
More effort helped Sonnet 5 and changed nothing for Opus 5.5
In our test of 26 September 2026, Sonnet 5 made up 14.7% of its IDs at low effort and 3.4% at max (p = 0.002). Opus 5.5 made up none of 466 IDs across low, medium and max. The gain has a price: Sonnet 5 cost $0.18 at low and $4.18 at max for the same 90 answers.
These rates cover memory alone, on a small test
We turned web search off, so these rates show what a model invents from memory. A model with search attached may score better, and then we would be measuring the search. Thirty claims give wide ranges, so small gaps can come from luck. We run each test three times because one run can point the wrong way. Most of these comparisons come back too close to call, and we publish those too.
Common questions
Does Opus 5.5 hallucinate?
Yes, though rarely in our tests. Claude Opus 5.5 made up 0 of 307 IDs on thirty claims on 29 September and 23 October 2026. On 100 harder questions it made up 3.4% on 16 October and 2.4% on 23 October.
What is the Claude Opus hallucination rate?
In our test of 29 September 2026, Claude Opus 5.5 measured 0.0%, likely between 0.0 and 2.4%. Claude Opus 5 measured 5.2% that day, likely between 2.5 and 10.3%.
How much does Claude hallucinate?
It depends on the model and the question. In our tests from September to October 2026, the current versions ran from 0.0% for Claude Opus 5.5 to 8.1% for Claude Haiku 5.5 on thirty claims. On 100 harder questions in October, they ran from 2.4% for Opus 5.5 to 23.1% for Sonnet 5.5.
Which Claude model hallucinates the least?
In our tests, Claude Opus 5.5 made up 0.0% on 29 September 2026 and Claude Fable 5.1 made up 1.4% on 3 September. Those are the lowest counts we have for Claude, and the likely ranges overlap, so we cannot rank them.
Who ran this test
Arun Agrahri, Founder of TrueStandard, runs this benchmark. No AI model scores it. Each citation ID is looked up in Crossref, DataCite and doi.org, and a person reads every odd result.
About TrueStandardCheck every citation before you publish
In our October 2026 tests, every Claude we ran made up IDs on hard questions. Paste a draft and four models from different labs check each claim and citation.
The claim checker is free. Pro is $20 a month for 250 drafts, Max $40 for 500, and Ultra $200 for 3,000.