GPT-6 Hallucination Rate by Version
By Arun Agrahri, Founder of TrueStandard. Published .
In our tests, GPT-6.1 Sol made up 2.9% of its citation IDs on 15 October 2026 and 1.8% on 23 October. GPT-6 Sol, tested beside it, made up 7.1% and 7.7%. In all five GPT steps we measured, the new version made up fewer IDs than the one it replaced on every test day. No single gap was wide enough to rule out luck.
Key takeaways
- GPT-6 Astra made up 3.0% of its IDs in our test of 7 September 2026, and 2.8% on 15 October. GPT-5.6 Sol, beside it, made up 8.8% and 7.3%.
- GPT-6.1 Sol made up 2.9%, then 1.8%, in our tests of 15 and 23 October 2026. It put no real ID on the wrong paper on either day.
- GPT-6 Luna, the fast tier, made up 19.8%, then 15.3%, in the same tests. GPT-5.6 Luna made up 29.8%, then 22.4%.
- On 100 harder questions in our tests of 15 and 16 October 2026, GPT-6.1 Sol made up 7.5%, then 4.3%. GPT-6 Luna made up 22.8%, then 26.8%.
- OpenAI moved the name Sol down one tier, so GPT-6 Sol replaced GPT-5.6 Terra, not GPT-5.6 Sol.
Each new GPT against the version it replaced
We gave each pair the same thirty claims on the same day and asked for sources with DOIs. A DOI is the permanent ID a journal gives a paper, so a made-up one leads nowhere. Half the claims have real papers behind them, and half are ones we invented, where the right answer is to decline. We then looked up every ID.
Made-up citation IDs for each pair of GPT versions, 2026
| Old to new | Tested | New | New likely range | Old | Old likely range | Luck | Result |
|---|---|---|---|---|---|---|---|
| GPT-5.6 Sol to GPT-6 Astra | 7 Sep 2026 | 3.0% | 1.0 to 8.4% | 8.8% | 5.0 to 15.1% | 0.096 | Too close to call |
| GPT-5.6 Sol to GPT-6 Astra | 15 Oct 2026 | 2.8% | 1.0 to 7.9% | 7.3% | 3.9 to 13.3% | 0.15 | Too close to call |
| GPT-6 Sol to GPT-6.1 Sol | 15 Oct 2026 | 2.9% | 1.0 to 8.1% | 7.1% | 3.5 to 13.9% | 0.20 | Too close to call |
| GPT-6 Sol to GPT-6.1 Sol | 23 Oct 2026 | 1.8% | 0.5 to 6.2% | 7.7% | 3.9 to 14.4% | 0.051 | Too close to call |
| GPT-5.6 Terra to GPT-6 Sol | 15 Oct 2026 | 7.1% | 3.5 to 13.9% | 11.4% | 6.8 to 18.5% | 0.35 | Too close to call |
| GPT-5.6 Terra to GPT-6 Sol | 23 Oct 2026 | 7.7% | 3.9 to 14.4% | 13.6% | 8.5 to 20.9% | 0.20 | Too close to call |
| GPT-6 Sol Pro to GPT-6.1 Sol Pro | 23 Oct 2026 | 2.7% | 0.9 to 7.6% | 6.5% | 3.2 to 12.8% | 0.21 | Too close to call |
| GPT-5.6 Luna to GPT-6 Luna | 15 Oct 2026 | 19.8% | 13.6 to 28.0% | 29.8% | 22.5 to 38.4% | 0.076 | Too close to call |
| GPT-5.6 Luna to GPT-6 Luna | 23 Oct 2026 | 15.3% | 9.9 to 22.8% | 22.4% | 16.0 to 30.5% | 0.19 | Too close to call |
New and Old show how many citation IDs no registry has issued, over three runs that day. Likely range is where the true rate most likely sits, 95 times in 100. Luck is the chance of a gap this wide if the two were equal, written as p in the text below. Under 0.05 is the usual bar. Compare inside one row only, because a rate can move between days.
Same-day comparisons in our tests where the new GPT made up fewer citation IDs than the one it replaced, from 7 September to 23 October 2026. None cleared the usual bar alone, so we report the direction and no size.
OpenAI moved the name Sol down one tier
GPT-5.6 shipped as Sol, Terra and Luna. GPT-6 ships as Astra, Sol and Luna. GPT-5.6 Sol was the flagship, and GPT-6 Sol sits below GPT-6 Astra at half the old price. So we compare GPT-6 Astra with GPT-5.6 Sol, and GPT-6 Sol with GPT-5.6 Terra. In our test of 15 October 2026 we ran GPT-5.6 Sol and GPT-6 Sol side by side. They made up 7.2% and 7.1% (p = 1.0).
Counting IDs on the wrong paper widens the gaps
A real ID on the wrong paper strands your reader too. In our test of 7 September 2026, GPT-6 Astra got 4.0% wrong against GPT-5.6 Sol's 12.0% (p = 0.032). In our test of 15 October, GPT-6 Sol got 9.1% wrong against GPT-5.6 Terra's 19.3% on 15 October (p = 0.050), then 7.7% against 23.7% on 23 October (p = 0.0016). GPT-6.1 Sol got 2.9% wrong against GPT-6 Sol's 9.1% on 15 October (p = 0.076).
In our tests of 2026, every new GPT made up IDs on at least one test day, and a link that resolves can still lead to the wrong paper. TrueStandard checks each claim and citation across four models from different labs, and it flags what none of them can back up.
Try the free claim checkerHarder questions raise every GPT rate
The thirty claims above lean on famous papers. We also asked 100 hard health and science questions from AA-Omniscience, a public question set from Artificial Analysis. GPT-6.1 Sol made up 7.5% on 15 October 2026 and 4.3% on 16 October, and it gave a working source on 72% and 75% of the questions. A model can lower its rate by answering less, so read the two columns together.
Made-up citation IDs on 100 hard questions, October 2026
| Model | Tested | Answered | Made up | Likely range |
|---|---|---|---|---|
| GPT-6.1 Sol | 15 Oct 2026 | 85% | 7.5% | 5.0 to 11.1% |
| GPT-6.1 Sol | 16 Oct 2026 | 84% | 4.3% | 2.5 to 7.2% |
| GPT-6 Luna | 15 Oct 2026 | 96% | 22.8% | 19.1 to 27.0% |
| GPT-6 Luna | 16 Oct 2026 | 98% | 26.8% | 22.8 to 31.2% |
Answered is the share of questions with at least one citation ID. Made up is the share of those IDs that no registry has issued, over three runs that day. Likely range shows where the true rate most likely sits, 95 times in 100.
More effort gave GPT-5.6 Luna no steady gain
In our test of 26 September 2026, GPT-5.6 Luna made up 23.0% at minimal effort, 14.1% at high and 17.8% at xhigh. Minimal against xhigh gives p = 0.35. It also began to offer sources for claims we invented as effort rose: 0 of 90 answers at minimal and low, 5 of 135 at medium and above.
These rates cover memory alone, on a small test
We turned web search off, so these rates show what a model invents from memory. A model with search attached may score better, and then we would be measuring the search. Thirty claims give wide ranges, so small gaps can come from luck. We run each test three times because one run can point the wrong way. Most of these comparisons come back too close to call, and we publish those too.
Common questions
What is the GPT-6 hallucination rate?
It depends on the tier. In our test of 7 September 2026, GPT-6 Astra measured 3.0%. In our test of 15 October, GPT-6 Sol measured 7.1% and GPT-6 Luna 19.8%. The table above gives each likely range.
Does GPT-6 Astra hallucinate?
Yes. In our tests, GPT-6 Astra made up 3.0% of its IDs on 7 September 2026, likely between 1.0 and 8.4%. On 15 October it made up 2.8%, likely between 1.0 and 7.9%.
What is the ChatGPT hallucination rate in 2026?
We do not measure the ChatGPT app. We test each model through its API with web search off. The app can search the web, so your results there may differ from the rates on this page.
Is GPT-6.1 Sol better than GPT-5.6 Sol?
In our test of 15 October 2026, GPT-6.1 Sol made up 2.9% and GPT-5.6 Sol made up 7.2%. The two sit in different tiers, so we report both rates and give no verdict.
Who ran this test
Arun Agrahri, Founder of TrueStandard, runs this benchmark. No AI model scores it. Each citation ID is looked up in Crossref, DataCite and doi.org, and a person reads every odd result.
About TrueStandardCheck every citation before you publish
In our 2026 tests, every GPT we ran made up IDs on some day. Paste a draft and four models from different labs check each claim and citation.
The claim checker is free. Pro is $20 a month for 250 drafts, Max $40 for 500, and Ultra $200 for 3,000.