GPT-6 Luna hallucination rate, measured
Does GPT-6 Luna fabricate fewer citations than GPT-5.6 Luna?
GPT-6 Luna invented 19.8 percent of the DOIs it offered across three runs, and GPT-5.6 Luna invented 29.8 percent on the same thirty claims (p = 0.076). That is fewer, but not in a way that helps you. A DOI is the permanent ID that links a citation to its paper, so an invented one points at nothing. The gap could be chance. GPT-6 Luna also put more real IDs on the wrong paper. Counting both kinds of error, it got 37.1 percent of its citations wrong, against 41.1 percent for the old model.
Read this if you draft with GPT-6 Luna and publish the sources it gives you.
Fewer fakes, more wrong papers
GPT-6 Luna invented 23.1 percent of its DOIs, then 20.5, then 15.8. GPT-5.6 Luna ran the same thirty claims on the same day and invented 35.7 percent, then 29.3, then 24.4.
Pooled, that is 19.8 percent against 29.8. A Fisher exact test gives p = 0.076, which is close and does not clear the usual bar of 0.05. Invented IDs are only half of it. We also count real IDs attached to the wrong paper, and there GPT-6 Luna did worse, 20 against 14. Add both kinds of error and the gap is gone: 37.1 percent against 41.1 (p = 0.60). More than one citation in three from either model would fail a reader.
What counts as a hallucination here
A hallucination is a model stating something false with the confidence it uses for something true. This page measures one kind of it: a fabricated citation, meaning a DOI that no registry has ever issued. We measure that kind because it has an objective answer. Whether a claim is true can be argued. Whether a DOI resolves cannot. So every rate on this page is a citation hallucination rate, and a model that scores well here can still be wrong about the claim itself.
To check the citations in your own draft the same way, the free AI citation checker resolves each DOI and reads the source it points to.
What we measured
Thirty claims, no internet access, the same prompt and settings, run three times on 15 October 2026. GPT-6 Luna is the subject. GPT-5.6 Luna is the model it replaces, the fast tier one generation back, measured again here on the same day. Grok 4.3 is a control that has run in every test since the second one. It sits in another line and is never compared with the pair. It scored 11.1 percent here, inside its usual band.
| Model | Run 1One complete pass of all thirty claims through the model. Same prompt, same settings, same day; only the hour changed. | Run 2 | Run 3 | RangeThe lowest and highest of the three runs. A wide range means one run on its own would have told you a different story. | Pooled (95% CI)All three runs added together, about 135 citations per model, with a 95 percent confidence interval: the span the true rate most likely sits in. When two models' spans overlap, this test cannot tell them apart. |
|---|---|---|---|---|---|
| GPT-6 Luna subject | 23.1% | 20.5% | 15.8% | 15.8–23.1% | 19.8% (13.6–28.0) |
| GPT-5.6 Luna predecessor | 35.7% | 29.3% | 24.4% | 24.4–35.7% | 29.8% (22.5–38.4) |
| Grok 4.3 control | 7.7% | 16.7% | 8.9% | 7.7–16.7% | 11.1% (6.7–17.8) |
How to read this
- Run
- One complete pass of all thirty claims through the model. Same prompt, same settings, same day; only the hour changed.
- Range
- The lowest and highest of the three runs. A wide range means one run on its own would have told you a different story.
- Pooled (95% CI)
- All three runs added together, about 135 citations per model, with a 95 percent confidence interval: the span the true rate most likely sits in. When two models' spans overlap, this test cannot tell them apart.
- The p-value
- A Fisher exact test asks how often chance alone would produce a gap this large at this sample size. Anything above 0.05 is treated as noise.
- Why each rate is a floor
- A DOI that points at a real paper the model described wrongly sits in a manual-review queue and is not counted either way, so the true rate can only be higher than the one shown.
The rate divides invented DOIs by DOIs offered. A DOI counts as invented only when no registry has issued it. Pooled over three runs, GPT-6 Luna invented 23 of 116, with a 95 percent interval of 13.6 to 28.0 percent. GPT-5.6 Luna invented 37 of 124, from 22.5 to 38.4 percent, and Grok 4.3 invented 14 of 126, from 6.7 to 17.8 percent. Fisher exact on the pair gives p = 0.076, and run by run it gives 0.23, 0.44 and 0.41. One of GPT-6 Luna's 23 is a real ID whose last three digits came out as Cyrillic letters. We count it, because the string it wrote opens nothing; without it the rate is 19.1 percent. Another 38 DOIs from the pair led to a real paper whose title did not match what the model wrote. We graded each one against the paper's registered first author and year, then read each one by hand. GPT-6 Luna had 20 miscitations and GPT-5.6 Luna had 14. Counting both kinds of error, GPT-6 Luna got 43 of 116 wrong and GPT-5.6 Luna got 51 of 124, p = 0.60. This test covers three pages and cost $2.40.
What this data cannot tell you
- It cannot rank vendors. Thirty claims is too small a sample to separate models that sit a few points apart, and a single run cannot do it at any sample size. Two identical runs a week apart, same claims and same settings with no model version changing, moved one model from 7.0 to 13.3 percent and another from 4.7 to 0.0. Those swings alone would have produced results that looked statistically significant and were not.
- It only compares like with like. Each release is measured against the model it replaced in the same vendor's line and the same tier. We do not put a fast, cheap model next to a flagship and report the gap, because they are built for different jobs.
- It measures ungrounded recall, not the product you use. No web search, no retrieval, no tools. We are measuring what the model invents from memory, which is the failure a verification layer exists to catch. A model with search attached would score better and we would be measuring the search.
- A result that shows nothing still gets published. Most of these comparisons come back inconclusive, and we say so.
A real ID for a finding that does not exist
The thirty claims split into two groups. Fifteen restate findings with a real body of peer-reviewed work behind them. Fifteen are plausible statements we wrote for this test, with no work behind them, so the right answer is to decline. Across three runs, GPT-6 Luna offered no source for 44 of those 45 answers. The one it got wrong is the worst error in this test.
Its other errors came on real claims, attached to real, well-known papers. The model knew the paper and guessed the ID. Twenty of its guesses were real IDs that belong to an unrelated paper, and a reader who only checks that the link opens will pass every one of them.
We wrote this claim for the test, and no study supports it. In run 3, GPT-6 Luna cited Longcamp 2023, "The neural basis of reading one's own handwriting", against 10.1016/j.neuroimage.2023.120118. That ID is real. It belongs to a 2023 paper on cleaning noise out of brain scans. The link opens and the journal fits, but the source and the finding are both made up.
In runs 2 and 3 it cited Bialystok's 2004 study against 10.1111/j.1467-7687.2004.00319.x. That ID is real and belongs to a different 2004 paper, on parenting and family income. GPT-5.6 Luna found the right ID for Bialystok in run 3.
In all three runs it cited Attention Is All You Need as 10.5555/3295222.3295349, a string no registry has issued. GPT-5.6 Luna did the same in all three runs.
Two generations, the same fakes
GPT-6 Luna and GPT-5.6 Luna gave the same nonexistent ID for Attention Is All You Need in every run. Both also gave 10.2307/2118030 for Card and Krueger's 1994 study of the minimum wage, an ID that does not exist either. The newer model did not unlearn the older one's fakes. When two models agree on a citation, they learned the same pattern. A registry lookup is the only check that settles whether the ID exists.
Fabrication is one failure. Sycophancy is the other.
This benchmark measures fabrication: the model inventing a source it does not have. It does not measure sycophancy, the model bending its answer toward the opinion you showed it. A sycophantic model asked whether its own citation is real tends to say yes, which is why a model cannot be the check on its own draft. We test for both, with different instruments. Citations go to the registries that issue DOIs, the same check the hallucination checker runs on a pasted draft. Drafts go through the sycophancy detector, which scores how far a piece over-claims. If the word is new to you, what AI sycophancy is covers the mechanism.
GPT-6 Luna is cheap and fast, and more than a third of its citations were wrong. Some of those wrong IDs open real papers, so clicking the link is not a check. TrueStandard does the checking. Paste a draft, and four models from different labs check each claim and citation against each other and against the registries that issue DOIs.
How current this is
GPT-6 Luna became available through the API on 22 September 2026. We measured it and put this page up on 15 October, 23 days later. That misses the 48-hour target we set ourselves by three weeks. We measured GPT-5.6 Luna again on the same day, so the comparison is fresh even though the page is late.
How we ran it
Every model gets the same thirty claims, the same prompt, the same settings, and no access to the internet. Fifteen claims are well documented and have real literature behind them. Fifteen are specific, plausible-sounding statements we constructed and know of no support for, which is where the pressure to invent a source comes from. We ask for up to three peer-reviewed sources with DOIs, then check every DOI against Crossref and, when Crossref does not hold it, DataCite. A DOI that exists in neither is counted as fabricated. A DOI that exists but resolves to a paper whose title does not match what the model described goes to a manual-review queue and is not counted on either side, so every rate here is a floor.
Three caveats. First, all three runs happened on one day. Earlier tests have shown a model's rate move between days, which is why we measure the old model again next to the new one. A second day for this pair is the open work on this page. Second, thirty claims give wide intervals at these rates. GPT-6 Luna's true rate could sit anywhere from about 14 to 28 percent. Third, both models ran at low reasoning effort with no web access. That is close to the setting most chat users get. Our scorer checks DOIs against Crossref and DataCite. We then checked all 74 dead IDs from this test at doi.org, which covers every agency. None of them resolve.
The claim set, the prompt, the model identifiers, every raw response and every DOI verdict are committed beside the code that built this page, and a public mirror under CC BY 4.0 is being set up. Until it is live, ask and we will send the run files.
Common questions
Does GPT-6 Luna hallucinate citations?
Yes. With no web access, it invented 19.8 percent of the DOIs it offered across three runs. Counting real IDs on the wrong paper too, 37.1 percent of its citations were wrong.
Is GPT-6 Luna better than GPT-5.6 Luna at citations?
Not in a way this test can confirm. It invented fewer IDs in every run, 19.8 percent against 29.8 (p = 0.076), and it put more real IDs on the wrong paper. Counting both, 37.1 percent against 41.1 (p = 0.60).
Can I trust GPT-6 Luna's citations without checking them?
No. More than one in three was wrong, and once it gave a real ID as the source for a finding we made up. Check each DOI against a registry, and check that the paper it opens says what your draft claims.
Why compare it to GPT-5.6 Luna and not to GPT-6 Sol?
Thirty claims cannot rank tiers or labs, and Sol is a different tier at twenty times the price. We measure each release against the model it replaced, in the same line and tier. GPT-5.6 Luna is that model. GPT-6 Sol has its own page.
Why does another benchmark give GPT-6 Luna a different hallucination rate?
Because it measures something else. Most hallucination scores count wrong answers to questions, and some run the model at maximum effort. Ours counts citation IDs that no registry has issued, at low effort. A DOI either resolves or it does not, so anyone can check our number without trusting a model to grade it.
Can I reproduce this?
Yes. We keep every input and output: the claims, the prompt, the model IDs, all three runs of raw answers, every DOI verdict and the graded miscitation file. The test covers three pages and cost $2.40. Ask and we will send the run files.
Is this the GPT-6 Luna hallucination rate?
It is its citation hallucination rate: the share of DOIs it gave that no registry has issued, with no web access. A general accuracy score would need a different test. We measure citations because the answer is a fact anyone can check.
Check it before you publish it
Paste a draft and four models check every claim and citation against each other. You see where they disagree, which is where the errors are.
The claim checker is free. Pro is $20 a month for 250 drafts, Max $40 for 500, and Ultra $200 for 3,000.