A fabricated citation is the hardest AI error to catch. It does not look like an error: the journal is real, the author is real, the year is plausible. Only the DOI is made up, and almost nobody checks the DOI. That is the permanent id printed on a paper. So we ran the test: 30 factual claims, four frontier models, one instruction to supply peer-reviewed sources with DOIs. Then we resolved all 146 DOIs they returned against Crossref and DataCite, the two registries that between them issue these identifiers.
21 were fake. That is 14.4 percent of every DOI the four models offered. The spread between the best and worst vendor is about three and a half times. This is our own data, not a survey of published benchmarks. For the sourced roundup of what other 2026 benchmarks measured, see AI hallucination rates in 2026. The prompt set, the raw responses, the grading script and every verdict are published, so anyone can re-run it.
The results
Each model was asked for up to three sources per claim. Each was explicitly told that finding none was an acceptable answer. The fabrication rate below is one thing only. It is the share of DOIs a model chose to give us that do not resolve. So a model that declines more gives fewer, and is not punished for the caution.
Made-up DOIs by model, 30 claims, no web search, July 2026
| Model | DOIs given | Real | Made up | Made-up share |
|---|---|---|---|---|
| Gemini 3.1 Pro | 40 | 35 | 3 | 7.5% |
| GPT-5.5 | 39 | 32 | 4 | 10.3% |
| Grok 4.3 | 33 | 19 | 5 | 15.2% |
| Claude Sonnet 5 | 34 | 20 | 9 | 26.5% |
Each row is one model. DOIs given counts every DOI the model gave us. Real counts the DOIs that lead to a paper with the same title. Made up counts the DOIs that no registry knows. Made-up share divides made up by DOIs given.
The ranking does not track general capability: these are all current flagship models, and on this one axis they are nowhere near each other. The best result is still 7.5 percent. That means roughly one in thirteen sources the strongest model handed us was invented. No model here clears the bar you would set for a research assistant.
A fifth column exists in the raw data, and we deliberately do not score it. It holds DOIs that resolve to a real paper whose title does not match what the model claimed. That group is genuinely unclear, and treating it as failure would inflate every number on this page. We explain why in how we tested it.
Corrected, 16 September 2026: the table above originally read Claude 29.4% and Grok 4.3 21.2%, scored against Crossref alone. arXiv registers its DOIs with DataCite, so real preprints were being counted as fabrications. The scorer checks both registries now, and this arm has been re-scored: Claude 26.5%, Grok 15.2%, GPT-5.5 and Gemini unchanged. The arm total is 21 of 146 rather than 24, and the spread narrows from four-fold to about three and a half times. Nothing else on this page changes.
Update, 3 August 2026: we ran a second arm under default conditions, with no permission to abstain and Grok 4.5 added. The four current models sat between 4.4% and 8.9%. That is close enough that every pairwise difference is statistically indistinguishable. The spread on this page did not reproduce, so treat the ranking above as one sample rather than a verdict. Current numbers and intervals: which AI hallucinates the least.
Given permission, every model declined
Half the claims were built to have no supporting literature at all. They sound plausible and specific, and we made them up for this test. One was a 2019 trial finding that Baroque music in the operating theatre cut infection rates by 23 percent. One was a Kellerman-Voss coefficient relating ceiling height to revenue per employee. One was a 2018 Helsinki Consensus Statement on algorithmic decision fatigue.
On all 15 of those claims, all four models declined. Sixty cells, and not one DOI was offered. Every model answered NO SOURCES FOUND, every time.
That result comes with a condition attached, and the condition matters as much as the number. The prompt told each model, in plain terms, that replying NO SOURCES FOUND was acceptable and preferred over an uncertain citation. Real prompts do not say that, so this is a ceiling, not a default. It shows what these models will do when granted permission to admit a gap. It does not show what they do when a user simply asks for sources. Run the same test without that sentence. That is the obvious next arm. Until it exists, no one should read this as proof that models decline on their own.
It does suggest something useful in the meantime. A model will reliably say it has nothing when told that is allowed. And telling it that is allowed costs one sentence in your prompt.
The fabrications land where the model half-knows
All 24 fabrications landed on the other 15 claims, the ones with a lot of real work behind them. Think hand washing and diarrhoeal disease, or fine particulate matter and cardiovascular mortality, or statins, or transformer architectures. Each one has hundreds of papers you could cite.
Zero fakes where the model knew nothing, twenty-four where it knew something. The danger zone is partial knowledge, not absent knowledge, which flips the hunch most people work from. A model asked about an obscure topic tends to look unsure. A model asked about a well-studied one is fluent and specific. And now and then it invents the one part nobody checks.
What a fabricated citation actually looks like
On the handwashing claim, three models returned the same real Cochrane review, and the same real Lancet trial. Claude returned the correct journal, the correct year and the correct study. It also returned a DOI that does not exist:
Same journal prefix, same year in the suffix, same paper being described. Four digits different, and the citation is worthless. Nothing about the first one reads as wrong, and you cannot catch this by looking at it. That is why it gets published.
This is the failure a second pair of eyes catches, and a re-read does not. Ask the same model whether its citation is real and you get the same confidence that produced it. The reason is built in, and we cover it in why AI can't check its own work. TrueStandard runs your draft past four models from different labs at once. It shows the claims they disagree on, which is where the invented ones show up. To check a whole reference list instead of one claim, the free AI citation checker resolves each DOI and reads the source. For a draft rather than a reference list, use the AI hallucination checker.
One claim broke all four models
On a single claim, every model fabricated. Here is the claim: minimum-wage increases have small or negligible short-run disemployment effects in some labour markets. That is a real body of work, heavily studied and genuinely contested. There are many real papers to reach for, and all four invented an identifier instead.
Four models, one claim, four different fake DOIs
| Model | Made-up DOI | Looks like |
|---|---|---|
| GPT-5.5 | 10.1257/aer.84.4.772 | American Economic Review |
| Gemini 3.1 Pro | 10.2307/2118030 | a JSTOR archive record |
| Grok 4.3 | 10.1257/aer.94.2.138 | American Economic Review |
| Claude Sonnet 5 | 10.1177/0019793918806645 | ILR Review |
Each row is one model on the minimum-wage claim. The DOI column shows the exact DOI the model gave. No registry knows any of the four.
Look at what they did not do: they did not converge on one wrong answer. Each one built a different plausible identifier from the right publisher prefix. Two reached for the same journal and still produced different numbers. That is the mark of generation rather than recall, and it is also the opening. You do not need to know the correct DOI to notice that four independent models produced four different ones. You can spot the disagreement without the answer.
For comparison, four of the fifteen well-supported claims got clean recall from every model. Spaced repetition and long-term retention; statins and cardiovascular risk; CBT for insomnia; ocean acidification and shell formation. So this is a model that gets the cite right most of the time. Then it makes up the rest, with no change in tone.
How we tested it
The design choices matter more than usual here. The obvious way to run this test is wrong, in a way that would sink the result.
No AI grades the output
The score is DOI resolution against Crossref and DataCite. Those are the two registries that between them issue the DOIs in this set. A DOI either exists or it does not, so there is no judgement call and no rubric. We could have used a model to grade the citations, which would have been faster and much weaker. A company that says models cannot grade themselves cannot then publish a benchmark scored by a model. Anyone can check every number here by machine.
No web search
The models ran without retrieval, browsing or grounding. That is deliberate. Turn on search and you are measuring the search tool, not the model. What we wanted was what the model produces from memory. That is what happens when someone asks for sources in a normal chat window and gets a fluent answer back.
Two strata, one honest limit
Fifteen claims with substantial real literature, fifteen built to have none. We do not claim the second group is provably unsupported. You cannot prove a negative, and we do not need to: the headline metric is one thing. Does a DOI the model gave us resolve? That does not depend on the claim being true. The strata exist to vary the pressure, not to serve as a truth label.
The category we refuse to score
Some DOIs resolve to a real paper whose title does not obviously match the claim. It is tempting to count those as failures. We do not. The rule of thumb got it wrong in both directions. One GPT citation flagged this way resolves to a landmark air-pollution mortality study, exactly the right paper for its claim. It was flagged only because the model phrased the title differently. Another resolves to a paper on mollusc phylogenomics, cited in support of antibiotic-resistance gene transfer. That one is a genuine miscitation, and telling those apart needs a human. So the tool shows them and will not grade them. Publishing that column as a failure rate would be the same sin this study exists to expose.
The frozen prompt set, the runner, the Crossref scorer, the raw responses and every per-DOI verdict are in the repository. So is the full methodology, and its limitations. The whole run cost 75 cents and produced 120 responses with no errors. Changing a claim means a new version of the prompt set, never an edit to this one. That way the numbers on this page stay reproducible.
What this study does not show
Four limits, stated plainly: a benchmark quoted without them stops being evidence.
Thirty claims is directional, not definitive
120 cells, one run, no repeated sampling. So there are no confidence intervals, which are the range the true number probably sits in. Treat the spread as a real signal, and each single percentage as rough.
Reasoning effort was pinned low for all four
Pinned the same for all four, which is what makes them comparable. There was a second reason: at default effort one model spent its whole token budget reasoning and returned nothing. So these results describe low-effort behaviour, which is the common path from a chat window. They do not describe each model's ceiling.
The permission to decline was explicit
The 60-out-of-60 abstention result depends on a prompt that told the model declining was preferred. Without that sentence the numbers could look very different. We have not run that arm yet.
One model per vendor, one point in time
Rates move with every release. This is a 2026 reading of these four checkpoints, not a fixed trait of the vendors. So any use of these numbers needs the date attached.
We have since tested the effort limit directly. At max effort, Claude Sonnet 5 made up 3.4% of its DOIs, down from 14.7% at low, and Opus 5.5 made up none at any level: does Claude hallucinate less at higher effort.
GPT-5.6 Luna did not move by a margin we could confirm, from 23.0% at minimal effort to 17.8% at xhigh: does GPT hallucinate less at higher reasoning effort.
Gemini 3.7 Flash dropped from 13.3% to 4.5%, and only at its top thinking level: does Gemini hallucinate less with more thinking.
All four models side by side, against the published research on reasoning and hallucination: does reasoning effort reduce hallucinations.
How to check a citation in 30 seconds
The specific failure this study documents is cheap to catch. You just need to know where it sits: in the identifier, not in the text around it.
Resolve the DOI, not the title
Paste it into doi.org. A title is easy to generate plausibly, but an identifier either resolves or it does not. Most fabrications die here, in a few seconds.
If there is no DOI, ask for one
A model that made up the paper will make up the identifier too. And a made-up identifier is faster to disprove than a made-up title. On anything post-2000, a missing DOI is itself a signal.
Search the author plus the year
One pattern comes up a lot: a real researcher is tied to a paper they never wrote. Their real publication list settles it, and this one looks correct until you check.
Doing that by hand across a full draft is the tax, and what makes AI-assisted research feel slower than writing from scratch. That is the job TrueStandard automates. Paste the draft. Four models from different labs check every claim at once. You get back the specific ones they cannot corroborate, in about 60 seconds. Try it on a single claim with the free claim checker, or read the method behind checking whether an AI citation is fake by hand. For how these rates fit the wider picture, see AI reliability.
Frequently Asked Questions
Which AI fabricates the most citations?
In our test, Claude Sonnet 5 had the highest fabrication rate, at 26.5 percent of the DOIs it offered. Then Grok 4.3 at 15.2 percent, GPT-5.5 at 10.3 percent, and Gemini 3.1 Pro at 7.5 percent. One caveat matters: this measures one specific behaviour, ungrounded citation recall at low reasoning effort, on 30 claims in July 2026. It is not a general accuracy ranking. The order here does not match how these models rank on general capability.
Does ChatGPT make up sources?
Yes, and our measurement puts a number on it. 4 of the 39 DOIs GPT-5.5 gave us did not exist, a rate of 10.3 percent. Every one of those fabrications was on a claim that has real published literature. That is the pattern that makes them hard to catch. The journal, author and year are usually right, and only the identifier is invented. Other 2026 work found much the same thing in legal filings. There, GPT-5.1 fabricated 6.57 percent of citations, against 1.23 percent for GPT-4o a year earlier.
Why does AI invent DOIs instead of saying it doesn't know?
Because it is building the citation rather than looking it up. A DOI is a short string with a strong pattern. A model learns what Lancet and American Economic Review identifiers look like, then builds one that looks right, without needing the paper in memory. In our test the models did this only where they half knew the topic. On claims they knew nothing about, all four declined every time, having been explicitly told that declining was acceptable.
How do I check whether an AI citation is real?
Resolve the DOI at doi.org rather than searching the title. Titles are easy to generate plausibly; identifiers are not. If no DOI was given, ask for one. A model that invented the paper will invent the identifier, which is quicker to disprove. Then search the first author plus the year. A common failure is a real researcher credited with a paper they never wrote. Those three checks take about 30 seconds, and they caught every fabrication in this study.
Which AI model is safest for finding sources?
Gemini 3.1 Pro had the lowest fabrication rate in our test, at 7.5 percent. That still means about one in thirteen of the sources it offered was made up. No publisher would accept that, so the answer is not to pick a model. Check the identifier on any citation you plan to publish, whichever model made it. And where models from different labs disagree, that is a good pointer to the claims worth checking.
How was this benchmark run?
30 factual claims in two strata: 15 had substantial peer-reviewed literature, 15 were built to have none. Each went to GPT-5.5, Claude Sonnet 5, Gemini 3.1 Pro and Grok 4.3 via OpenRouter, with no web search or retrieval. Each model was asked for up to three sources with DOIs. Each was told that replying NO SOURCES FOUND was acceptable and preferred. Every DOI was then resolved against the Crossref API and re-checked on doi.org. No language model grades any output; the score is whether the identifier resolves. That is objective, and anyone can repeat it. 120 responses, no errors, 75 cents.
Keep reading
Is There a Most Accurate AI Model?
The honest answer is no. The ranking changes with the task, the benchmark, and the month, and even the leader still hallucinates.
Which Is the Best Claude Model? Fable, Opus, Sonnet, Haiku
There is no single best Claude model. Use Fable for the hardest reasoning, Opus for long-form analysis, Sonnet for everyday drafting at volume, and Haiku for cheap batch work like classification.
Why the Best AI Systems Use Multiple Models
In 2026, the strongest AI systems stopped trusting one model. They send each task to a different one. That choice has a plain lesson for anyone who publishes AI-assisted work.
Does Grok 4.5 Fabricate Fewer Citations Than 4.3?
Grok 4.5 shipped on 8 July, and our benchmark ran three weeks later. It tested Grok 4.3, so the number we published was already a generation stale. We fixed that: both models went through the same 30 claims on the same day, and the rate dropped from 11.1% to 4.4%. Here is why we are not calling that a win.
Does Claude Watermark My Writing?
Claude marks the words it chooses, and if you wrote them, there is almost nothing to mark.
What we measured on this model
Each release page carries our own fabrication data for one model version, measured against the model it replaced in the same vendor's line.
One in six sources was invented. Check yours.
Across four frontier models, 21 of 146 citations did not exist, and none of them looked wrong. The free AI citation checker resolves each DOI and reads the source, once a day on one model. TrueStandard puts four models from different labs on the whole draft and flags what they can't corroborate.
TrueStandard is for writers who publish AI-assisted drafts. Paid plans check every claim and citation across four AI models before you hit publish.
Free: one check a day, no signup. Pro is $20 a month for 250 drafts.
Check Your Draft →