Which AI Hallucinates the Least?
On 16 October 2026, three pro models invented the fewest sources: Grok 4.7 at 2.7% of the DOIs it gave, Claude Opus 5.5 at 3.4% and GPT-6.1 Sol at 4.3%. The test could not tell the three apart.
GPT-6.1 Sol gave a working source on 72% of the questions on 15 October and 75% on 16 October, the most of any model on both days.
— TrueStandard benchmark v20, three runs on each of 15 and 16 October 2026
The short answer
We asked eight models for sources on 100 hard health and science questions, three times each, on two days, then looked up every DOI they gave us. Grok 4.7 answered 30% of the questions, Opus 5.5 63% and GPT-6.1 Sol 84%. GPT-6.1 Sol gave a working source on 75% of them. A DOI is the permanent ID a journal gives each paper, so a fake one is easy to prove. Any score built on one column can be topped by saying nothing, so read the answer rate beside the error rate. Then read the tier. Fast and pro models cost different amounts and do different jobs, so they sit in separate tables. Then read both days. We ran the test on 15 and 16 October, and two calls from the first day did not hold on the second. Every model here made up sources on these questions.
GPT-6.1 Sol gave a working source on 75% of the questions on 16 October, and 72% the day before. That is the most of any model on both days. Grok 4.7 and Claude Opus 5.5 invented about as few sources, and they answered far fewer questions.
The measurement
We took 100 health and science questions from AA-Omniscience, a public question set from Artificial Analysis. We asked each model for up to three peer-reviewed sources with DOIs, and no model had web access. We ran the 100 questions three times on 15 October 2026 and three more times on 16 October. Claude Opus 5.5 ran on 16 and 23 October instead, so it has no 15 October row. Then we looked up every DOI in Crossref and DataCite, the two main registries for DOIs, and at doi.org. Each row shows one model on one day of three runs.
Fast tier
Each vendor's fast model. On made-up DOIs, Claude Haiku 5.5 and GPT-6 Luna were statistically indistinguishable on 15 October (p = 0.13). On 16 October Haiku invented fewer (p = 0.014). The two days disagree, so we do not rank them. GPT-6 Luna answered almost every question and gave a real source more often on both days. Gemini 3.8 Flash invented more than Grok 4.3 on 15 October (p = 0.0019), but not by enough to call on 16 October (p = 0.21).
Fast models on 15 and 16 October 2026
| Model | Day | Answered | Made-up DOIs | Likely range | Run spread | Real source |
|---|---|---|---|---|---|---|
| GPT-6 Luna | 15 Oct | 96% | 22.8% | 19.1–27.0% | 19.3–26.6% | 50% |
| GPT-6 Luna | 16 Oct | 98% | 26.8% | 22.8–31.2% | 22.9–33.3% | 51% |
| Claude Haiku 5.5 | 15 Oct | 51% | 17.9% | 13.9–22.8% | 16.7–19.4% | 39% |
| Claude Haiku 5.5 | 16 Oct | 53% | 19.0% | 15.0–23.7% | 15.8–22.9% | 41% |
| Grok 4.3 | 15 Oct | 71% | 34.2% | 30.5–38.1% | 33.0–35.0% | 28% |
| Grok 4.3 | 16 Oct | 72% | 36.9% | 33.2–40.7% | 34.7–38.4% | 30% |
| Gemini 3.8 Flash | 15 Oct | 96% | 42.5% | 39.1–46.0% | 41.0–44.4% | 26% |
| Gemini 3.8 Flash | 16 Oct | 97% | 40.3% | 36.9–43.7% | 39.4–41.3% | 28% |
Each row shows one model on one day, over three runs of the same 100 questions. Answered is the share of questions with at least one DOI. Made-up DOIs is the share of DOIs that no registry knows. Likely range shows where the true rate most likely sits, 95 times in 100. Run spread shows the lowest and the highest run. Real source is the share of questions with a DOI that leads to the named paper.
Pro tier
Each vendor's flagship, with Claude Opus 5.5 in place of Claude Sonnet 5.5 from 16 October. Opus 5.5 has no 15 October row. Its made-up rate went from 3.4% on 16 October to 2.4% on 23 October, a change that chance explains (p = 0.52). On 16 October, Grok 4.7, Opus 5.5 and GPT-6.1 Sol were statistically indistinguishable on made-up DOIs. On 15 October Grok 4.7 measured lower than GPT-6.1 Sol (p = 0.025), so the days disagree and we do not rank them. GPT-6.1 Sol gave a real source most often, and Gemini 4 Argon made up the most, on both days.
Pro models on 15 and 16 October 2026. Claude Opus 5.5 ran on 16 and 23 October.
| Model | Day | Answered | Made-up DOIs | Likely range | Run spread | Real source |
|---|---|---|---|---|---|---|
| GPT-6.1 Sol | 15 Oct | 85% | 7.5% | 5.0–11.1% | 6.9–8.4% | 72% |
| GPT-6.1 Sol | 16 Oct | 84% | 4.3% | 2.5–7.2% | 3.9–5.1% | 75% |
| Claude Opus 5.5 | 16 Oct | 63% | 3.4% | 2.0–5.7% | 1.6–4.6% | 58% |
| Claude Opus 5.5 | 23 Oct | 64% | 2.4% | 1.3–4.6% | 2.4–2.5% | 59% |
| Gemini 4 Argon | 15 Oct | 90% | 21.9% | 19.1–25.0% | 19.5–24.3% | 51% |
| Gemini 4 Argon | 16 Oct | 91% | 17.6% | 15.0–20.5% | 16.8–19.0% | 56% |
| Grok 4.7 | 15 Oct | 32% | 2.5% | 1.1–5.7% | 0.0–4.1% | 30% |
| Grok 4.7 | 16 Oct | 30% | 2.7% | 1.1–6.1% | 0.0–5.8% | 29% |
Each row shows one model on one day, over three runs of the same 100 questions. Answered is the share of questions with at least one DOI. Made-up DOIs is the share of DOIs that no registry knows. Likely range shows where the true rate most likely sits, 95 times in 100. Run spread shows the lowest and the highest run. Real source is the share of questions with a DOI that leads to the named paper.
How to read the table. Answered is the share of questions with at least one DOI. Made-up DOIs is the share of DOIs that no registry knows: not Crossref, not DataCite and not doi.org. Likely range is the band that holds the true made-up rate with 95% confidence, a Wilson interval over the three runs of that day. Run spread is the lowest and the highest single run on that day. Real source is the share of questions with at least one DOI that leads to the paper the model named.
This is benchmark v20. We compare the days and never add them together. Each model keeps its rows together, in the order of its Real source figure on 16 October. On 16 October, eight Grok 4.3 answers failed with a capacity error, and we asked those questions again once. The test cost $7.03 on 15 October and $6.23 on 16 October. The two Opus 5.5 days cost $3.23 and $3.20.
On 16 October, the Opus 5.5 API sent back no text on the same 20 of the 100 questions in every run. It did the same on one more question once, and it reported a content filter each time. On 23 October, it did the same on 19 of those 20 questions in every run, and on one of them once. We count these questions as not answered. On the other 80 questions on 16 October, Opus 5.5 answered 79% of the time. A third Grok 4.3 day on 23 October cost $0.87, and it is not in the table.
We also test each new frontier model against the one it replaced, three times, on the model release pages
Read the columns together. Grok 4.7 made up the fewest sources, and it answered under a third of the questions on both days. If you rank on made-up DOIs alone, you pick the model that says least.
Four more limits apply. First, the tables do not rank vendors. In each tier the lowest rates overlap, and two results from the first day did not hold on the second. Second, Claude Opus 5.5 ran on 16 and 23 October, not on 15 and 16 October. Its answer rate counts about 20 questions that a content filter blocked.
Third, the made-up column is a lower bound. About a third of the DOIs led to a real paper with a different title. Those DOIs count as wrong only after a person checks them. Fourth, do not compare a fast row with a pro row.
Corrections: on 15 October we found that our DOI reader cut some DOIs with brackets short, and we fixed it. On 16 October we found one real paper that the DOI agency of China registered. Neither Crossref nor DataCite covers that agency. We had counted it as fake five times, three of them for Grok 4.7. Our reader now checks doi.org too, and we corrected every figure above.
A low fabrication rate can mean the model said little
Grok 4.7 invented 2.5% of the DOIs it gave on 15 October and 2.7% on 16 October. It answered about 30% of the questions both days. GPT-6.1 Sol answered 84 to 85% and gave a working source on about three questions in four. Its rate of 7.5%, then 4.3%, was higher than Grok 4.7's by enough to call on the first day and not on the second. The fast tier shows the same split: Claude Haiku 5.5 answered about half the questions, GPT-6 Luna nearly all of them. A model that declines does not invent a source, and that is a fair choice. It is also why a table sorted on fabrication alone rewards silence. We showed the same thing in August with four older models.
What changed between the two days
We ran the identical test on two days, one day apart, and changed nothing. Gemini 4 Argon went from 21.9% to 17.6%. Chance alone would move it that far about 4 times in 100 (p = 0.042). Two calls from the first day did not hold. Gemini 3.8 Flash invented more than Grok 4.3 on 15 October and not by enough to call on 16 October. Claude Haiku 5.5 and GPT-6 Luna were too close to call on 15 October, and Haiku was lower on 16 October. On both days GPT-6.1 Sol gave the most real sources, Grok 4.7 answered about a third of the questions, and every model invented some.
Each model makes up its own sources
On 15 October the eight models then in the table gave 907 different DOIs that do not exist, and only 3 of them came from more than one model. On 16 October the eight models now in the table gave 859, and only 1 came from more than one model. The rest were each model's own invention, so when four models disagree about a source, the disagreement is a reason to look. The proof is still the lookup: a DOI resolves or it does not.
Harder questions move the rate more than the model
Grok 4.3 is the one model we measured in August and in October, on the same 100 questions. It invented 30.9% of its DOIs on 26 August, then 34.2% on 15 October, 36.9% on 16 October and 35.0% on 23 October. Only the 16 October reading sits further from August than chance would explain (p = 0.033), and the three October days agree with each other. So Grok 4.3 read about 35% in October against 31% in August, and one day in three does not confirm that it drifted. On our older set of 30 general claims, the same model invented 10.8%. The test changed the rate about three times over. Any hallucination rate you read is a rate on some set of questions, and it does not travel to another set.
What this means if you publish
Picking a model does not remove the risk. The lowest rate on this page is still 2.5% of DOIs, and the highest is 42.5%, more than 2 in 5. Look up every DOI before it ships. A citation exists or it does not, and you do not have to trust any model to find out, ours included.
Fake DOIs on 16 October that more than one model produced. On 15 October it was 3 of 907
October days on which Grok 4.3 sat further from its August rate than chance would explain, on the same 100 questions. It read 30.9% on 26 August, then 34.2%, 36.9% and 35.0%
DOIs on 16 October that led to a real paper, but not the one the model named. They sit outside the fabricated column, so that column is a lower bound
The fix is not to hunt for a single more accurate model. Every large language model predicts fluent, plausible text, so each one can be confidently wrong on its own. What changes the odds is agreement. When several independent models are asked the same thing and all land on the same answer, the chance they share the exact same hallucination drops sharply. When they disagree, you have found the precise claim to check by hand before it ships.
Questions about this benchmark
So which AI hallucinates the least?
No single model, on our test. On 16 October 2026, Grok 4.7, Claude Opus 5.5 and GPT-6.1 Sol invented 2.7%, 3.4% and 4.3% of their DOIs, too close to tell apart. Grok 4.7 answered only 30% of the questions. GPT-6.1 Sol answered 84% and gave a working source on 75% of them, the most of any model on both days. Among fast models, Claude Haiku 5.5 and GPT-6 Luna split: too close to call on 15 October, Haiku lower on 16 October. No model here is safe to cite without checking.
Why measure DOIs instead of asking whether the claim is true?
A DOI either resolves or it does not, so no judge is needed. Every benchmark scored by a language model inherits that model's bias toward its own answers. We argue that models cannot reliably grade themselves, so we cannot publish a benchmark that a model graded. There is no AI anywhere in our scoring. Only the DOI registries decide.
Why are there two tables?
Fast models and pro models cost different amounts and do different jobs. A pro model with a lower rate than a fast one tells you little about which to use for a given task. So we compare models only within a tier. Within each tier we also test whether a gap is real, and say so when it is not.
You have corrected these numbers before. Why trust the new ones?
Because we find our errors and publish them. On 15 October we found that our DOI reader cut short some older DOIs that use brackets, so six real papers counted as fake. On 16 October we found a real paper registered with China's DOI agency, which the two registries we query do not cover. It had counted as fake five times. Every defect so far has made rates look worse than they were. Every run, every scoring change and every old result is kept so the difference can be checked.
How often is this updated?
When the models change. The table this one replaces measured four models on 26 August 2026, and three of them had newer versions by October. We ran the new table on 15 October and again on 16 October, and swapped Claude Sonnet 5.5 for Anthropic's flagship, Claude Opus 5.5, on the second day. Opus 5.5 ran its second day on 23 October, and its rate did not move beyond chance. The questions stay fixed, so a rate from one date can be set beside a rate from another.
Do not pick a model. Check the citation.
TrueStandard runs your draft past four models and looks up what they cite, so a fake source gets caught before your name is on it. In our October test, every model on this page made up sources. One claim a day is free; Pro is $20 a month for 250 drafts.