We ran Gemini 4 Hydrogen against Gemini 3.8 Flash on 13 November 2026, the day after Google launched it with a claim of more than 5x fewer fabricated claims. At the same low thinking setting, Gemini 4 Hydrogen made up 0.9 percent of the citation IDs it gave, and Gemini 3.8 Flash made up 22.0 percent (p < 0.000001). Those rates come from one day, on a test of citations.
At each model's default setting the gap was smaller, 0.0 percent against 6.0, and Hydrogen declined more requests for real papers. Google has not said how it measured its figure, so our test can neither confirm it nor refute it.
The short answer
On 13 November 2026, at low thinking, Gemini 4 Hydrogen made up 1 of the 113 citation IDs it gave. Gemini 3.8 Flash made up 31 of 141. A citation ID, or DOI, is the permanent link a journal gives a paper, so a made-up one leads nowhere.
Citation IDs from each model, three runs at each level on 13 November 2026
| Model and setting | Made up | Likely range | Wrong | Wrong likely range |
|---|---|---|---|---|
| Gemini 4 Hydrogen, low | 0.9% | 0.2–4.8% | 0.9% | 0.2–4.8% |
| Gemini 3.8 Flash, low | 22.0% | 15.9–29.5% | 41.8% | 34.0–50.1% |
| Gemini 4 Hydrogen, high (default) | 0.0% | 0.0–3.7% | 0.0% | 0.0–3.7% |
| Gemini 3.8 Flash, medium (default) | 6.0% | 3.1–11.4% | 12.0% | 7.5–18.6% |
Made up counts IDs that no registry has issued. Wrong adds real IDs on the wrong paper. Likely range shows where the true rate most likely sits, 95 times in 100. A Fisher exact test shows how often luck alone opens a gap this wide. At low, p < 0.000001 for made up. At the defaults, p = 0.012 for made up and p = 0.00011 for wrong.
At low, Hydrogen was lower in every run, and each run cleared the usual bar of 0.05 by itself. At the defaults, no single run cleared it, though the pooled gap did. With real IDs on the wrong paper counted too, the low gap widens to 0.9 percent against 41.8.
What Google claimed
Google's launch post calls Hydrogen a major upgrade for teams moving from Gemini 3.8 Flash, cutting fabricated claims by more than 5x. It also reports a 96.7 percent honesty rate on its own internal test of agents. Unless you set another level, Hydrogen thinks at high.
These are Google's numbers, from Google's tests. The post names no public test and no thinking level for the 5x figure, so we could not re-run that test.
How we tested it
We gave both models the same thirty claims and asked for up to three peer-reviewed sources for each, with no web access. Fifteen claims have real research behind them. Fifteen are plausible statements we wrote ourselves, so the right answer is to decline. We ran each model three times at low thinking, and three more times at its default: high for Hydrogen, medium for 3.8 Flash. All twelve runs were on 13 November, through Google's own API.
Every ID went to Crossref and DataCite, the registries that issue DOIs, and we re-checked each dead one at doi.org. No model graded anything. The test cost $3.29. The full tables, with a control model, are on the Gemini 4 Hydrogen release page.
Where Gemini 4 Hydrogen still failed
Its one fake came at low thinking, on a claim about sleep loss and attention. It cited a real 2001 paper led by Doran as 10.4449/aib.v139i3.503. No registry has issued that ID. The two other sources in the same answer were correct.
At its default, high, it made up nothing, and it also answered less. It gave an ID on 38 of 45 asks about real research, against 45 of 45 for 3.8 Flash. It declined the claims on chatbots agreeing with their users and on transformer attention every time, and the minimum-wage claim once. The last two carry the two dead IDs we see most often, each in 23 of our tests.
At low, Gemini 3.8 Flash made up 31 IDs, 11 of them on one claim about antibiotic resistance genes, and it put 28 real IDs on the wrong paper. Neither model offered a source for any of the fifteen invented claims, at either setting.
The one fake had a real paper, an ID in the right shape and two correct sources beside it, so nothing in the answer marked it. A registry lookup and a second model catch it. TrueStandard does both: paste your draft, and four models from different labs check each claim and citation in about 60 seconds, with every disagreement shown.
What this result leaves open
- This is one day. We have not repeated the test, and a gap on another model has lost its significance on a second day before.
- The same setting name buys a different amount of thinking. At low, Hydrogen thought before every answer and 3.8 Flash before about half of them. That is why we also ran the defaults.
- The measure differs. We count false citations on thirty claims. Google has not said what its 5x figure counts. A gap in the same direction says nothing about the size of the other.
- We ran xhigh, the top level of Hydrogen, on the 15 real claims only. It made up 2 of 122 IDs, within chance of high and low, at about five times the cost of high per answer. We did not test either model with search turned on.
What to do with this
If you run Gemini 3.8 Flash at low thinking, Hydrogen cut made-up citations sharply on this test. If you run 3.8 Flash at its default, the gain was smaller, and Hydrogen cost about three times as much on the same answers. Google lists Hydrogen at $1.50 per million input tokens and $7.50 per million output tokens, a token being a word piece. It lists 3.8 Flash at half that until 1 January 2027, when the price of 3.8 Flash doubles.
Keep checking citations either way. In our run, the one fake from Hydrogen sat beside two real sources. A reader who finds a fake like that finds it after you publish.
Common questions
Does Gemini 4 Hydrogen hallucinate less than Gemini 3.8 Flash?
On citations, on one day, yes. On 13 November 2026, at low thinking, it made up 0.9 percent of its IDs against 22.0 percent (p < 0.000001). At each model's default it was 0.0 against 6.0 percent (p = 0.012). We have no second day yet.
Does this confirm Google's claim of more than 5x fewer fabricated claims?
No, and it does not refute it. Google has not said which test or setting the figure comes from. Ours counts false citations on thirty fixed claims. Both point the same way, but they measure different things.
What is the Gemini 4 Hydrogen hallucination rate?
On our citation test, it made up 0.9 percent of the IDs it gave at low thinking on 13 November 2026, likely between 0.2 and 4.8 percent. At its default it made up none of 99. That covers citations from memory with no web access. A rate for general answers needs a different test.
Can I trust Gemini 4 Hydrogen's citations?
Not without a check. Its one fake had a real paper and two correct sources beside it, so the answer gave no warning. Look up each DOI before you cite it.
Which thinking level should I use for citations?
On this test, Hydrogen made up almost nothing at low and nothing at high, but high cost more than three times as much and declined more asks about real research. 3.8 Flash did much better at its default, medium, than at low: 6.0 percent against 22.0.
Keep reading
Is There a Most Accurate AI Model?
The honest answer is no. The ranking changes with the task, the benchmark, and the month, and even the leader still hallucinates.
Why AI Is Confidently Wrong
AI is confidently wrong because a model has no internal sense of when it is wrong. In one Carnegie Mellon task, chatbots showed about 90 percent confidence at about 65 percent accuracy.
Should You Stop Using ChatGPT?
No. Keep using ChatGPT to draft, and stop trusting it unchecked. Researchers found AI made experts measurably worse on hard tasks, and it tends to agree with you when it should push back.
AI Hallucination Rates in 2026, by Model
On hard factual questions, AI hallucination rates across 26 top models run from 22 to 94 percent, per the Stanford AI Index 2026. Each number here names the benchmark it came from and what that benchmark measured.
TrueStandard vs FactCheckTool
These two tools look alike, but they solve opposite problems. One tells you if the media you read is fake. The other tells you if the draft you are about to publish is true.
What we measured on this model
Each release page carries our own fabrication data for one model version, measured against the model it replaced in the same vendor's line.
Check the IDs before your readers do
Gemini 4 Hydrogen's one fake sat beside two correct sources. TrueStandard checks your draft's claims and citations across four models from different labs.
TrueStandard is for writers who publish AI-assisted drafts. Paid plans check every claim and citation across four AI models before you hit publish.
Free: one check a day, no signup. Pro is $20 a month for 250 drafts.
Start Verifying →