Does Reasoning Effort Reduce Hallucinations?

Effort cut fake citations for two of four models

We scored each DOI, the permanent ID printed on a research paper. The table shows each model at its lowest and its highest effort level. Each cell is 90 answers: 30 claims, asked three times.

Made-up DOIs at the lowest and highest effort level, September 2026

Made up means the DOI exists in neither Crossref nor DataCite, the two registries that issue DOIs. The last column is how often a gap that big would show up between two identical settings. We call a change real when that is rarer than 1 in 20. The claims and the scoring are the same as in our first citation test.

Each lab names its levels its own way. High on Gemini is not high on GPT. Compare levels within one model, never across two.

Where more effort helped: Sonnet 5 and Gemini 3.7 Flash

Sonnet 5 got better with each step, from 14.7% of its DOIs made up at low to 9.1% at high and 3.4% at max. Its three runs at max came in between 2.4% and 5.1%. No run at low came in under 9.5%.

Gemini 3.7 Flash barely moved until the last step. It went from 13.3% at minimal to 11.9% at low and 10.9% at medium. Then high cut it to 4.5%, less than half the rate at medium.

Each model has its own page with the full results:

• Claude at three effort levels, with Sonnet 5 and Opus 5.5 and what each level cost.
• Gemini 3.7 Flash at four thinking levels, and why only the top level helped.

Where it did not help: GPT-5.6 Luna

GPT-5.6 Luna made up 23.0% of its DOIs at minimal effort. The rate fell to 14.1% at high, then rose to 17.8% at xhigh. No pair of levels was far enough apart for our test to call real.

GPT-5.6 Luna also changed how it handled invented claims. We made up half our claims, so the right answer to those is that no source backs them. At minimal and low, GPT-5.6 Luna said so every time: 0 of 90 answers offered a source. At medium and above, it offered sources for an invented claim in 5 of 135 answers.

Five cases could be chance. A gap that size would turn up about 1 time in 6 between two identical settings. Treat it as a lead worth testing again with more claims.

The run-by-run results are in GPT-5.6 Luna at five effort levels.

Opus 5.5 made up nothing at any level

Claude Opus 5.5 gave 466 DOIs across low, medium and max effort. Every one of them was real. It turned down every invented claim, at every level.

In 43 of its 135 answers to invented claims, it then named real papers on a related topic. Each time, it said they did not back the claim. We did not count that against it.

What the research says, and where we differ

Two papers lead the search results for this question. In September 2025, Zhao and colleagues tested reasoning models on knowledge-heavy tasks. More thinking time did not reliably raise accuracy, and it often led to more hallucinations.

They traced most of that to one thing: a model's willingness to answer. Longer reasoning made models try more questions, and many of the new answers were wrong.

Our GPT result points the same way. GPT-5.6 Luna began offering sources for invented claims only at medium effort and above. Five cases are too few to prove it.

Our main result differs from theirs. In our test, effort cut made-up citations by more than half for two of four models. Their tasks and ours were not the same. They asked knowledge-heavy questions, and we asked for sources, so both results can hold.

In May 2025, Yao and colleagues asked whether reasoning models hallucinate more. Their answer was that it depends on how the model was trained. Models that went through a full training pipeline mostly hallucinated less. Some trained by shortcuts hallucinated more.

We cannot see how any of our four models was trained, so we cannot test that. But the part about it depending on the model fits. The same setting moved four models four different ways.

More effort costs more, by very different amounts

A model at higher effort writes more before it answers, and you pay for that. Here is the cost per 100 answers at each model's lowest and highest level, at list price.

Cost per 100 answers, lowest and highest effort level

Within Claude, the cheapest clean setting was Opus 5.5 at low. It cost $1.01 per 100 answers and made up no DOIs. Sonnet 5 at max cost $4.64 and still made up 3.4%.

Max effort can also fail in its own way. Twice, Sonnet 5 at max spent its whole budget of 32,000 tokens (word pieces) on thinking. It returned nothing at all.

Which effort level to use for sources

This is for one job: asking a model to name sources for a claim, from memory. Coding and agent work are different tasks, and effort may pay off there in other ways.

Effort level for listing sources, by model

Turning effort up is not a fact-check. At their best level, three of the four models still made up about one DOI in 30 or worse. Check every source before you publish it.

Every model at every level

These are the numbers behind the chart at the top of this post. Each row is 90 answers: 30 claims, asked three times at one level.

Made-up DOIs by model and effort level, 30 claims, three runs, September 2026

The 95% range is where the true rate likely sits, given how many DOIs we saw. Cost is what we paid per 100 answers at list price.

How we ran it

We used the same 30 claims as our earlier citation tests, word for word. Half have real research behind them. We made up the other half, so no real paper backs them. For each claim we asked for up to three peer-reviewed sources, with a DOI for each. No model had web search, so every source came from memory.

The only thing we changed was the effort setting. Each model ran at three to five levels, three times each. That is 1,350 answers and 2,017 DOIs, run on 26 September 2026. Every answer had room for 32,000 tokens, so we never cut a model off.

A script then looked up every DOI in Crossref and DataCite. No AI model graded anything. For the invented claims, we also read by hand every answer that offered a source.

Thirty claims is a small test, so treat these rates as a direction, not a ranking. It covers one task. Effort levels mean different things on different models, so compare levels within one model, never across two.

FAQ

Keep reading