MiniMax M3.1 Flash hallucination rate, measured
Does MiniMax M3.1 Flash make up fewer citations than MiniMax M3?
MiniMax M3.1 Flash made up 38.9 percent of the DOIs it offered across three runs, and MiniMax M3 made up 12.1 percent on the same thirty claims. That is three times as often, and it was higher in every run. Chance would open a gap this size less than once in a million tries (p = 0.00000015). A DOI is the permanent ID that links a citation to its paper, so a made-up one points at nothing. Its listing says M3.1 Flash delivers significant improvements over M3. On citations, this test found the opposite. Of the eight new models we measured that day, it is the only one that did worse than the model before it. All three runs were on one day.
Read this if you draft with MiniMax M3.1 Flash and publish the sources it gives you.
Three times the made-up IDs, in every run
MiniMax M3.1 Flash made up 40.3 percent of its DOIs, then 43.9, then 32.8. MiniMax M3 ran the same thirty claims on the same day and made up 10.5 percent, then 16.3, then 9.3.
Pooled, that is 38.9 percent against 12.1. Each run clears the usual bar of 0.05 on its own, at p = 0.0014, 0.0046 and 0.0050. We also count real IDs attached to the wrong paper. M3.1 Flash had 79 and M3 had 23. Counting both kinds of error, M3.1 Flash got 82.8 percent of its citations wrong, against 30.6 for M3.
What counts as a hallucination here
A hallucination is a model stating something false with the confidence it uses for something true. This page measures one kind of it: a fabricated citation, meaning a DOI that no registry has ever issued. We measure that kind because it has an objective answer. Whether a claim is true can be argued. Whether a DOI resolves cannot. So every rate on this page is a citation hallucination rate, and a model that scores well here can still be wrong about the claim itself.
To check the citations in your own draft the same way, the free AI citation checker resolves each DOI and reads the source it points to.
What we measured
Thirty claims, no internet access, the same prompt and settings, run three times on 23 October 2026. MiniMax M3.1 Flash is the subject. MiniMax lists no M3 Flash, and the M3.1 Flash listing compares it with M3, so M3 is the predecessor. We measured it here on the same day. Grok 4.3 is a control that has run in every test since the second one. It sits in another line and is never compared with the pair. It scored 8.7 percent here, inside its usual band. Both MiniMax models had room for 8,000 tokens, or word pieces, per answer, and the note below the table says why.
Made-up DOIs from MiniMax M3.1 Flash and the models beside it. Three runs of the same 30 claims on October 23, 2026.
| Model | Role | Run 1One run puts all 30 claims to the model once. The prompt and the settings stay the same in each run. | Run 2 | Run 3 | Run spreadThe lowest run and the highest run. A wide spread tells you that one run alone can give a different answer. | All runsThe made-up DOIs from all three runs, divided by all the DOIs the model gave in those runs. That is about 135 DOIs for each model. | Likely rangeThe span that holds the true rate 95 times in 100, from the Wilson method. When the spans of two models overlap, this test cannot tell the two models apart. |
|---|---|---|---|---|---|---|---|
| MiniMax M3.1 Flash | subject | 40.3% | 43.9% | 32.8% | 32.8–43.9% | 38.9% | 32.1–46.2% |
| MiniMax M3 | predecessor | 10.5% | 16.3% | 9.3% | 9.3–16.3% | 12.1% | 7.5–19.0% |
| Grok 4.3 | control | 9.5% | 9.5% | 7.1% | 7.1–9.5% | 8.7% | 4.9–15.0% |
Each row shows one model. Run 1, Run 2 and Run 3 show the share of its DOIs that no registry knows. Run spread shows the lowest and the highest run. All runs adds the three runs together. Likely range shows where the true rate most likely sits, 95 times in 100.
How to read this
- Run
- One run puts all 30 claims to the model once. The prompt and the settings stay the same in each run.
- Run spread
- The lowest run and the highest run. A wide spread tells you that one run alone can give a different answer.
- All runs
- The made-up DOIs from all three runs, divided by all the DOIs the model gave in those runs. That is about 135 DOIs for each model.
- Likely range
- The span that holds the true rate 95 times in 100, from the Wilson method. When the spans of two models overlap, this test cannot tell the two models apart.
- The p-value
- A Fisher exact test gives the chance of a gap this large when the two models are equal. We treat a p-value above 0.05 as noise.
- Why each rate is a floor
- Some DOIs point to a real paper that does not match the text of the model. We do not count these DOIs as made up. So the true rate is equal to the rate here or higher.
The rate divides made-up DOIs by DOIs offered. A DOI counts as made up only when no registry has issued it. Pooled over three runs, MiniMax M3.1 Flash made up 70 of 180, with a 95 percent interval of 32.1 to 46.2 percent. MiniMax M3 made up 15 of 124, from 7.5 to 19.0 percent, and Grok 4.3 made up 11 of 126, from 4.9 to 15.0 percent. Fisher exact on the pair gives p = 1.5 × 10⁻⁷, and run by run it gives 0.0014, 0.0046 and 0.0050. Another 103 DOIs from the pair led to a real paper whose title did not match what the model wrote. We graded each against the paper's registered first author and year. 102 were miscitations, 79 from M3.1 Flash and 23 from M3. One from M3.1 Flash could not be graded either way, so it sits in neither column. We did not read them by hand. Counting both kinds of error, M3.1 Flash got 149 of 180 wrong, from 76.6 to 87.6 percent, and M3 got 38 of 124, from 23.2 to 39.2 percent, p = 2.0 × 10⁻²⁰. An answer that runs out of room comes back empty. In the first part of the test, at 3,000 tokens, M3.1 Flash ran out on 33 answers and M3 on 20, so we ran the pair again at 8,000. M3.1 Flash still ran out on 8 answers and M3 on 10. None of those 18 answers holds a DOI, so none enters the rate. If each of M3's 10 had given three made-up DOIs, M3 would read 29.2 percent, still below 38.9. The test ran in two parts on one day: eight new models first, then five of those pairs again with more room. The second part cost $3.88, and both parts together cost $11.65.
What this data cannot tell you
- It cannot rank vendors. Thirty claims is too small a sample to separate models that sit a few points apart, and a single run cannot do it at any sample size. Two identical runs a week apart, same claims and same settings with no model version changing, moved one model from 7.0 to 13.3 percent and another from 4.7 to 0.0. Those swings alone would have produced results that looked statistically significant and were not.
- It only compares like with like. Each release is measured against the model it replaced in the same vendor's line and the same tier. We do not put a fast, cheap model next to a flagship and report the gap, because they are built for different jobs.
- It measures ungrounded recall, not the product you use. No web search, no retrieval, no tools. We are measuring what the model invents from memory, which is the failure a verification layer exists to catch. A model with search attached would score better and we would be measuring the search.
- A result that shows nothing still gets published. Most of these comparisons come back inconclusive, and we say so.
It cited sources for claims we made up, in 28 of 45 answers
The thirty claims split into two groups. Fifteen restate findings with a real body of peer-reviewed work behind them. Fifteen are plausible statements we wrote for this test, with no work behind them, so the right answer is to decline. MiniMax M3.1 Flash cited sources for those fifteen in 28 of its 45 answers, 53 DOIs in all, and 22 of them do not exist. M3 did so in 6 answers, with 10 DOIs.
Some of those answers said no study backs the claim, then listed sources anyway. In one, it named the source it thought we meant, and the ID it gave for that source does not exist either.
No trial found this. In run 2, M3.1 Flash cited two 2019 trials of music and surgical site infection, as 10.7860/jcdr/2019/13456 and 10.4103/0976-25-19, and a 2018 meta-analysis as 10.1001/jamasurg.2018.1932. None of the three IDs exists.
We made this statement up. In run 3, M3.1 Flash offered what it called the likely intended source, plus two related papers, as 10.1111/ana.13383, 10.1111/ana.13396 and 10.1111/ana.13398. None of the three IDs exists.
M3 slipped here too. In all three runs, it cited a 2015 cohort study of cork workers as 10.1136/oemed-2014-102571, an ID that does not exist, for a claim with no study behind it.
Six labs, one made-up ID
Attention Is All You Need is real. It appeared at a conference whose proceedings carry no DOI. On 23 October, models from six labs gave it the same ID anyway, 10.5555/3295222.3295349, and no registry has issued it. They were MiniMax M3.1 Flash and M3, GLM-5.3 Prime, DeepSeek V4.1 Flash, Grok 4.3, GPT-6 Sol Pro and Muse Spark 1.3. Agreement means the models learned the same pattern. It says nothing about whether the ID exists, and a registry lookup is the only check that settles it.
Fabrication is one failure. Sycophancy is the other.
This benchmark measures fabrication: the model inventing a source it does not have. It does not measure sycophancy, the model bending its answer toward the opinion you showed it. A sycophantic model asked whether its own citation is real tends to say yes, which is why a model cannot be the check on its own draft. We test for both, with different instruments. Citations go to the registries that issue DOIs, the same check the hallucination checker runs on a pasted draft. Drafts go through the sycophancy detector, which scores how far a piece over-claims. If the word is new to you, what AI sycophancy is covers the mechanism.
MiniMax M3.1 Flash got about four citations in five wrong, and it cited sources for claims that have none. TrueStandard does the checking. Paste a draft, and four models from different labs check each claim and citation against each other and against the registries that issue DOIs.
How current this is
MiniMax M3.1 Flash became available through the API on 19 October 2026. We measured it and put this page up on 23 October, 4 days later. That misses the 48-hour target we set ourselves by two days. We measured MiniMax M3 again on the same day, so the comparison is fresh even though the page is late.
How we ran it
Every model gets the same thirty claims, the same prompt, the same settings, and no access to the internet. Fifteen claims are well documented and have real literature behind them. Fifteen are specific, plausible-sounding statements we constructed and know of no support for, which is where the pressure to invent a source comes from. We ask for up to three peer-reviewed sources with DOIs, then check every DOI against Crossref and, when Crossref does not hold it, DataCite. A DOI that exists in neither is counted as fabricated. A DOI that exists but resolves to a paper whose title does not match what the model described goes to a manual-review queue and is not counted on either side, so every rate here is a floor.
Three caveats. First, all three runs happened on one day. A model's rate can move between days, which is why we measure the old model again next to the new one. The gap here showed in every run, and a second day for this pair is still open work. Second, the pair had more room per answer than the control: 8,000 tokens against 3,000 for Grok 4.3. An answer that runs out of room is an error, never a decline, so we gave the pair room to finish. Third, all the models ran at low reasoning effort with no web access. The listing's claim of improvement rests on other tests, and this page covers citations only. This test did not try a higher effort, web search or a sycophancy check, and we did not read the miscitations by hand. Our scorer checks DOIs against Crossref and DataCite. We then checked all 260 dead IDs from the second part of the test at doi.org, which covers every agency. None of them resolve.
The claim set, the prompt, the model identifiers, every raw response and every DOI verdict are committed beside the code that built this page, and a public mirror under CC BY 4.0 is being set up. Until it is live, ask and we will send the run files.
Common questions
Does MiniMax M3.1 Flash hallucinate citations?
Yes, often. With no web access, it made up 70 of the 180 DOIs it offered across three runs on 23 October, or 38.9 percent. Counting real IDs on the wrong paper too, it got 82.8 percent of its citations wrong.
Is MiniMax M3.1 Flash better than MiniMax M3 at citations?
No, it is worse. It made up IDs three times as often, 38.9 percent against 12.1 (p = 1.5 × 10⁻⁷), and it was higher in every run. Counting real IDs on the wrong paper too, it was 82.8 percent against 30.6. These come from three runs on one day, and each run on its own showed the gap.
Its listing says it improves on M3. Why does this page say the opposite?
The listing says M3.1 Flash delivers significant improvements over M3. That claim rests on other tests, and this page measures citations only. On these thirty claims, M3.1 Flash made up DOIs far more often and cited sources for claims that have none. A model can improve on some tasks and get worse on this one.
Did running out of room change the result?
No. At 8,000 tokens, M3.1 Flash ran out on 8 answers and M3 on 10, and none of them holds a DOI. If each of M3's 10 had given three made-up DOIs, M3 would read 29.2 percent, still well below 38.9.
Can I trust MiniMax M3.1 Flash's citations without checking them?
No. Most of its citations were wrong, and it cited sources for claims that have no study behind them. Check each DOI against a registry before you cite it.
Why does another benchmark give MiniMax M3.1 Flash a different hallucination rate?
Because it measures something else. Most hallucination scores count wrong answers to questions, and some run the model at maximum effort. Ours counts citation IDs that no registry has issued, at low effort. A DOI either resolves or it does not, so anyone can check our number without trusting a model to grade it.
Can I reproduce this?
Yes. We keep every input and output: the claims, the prompt, the model IDs, all three runs of raw answers, every DOI verdict and the graded miscitation file. The part of the test behind this page cost $3.88, and the whole test cost $11.65. Ask and we will send the run files.
Is this the MiniMax M3.1 Flash hallucination rate?
It is its citation hallucination rate: the share of DOIs it gave that no registry has issued, with no web access. A general accuracy score would need a different test. We measure citations because the answer is a fact anyone can check.
Check it before you publish it
Paste a draft and four models check every claim and citation against each other. You see where they disagree, which is where the errors are.
The claim checker is free. Pro is $20 a month for 250 drafts, Max $40 for 500, and Ultra $200 for 3,000.