AI Reliability

When AI Cites Studies That Don't Exist

AI can invent whole sources: court cases, studies and DOIs. A DOI is the permanent ID that links a citation to its paper. AI cites fake ones with the same confidence it uses for real ones. Here is why it happens, the damage so far, and how to catch a fake before your name is on it.

When AI Cites Studies That Don't Exist

Yes, AI often cites sources that do not exist. Independent tests put the fake-source rate anywhere from 18 percent to more than 50 percent, depending on the model. Language models invent studies, court cases and DOIs that look completely real. They state them in the same confident voice they use for real ones. Check every citation before you publish it.

This is not a rare glitch that a better model will iron out. It follows from how these systems write text. It has already led to sanctions for lawyers, slipped past peer review and forced newsrooms into mass corrections. This guide walks through the recorded cases and the cause behind them. It covers a subtler failure that most citation checkers miss. And it gives you a practical way to catch a fake source before it ships with your name on it.

Invented sources have already reached court filings and medical journals

The people it caught were professionals with every reason to check. They trusted the output because it looked right.

The original example is Mata v. Avianca. In 2023, New York attorney Steven Schwartz used ChatGPT to write a brief. It cited six court cases that did not exist, including a made-up Varghese v. China Southern Airlines. ChatGPT even told him the cases could be found in Westlaw and LexisNexis. Judge P. Kevin Castel sanctioned the lawyers and their firm 5,000 dollars, and the case made headlines worldwide (opinion and order). It was not a one-off from a careless solo lawyer. In April 2026, Sullivan and Cromwell admitted to a federal bankruptcy judge that one of its filings held AI-invented citations. It is one of the most prestigious firms in the world. There were more than 40 errors in all, including quotes attributed to the court that it never said. Opposing counsel caught them, and the firm's own review did not (reported by Bloomberg Law and CNN, April 2026).

These cases are part of a larger count. Damien Charlotin keeps a public database of court decisions that involve AI-hallucinated content. By mid-2026 it held more than 1,600 cases worldwide, and the count climbs almost daily. The problem reaches beyond law, too. In May 2026, The Lancet published an audit by the Columbia University School of Nursing. It checked 97 million references across 2.5 million biomedical papers. It found 4,046 fake citations, sources that are in no science database. They sat in 2,810 published, peer-reviewed papers (Columbia). A separate Nature investigation that year made its own estimate. More than 110,000 papers published in 2025 may carry invalid, AI-made references (Nature).

A few of the recorded cases

Case What the AI invented What it cost Year
Mata v. Avianca Six court cases, including a fictitious Varghese v. China Southern Airlines $5,000 sanction and worldwide headlines 2023
Sullivan & Cromwell filing ~28 fabricated or misquoted citations in a court brief Public apology to a federal judge 2026
Columbia / The Lancet audit 4,046 fake references across 2,810 papers Invented sources inside peer-reviewed science 2026
Nature investigation An estimated 110,000+ papers with invalid AI references Polluted scientific literature 2026

The common thread is not carelessness. A fake citation looks just like a real one, so normal review passes right over it. For the full running list across law, academia and media, see our reference of AI fake-citation disasters.

A model copies the shape of a citation and never checks the source

Knowing why it happens tells you where to look when you check a draft.

A language model does not look a citation up in a database. It writes the most likely next token, one after another, from patterns in its training data. A token is a word or a piece of a word. A citation has a predictable shape: an author, a year, a journal, a volume and a page range. So the model can write a well-formed reference with nothing real behind it. The format is easy to copy. The model never checked the paper it points to. That is the core of the problem, and it traces back to the structural cause of AI hallucination.

The obvious fix is to give the model a real database to draw from. It helps, but it does not solve the problem. A 2024 Stanford study tested the leading grounded legal research tools, the ones sold to end hallucination. Lexis+ AI and Westlaw's AI-Assisted Research still invented between 17 and 33 percent of the time. Westlaw's tool was wrong about a third of the time (Stanford HAI). Retrieval, where the tool looks up real documents first, narrows the gap without closing it.

The raw rates are worse than most people assume. In controlled studies, ChatGPT-3.5 invented roughly half of its sources. That was 47 percent in one Cureus analysis and 55 percent in Scientific Reports. GPT-4 cut that to about 18 percent, which is real progress. Yet among its genuine citations, it still got key details wrong about a quarter of the time (Walters and Wilder). Newer does not mean safe, either. A 2026 benchmark of legal citations found GPT-5.1 invented sources at 6.57 percent. The GPT-4o release a year earlier was at 1.23 percent (Who Checks the Citations?). The number moves with the model and the topic, and it never reaches zero.

The harder failure: a real source that does not say what the AI claims

Most people picture one kind of fake citation: a source that does not exist. That one is fairly easy to catch. The more dangerous failure is a real source, cited correctly, that does not say what the AI claims it says.

The same benchmark that counted fake legal citations sorts the failures into five types. Checking whether a link works catches only the first. The model can point you to a real paper that argues the opposite of the claim. It can attach a real, working DOI that opens an unrelated study. It can quote a source that never contained the quote. Each of these passes the shallow test of whether the link works, and each is still wrong.

Five ways an AI citation fails

Failure type What it looks like Caught by a link check?
Non-existent source The paper or case does not exist at all Yes
Wrong source A real, working DOI that opens an unrelated paper No
Wrong location Real source, but the wrong page or section No
Misquote A quote attributed to a real source that never said it No
Misrepresentation Real source that does not support the claim No

This is not a rare edge case. Researchers checked ChatGPT-5's citations against the papers it cited. Roughly a third of its claims carried errors against the real, correctly cited source (Journal of Technology in Behavioral Science, 2026). A separate study looked at fake citations that came with a DOI. Of those, 64 percent resolved to a real but unrelated published paper. Someone who clicks the link lands on a real article (EurekAlert release on the JMIR Mental Health study, 2025). So a citation checker that only confirms the link loads gives you false confidence. The question that matters is whether the source says what the AI says it says.

Peer reviewers' AI tools have flagged real references as hallucinated

The failure runs in both directions. In April 2026, a researcher posted on r/LanguageTechnology about a peer reviewer. The reviewer had used an LLM to check the paper, and it flagged real PubMed-indexed citations as fake. The reviewer comment: "Seems to be a hallucinated reference, duplicate or erroneous references," followed by a list of supposedly faked citations. The references were real. (r/LanguageTechnology thread) A preprint from the same time lists 17 similar cases. In each, an LLM-based peer-review system wrongly called real PubMed citations fake. (clawrxiv preprint)

This loop is why single-vendor self-checking cannot be trusted. The same kind of model that produces fake references is sometimes used to check references. It then wrongly rejects real ones, so the error runs both ways.

The Lancet audit found a steep rise in fake medical references

On May 7, 2026, a peer-reviewed letter in The Lancet reported a first. It was the first large-scale estimate of AI-driven fake citations in scientific literature with a sound method behind it. An audit of 2.5 million biomedical papers found a 12-fold rise in fake sources since 2023. About 3,000 published medical papers were found to contain fake citations. (Nature coverage · STAT coverage · EurekAlert summary)

Two implications matter for anyone publishing under their own name.

The rise tracks the spread of AI-assisted writing

The likely cause is researchers who draft with AI and cite what the model returns, with no independent check. The 12-fold growth curve follows the rollout of ChatGPT, Claude and Gemini.

Automated screening will produce false positives

The same letter notes that automated screening will produce false positives. The way out is to screen with a different method from the one that introduced the errors. Otherwise the peer-review false-flag pattern repeats at industrial scale.

The lesson is the same for writers outside medicine. When you publish a claim, the cost of an unchecked AI-generated source can now be measured. Screening after publication is becoming standard, so the error no longer stays your private secret.

Asking the same AI to check itself repeats the error

Almost everyone reaches for the same instinct: ask the same AI whether its citation is real. It does not work, and knowing why points straight at what does.

A model asked to check its own output runs the same probability distribution that produced the citation. That distribution is the spread of answers the model finds likely. Say it invented a source because the pattern looked right. Ask whether the reference is real, and you get the same confident yes, for the same reason. Nothing independent has entered the loop. Pasting it into a fresh window of the same model is weaker than it feels, too. It has the same architecture, the same training and the same blind spots. Real checking needs a judge that was trained differently and does not share the original error. This is the core of why AI cannot check its own work.

A model grading itself shares its own blind spots. TrueStandard is built to close that gap. It runs your draft across four frontier models from different labs at once. Say only one model knows a source, because only that model made it up. The others fail to back it up, and that disagreement is the flag. What catches the fake is independence between the models, more than careful reading.

Two checks for every source: it exists, and it says what you claim

You do not need to become a forensic librarian. You need a habit and, for anything you publish at volume, a faster version of it.

Start by treating every AI citation as a claim to verify, not a fact you already have. Then go to the primary source and ask two questions. Does this source exist? And does it say what the draft claims? Checking only the first, whether the link works, is what lets misrepresented and misquoted sources through. For a single high-stakes citation, tracing it by hand is the gold standard and worth the few minutes. But checking every citation in every draft by hand on a deadline does not scale, so most people skip it and hope.

The faster version keeps the independence and drops the manual work. Ask several models trained by different labs the same question. Then read where they disagree. A source one model invented cannot be backed up by models that never shared that error. So a fake citation shows up as a split. When independent models agree, that agreement is a real signal the source is sound. When they split, they point you to the exact citation a person needs to confirm. You do not have to reread the whole draft to find it.

Ways to check an AI citation, compared

Method What it catches Time cost
Ask the same AI to check itself Little; it shares the original blind spot Seconds
Click the link to see if it resolves Only fully-invented sources; misses misrepresentation Seconds
Trace each citation to its primary source by hand Nearly everything, if you have the time High, per citation
Cross-check across independent models Fabrications, plus the exact claims to verify About 60 seconds

That last row is what TrueStandard does for you. Paste your draft. Four frontier models from different vendors check every claim and citation in parallel. In about 60 seconds you get back the ones they cannot agree on, ranked by how far apart they are. To learn the manual method first, read our guide to checking whether AI citations are fake. And how accurate is ChatGPT puts the wider error rate in context.

Frequently Asked Questions

Does ChatGPT make up citations and sources?

Yes, often. A language model writes the most likely text instead of looking up a real record. So it can produce a well-formed citation, with author, journal, year and DOI, and nothing real behind it. In controlled studies, fake-source rates ran from about 18 percent for GPT-4 to over 50 percent for GPT-3.5. Even tools grounded in real documents still invented 17 to 33 percent of the time. Treat any AI citation as unproven until you check it.

Why does AI invent citations?

A citation has a very predictable structure. So a model can write a convincing one from pattern alone. The model was never checking a database. It was predicting what a plausible source looks like. The format is easy to fake, and the model never read the source itself. That is why a citation can look right and still point to nothing.

How common are fake AI citations?

Common enough to assume it will happen. Lab studies measured fake-source rates of roughly 47 to 55 percent for GPT-3.5 and about 18 percent for GPT-4. A 2024 Stanford study found grounded legal AI tools still invented 17 to 33 percent of the time. In the wild, a 2026 audit found fake citations in nearly 3,000 peer-reviewed medical papers. So the problem is reaching published work, not just draft output.

If the citation link works, is the source real?

Not necessarily. A working link tells you a page exists, not that it supports the claim. In one 2025 study, 64 percent of fake citations that included a DOI resolved to a real but unrelated paper. The link opened a real article that had nothing to do with the claim. You have to check whether the source says what the AI says it says, not just whether it loads.

Can AI check whether its own citations are real?

No, not reliably. Asking a model to check its own output runs the same process that produced the error. So it tends to confirm a fake citation with the same confidence it invented it. Re-pasting into a fresh window of the same model has the same blind spot. Reliable checking needs independence. That means a human tracing the primary source, or several models trained by different labs cross-checking each other.

How do I check if an AI citation is real?

Go to the primary source and ask two things. Does it exist? And does it actually support the claim? For a single important citation, trace it by hand. For a whole draft, run the claims across several independent models and look for disagreement. A fake source cannot be backed up by models that did not share the hallucination. That points you at exactly which citation to confirm.

Is it safe to use AI for academic research if I check the citations myself?

Yes, if checking means two things. Verify each citation against its primary source, not just that it exists. And verify that the quoted content matches the source. Most researchers check only that the citation appears to exist. The Lancet finding suggests that version is no longer enough. Multi-vendor verification cuts this to seconds per claim.

Which AI hallucinates the fewest citations?

It varies by model and topic, and newer does not reliably mean safer. One 2026 benchmark found a later GPT model invented legal citations more often than an earlier one. So picking a single best model is not a fix. Whichever one you choose still invents on the specific claims that matter most. The lasting answer is to check with independent models rather than trust one.

Keep reading

Is AI accurate for your field?

The failure modes change by profession. These break down what AI gets wrong in specific fields, with the incidents and the checks that catch them.

What we measured on this model

Each release page carries our own fabrication data for one model version, measured against the model it replaced in the same vendor's line.

Catch the Fake Citation Before Your Readers Do.

AI invents sources that look identical to real ones, and a working link does not mean the source is real. Paste your draft into TrueStandard. Four frontier models check every claim and citation in about 60 seconds. They flag the ones they cannot back up, before your readers or your editor find them.

TrueStandard is for writers who publish AI-assisted drafts. Paid plans check every claim and citation across four AI models before you hit publish.

Free: one check a day, no signup. Pro is $20 a month for 250 drafts.

Check Your Draft →