Asking TypeSafe's Jev whether a sentence is a checkable claim costs about $0.0000017. Putting one claim through our full council costs about $0.0027, which is roughly 1,600 times as much. A ratio that size makes the decision look obvious: put the cheap model in front of the expensive one and stop paying to check sentences that are not claims.
We built it and measured it against the real pipeline on 18 September 2026. The bill moved by 1.7 percent, against a run-to-run spread of about 2 percent. That is no saving at all, and finding out why was worth more than the money would have been.
The short answer
In a research harness, filtering a 28-sentence draft down to its 13 real claims before sending anything to four chat models cut the cost by 26 percent. That number is real and it is not a production number.
In the actual pipeline the same filter saved 1.7 percent, which is inside the noise between two runs of the identical arm. The pipeline was already doing the filtering, in one call, for less than a tenth of a cent.
One 28-sentence draft, three runs per arm, 18 September 2026
| Measured on | Without Jev | With Jev | Saving |
|---|---|---|---|
| A research harness against a straw baseline | $0.04870 | $0.03581 | 26% |
| The real pipeline, three runs per arm | $0.03861 | $0.03795 | none measurable |
Fast tier, web search off, three runs per arm. The run-to-run spread on a single arm is about 2 percent, which is larger than the difference between the arms.
We are publishing the disappointing number because the exciting one was ours to check and we checked it. A 26 percent saving measured against a straw pipeline is not a saving; it is a description of the straw.
What the per-judgment price promised
Jev answers a fixed question about supplied material and returns a typed value with a probability. Asked whether each sentence of a draft is a checkable claim, it got all 84 judgments right in one call, in about a second and a half, for $0.000143.
Set that against the council. Every claim that survives extraction goes to several models in parallel, and a claim costs roughly $0.0027 through the full pyramid on the pro tier. The ratio between a shaped judgment and a knowledge judgment is about 1,600 to one. Hold on to the tier, because the measurements below run on the fast tier, which is cheaper per claim and skips the pyramid entirely.
Our test draft had 28 sentences and 13 of them were real factual claims. Stop sending the other 15 anywhere expensive and the bill should fall by something like a third. That is the calculation we ran, and it is the one we would have published if we had not tested it.
A ratio between two unit prices is an argument about a system you have not measured. It tells you what one judgment costs; it says nothing about how many judgments your system makes, or what it was already doing to avoid making them. That gap is where most cost-saving estimates live.
What the pipeline actually did
We put the triage step in front of the real path rather than a reconstruction of it. It splits the draft into sentences, scores each one, and sends only the survivors to extraction and then to the council. Fast tier, web search off, three runs of each arm so a single slow call could not decide the answer.
Without Jev the draft cost $0.03861. With Jev it cost $0.03795. That is 1.7 percent, and two runs of the same arm differ by about 2 percent, so the measurement cannot tell the two apart.
The triage itself worked exactly as it had in the harness. It kept all 13 of the claims we had labelled by hand, in all three runs, and dropped the opinions and transitions. The step is accurate. It is just not where the money is.
We are not claiming a latency win either. The runs came in at 40.3 seconds and 35.8 seconds, three runs each, which at this sample size is a difference between two mornings rather than between two designs.
Where the saving went
Extraction was already doing the job. Before anything reaches the council, one call to a single cheap model reads the draft and pulls out the claims, and that call costs about $0.00095. It also caps the result at twelve claims. By the time the expensive models see anything, the opinions and the transitions are already gone.
So the triage step was buying a filter the pipeline had bought once already, and buying it earlier does not make the second one cheaper. The saving we modelled came from comparing against a pipeline that sends every sentence to four models. Ours never did that.
There is a second reason, and it generalises further. Ensemble cost tracks how much the models write, not how much they read. In the harness run, one of the four models emitted 2,156 output tokens for 13 claims and 2,101 for 28. Cutting the input nearly in half moved its bill by about two percent, because the input was never what it was charging for.
That is worth carrying into any estimate involving a model that talks. Halving the prompt does not halve the invoice when the invoice is mostly output. A model asked for a considered answer writes about as much regardless of how much you gave it.
The part that worked
The other half of the integration was a grounding pass. For each claim that carries a source, fetch the source and ask whether it supports the claim as stated. Then keep the probability that comes back.
We ran it through the live resolver rather than a fixture. Two claims drawn from published standards documents, both true, both supported by the pages they cite. One fabricated statistic about cutting incidents by 40 percent, invented for the test and present in none of the sources. The whole pass took 0.69 seconds and cost $0.000296.
Three claims against two live sources
| Claim | Actually | Support probability |
|---|---|---|
| First claim, supported by the page it cites | True | 0.97 |
| Second claim, supported by the page it cites | True | 0.97 |
| Fabricated statistic, present in no source | Invented for this test | 0.03 |
Three claims against two live standards pages, fetched at run time, 18 September 2026. This is an existence proof that the wiring works end to end, not an accuracy figure.
Three claims is three claims. It shows the path works against real fetched text, and it is not a measurement of how often the answer is right. The accuracy figure for this task comes from a separate run of 108 hand-labelled claims. We had measured those with our benchmark's own wording rather than the wording the pipeline actually sends.
So we re-ran all 108 through the production path, with the production wording, and the headline did not move. It was 96.3 percent either way, 104 right out of 108, for three hundredths of a cent. The four misses were not the same four. Three of them were shared, including a true claim that a six-week delay carries the same penalty as a five-week delay, which it scored at 0.10. Rewording moved which items it got wrong without moving how many.
That is a smaller result than it sounds and we want to be exact about it. One run per wording means the per-tier differences are single items, not findings. What it does establish is that the accuracy we published was not an artefact of our benchmark's phrasing, which is the check we would have demanded of anyone else.
What a probability adds to a check we already run
We want to be precise about what is new here, because the easy version of this claim is false. Our citation checker already grounds claims against fetched sources using a language model, and has since long before this model existed. We wrote about what it catches in the run where four AI judges scored dead citations like live ones.
What Jev adds is a number on the same fetched text. A language model returns a verdict and an explanation; a decision model returns a probability from a distribution, on every claim, for a fraction of the cost. That is what makes a threshold possible: send anything under 0.5 to a person, let the rest through, and know what the line means.
It is an addition to a capability, not a new capability. Saying otherwise would have made a better headline and it would have been wrong.
Notice which half of the work moved and which did not. The cheap typed judgment is good at deciding what deserves attention. Deciding whether a claim is actually true still takes several models with sources in front of them, disagreeing with each other in public. That is the part we have never been able to make cheap.
What we shipped
The grounding pass went out on 18 September 2026 and is on by default, with a switch to turn it off if it misbehaves. It runs on claims whose sentence carries a citation our resolver can fetch, it carries the probability through to the report, and it never changes the verdict the council reached. Triage went out switched off, because a step that saves nothing measurable is not worth an extra call in the path and an extra thing that can break.
Here is what it looks like on a report. A claim whose sentence carries a citation we can fetch gains one extra badge next to the council's verdict, with the probability written as a whole percent. On a run through the live site, a claim that RFC 9110 defines HTTP semantics came back Verified, with CITED SOURCE SAYS THIS and 96 percent beside it. An invented line about RFC 9112 cutting request smuggling incidents by 40 percent came back Challenged, with CITED SOURCE DOES NOT SAY THIS and 3 percent. A claim carrying no citation gets no badge, because there is nothing to check it against.
That is one run, and it proves one thing: the feature is live and behaves the way the measurement said it would. It is not an accuracy figure. The accuracy figure is the 96.3 percent over 108 claims above. The badge shows up in the web report only, not in the command-line tool or the API, and it never moves the verdict the council reached.
Adjudication is not going to Jev at all. Asked to judge whether a claim was true with no source attached, it declined twelve of the thirteen claims we gave it and asked for a live lookup. That is the correct answer, and a clear statement of where it stops.
So the architecture is the one the measurements pointed at rather than the one the price list suggested. Jev takes the shaped judgments, the ones a piece of text in front of it can settle. Several models take the knowledge judgments. We measured each half separately first: the accuracy of its grounding over 108 claims, the speed claim everyone has been arguing about, and the question it refuses to answer.
What a per-unit price does not tell you
The 1,600 to one ratio was correct throughout. It was also the wrong number to plan with, and nothing about checking it harder would have helped, because the error was not in the number.
A unit price answers what one judgment costs. The question we needed answered was how many judgments the system makes, what it already does to avoid making them, and what the expensive part is actually charging for. All three of those live in the system, not in the price list, and none of them can be read off a vendor page.
Our harness got this wrong in the most ordinary way available. It compared against a version of the pipeline that sends everything to everyone, because that version is easy to write. Then the real pipeline turned out to be smarter than the straw man we had built to beat.
It is the same failure we spend our days catching in other people's drafts: a number that is true, measured against something nobody actually does.
Frequently Asked Questions
Does Jev reduce LLM costs in a verification pipeline?
Not in ours. Filtering a 28-sentence draft down to its 13 real claims before the expensive models saw it cut 26 percent in a research harness and 1.7 percent in the real pipeline. That was measured on 18 September 2026 over three runs per arm. Two runs of the identical arm differ by about 2 percent, so there is no measurable saving. The pipeline was already extracting claims in a single call costing about $0.00095 before anything expensive ran.
Why did the cost saving disappear in production?
Two reasons. The pipeline already separated claims from the rest of the draft in one cheap call, so the new step was buying a filter that had been bought already. And ensemble cost tracks how much the models write rather than how much they read. One model emitted 2,156 output tokens for 13 claims and 2,101 for 28, so halving the input barely moved its bill.
Was the 26 percent saving fake?
No, it was measured, and it was measured against the wrong baseline. The harness compared a filtered draft against a pipeline that sends all 28 sentences to four models. No real pipeline does that, ours included. The number describes the straw baseline rather than any saving available to a working system, which is why we are publishing both figures side by side.
What part of the integration actually worked?
The grounding pass. For each claim carrying a source, the source is fetched and the model is asked whether it supports the claim as stated. Against two live standards pages it returned 0.97 on both true claims and 0.03 on a statistic fabricated for the test, in 0.69 seconds for $0.000296. That is three claims, so it proves the path works end to end rather than measuring how often the answer is right.
Is a grounding check a new capability?
No, and it would be convenient to say so. Our citation checker already grounds claims against fetched sources with a language model. What the decision model adds is a calibrated probability on the same fetched text, on every claim, cheaply enough to run on all of them. That is what makes a threshold meaningful: under 0.5 goes to a person, and the line means something.
Did adding Jev make verification faster?
We cannot say that it did. The arms came in at 40.3 seconds and 35.8 seconds over three runs each, which at that sample size is noise rather than a result. The reason is the same as for cost: the time goes on models composing their answers, and that does not shrink because the input got shorter.
Is the triage step switched on?
No. It is built, it is accurate, and it is off. It kept all 13 hand-labelled claims in three runs of three, so it works. A step that saves nothing measurable is not worth an extra call in the path and an extra thing that can fail. The flag stays off until something changes about the cost shape that made it pointless.
Why not let Jev decide whether a claim is true?
Because it declines. Asked to judge 13 factual claims with the sources stripped off, it settled one and returned a needs-a-live-lookup probability between 0.51 and 0.82 for the other twelve. We gave it no option to abstain. It judges what the text in front of it can settle. Whether a claim is true in the world is a different question and it says so.
How should I estimate what a cheaper model would save me?
Count the calls your system actually makes, and check that both sides of your ratio describe the same configuration. Do not plan from the price of one judgment. Ask what your pipeline already does to avoid the expensive calls, and check whether the expensive part bills mostly for input or mostly for output. A model asked for a considered answer writes about the same amount regardless of prompt size, so shrinking the prompt does not shrink that invoice. A ratio between two unit prices is an argument about a system nobody has measured.
What is a shaped judgment versus a knowledge judgment?
A shaped judgment can be settled from the material in front of the model. Is this sentence a checkable claim, does this passage support this claim as written, how urgent is this ticket. A knowledge judgment needs a fact from the world that nobody has supplied, such as whether a claim is true. A decision model is good and very cheap at the first kind, and it declines the second.
Keep reading
Why AI Hallucinations Are Structural
DELEGATE 52, GPT-5.5, and a Purdue impossibility proof. Three April 2026 results that move 'hallucinations are structural' from take to documented fact.
AI Fake-Citation Disasters: A 2026 Reference
A running list of documented cases where AI made up citations in law, academia, and media. Every entry is sourced, and it is here to be linked, cited, and updated as new cases surface.
How to Check If AI Citations Are Fake
Four checks catch a fabricated reference before your readers do, and one of them is new. Here is the change: in 2026, a DOI that resolves no longer means the citation is real.
Can One AI Reliably Fact-Check Another AI?
If ChatGPT wrote the draft, can Claude safely verify it? Sometimes helpful, not sufficient by default. The reason is what these models share, not what they don't.
AI Detector vs Fact Checker
One asks who wrote this. The other asks is this true. Before you publish, only one of those questions protects your name, and most teams are watching the wrong one.
The claim is still the expensive part
Deciding which sentences are worth checking got cheap. Deciding whether they hold up did not. TrueStandard runs your draft past four or five models with sources in front of them and shows you every claim they disagree on.
Start Verifying →