Is AI accurate for customer support?
Not on its own. On realistic support benchmarks the best AI agents solve fewer than half of the tasks, and in production they confidently invent policies, prices, and refunds that were never real. A support bot is genuinely useful for triage, drafting replies, and answering well-documented questions, but any answer that commits your company to a policy, a price, or a refund has to be grounded in your actual source of truth and checked, because a court has already ruled that whatever the bot says, you are on the hook for.
Fluent, friendly, and wrong often enough to lose the customer
On tau-bench, a benchmark that puts AI agents through realistic airline and retail support scenarios with real tools and policies, the best model tested (GPT-4o) solved fewer than half of the tasks. Worse for a support queue: when the same common issue was sent eight times, the agent got all eight right less than a quarter of the time. It is not that the bot fails on obscure edge cases; it is unreliable on ordinary ones, at scale.
The failures are not random noise. A customer asks, in plain language, what your policy is, and the bot answers in plain language, confidently, whether or not it actually knows. When the honest answer is 'that is not our policy' or 'let me check,' the model tends to produce a fluent, specific, wrong answer instead: a refund window, a device limit, a discount that sounds exactly like a real policy because it is written in the same voice as one. The customer acts on it, and the gap only surfaces when they try to claim what the bot promised.
of realistic customer-support tasks were solved by the best AI agent tested (GPT-4o) on tau-bench, a benchmark of airline and retail support scenarios with real tools and policies.
Why support bots invent policies with total confidence
A large language model does not look up your policy. It predicts the next most plausible words, and a support answer has a very predictable shape: a specific number, a clear rule, a reassuring tone. So when a customer asks about a bereavement refund, a second login, or a discount, the model generates a plausible-sounding policy, a ninety-day window, one device per subscription, a percentage off, regardless of whether that rule exists anywhere in your documentation. A fabricated policy reads exactly like a real one, which is why customers and agents both take it at face value.
Retrieval helps but does not solve it. Even a bot wired to your help center will pull the wrong article, blend two policies, or overstate what a document actually says, and the model cannot reliably tell when it has done this. Two things make support uniquely exposed. First, the bot speaks for the company, so its invented answer is not an opinion, it is a commitment a customer can act on and, as Air Canada learned, hold you to. Second, the same question asked twice can return two different answers, so you cannot even QA it by testing it once.
When customers acted on what the bot said
Air Canada (2024) — the airline held liable for its bot
Air Canada's website chatbot told Jake Moffatt he could book a full-fare ticket and apply for a bereavement discount retroactively within ninety days. No such policy existed; the real policy excluded refunds once travel was complete. The British Columbia Civil Resolution Tribunal ordered Air Canada to pay him about 812 dollars and flatly rejected the airline's argument that the chatbot was a separate entity responsible for its own words. The company, it ruled, is responsible for all the information on its site, chatbot included.
Cursor (2025) — a fake policy that triggered cancellations
When users of the coding tool Cursor were unexpectedly logged out across devices, its AI support bot, signing emails as 'Sam,' told them this was a new policy: one device per subscription, by design. There was no such policy. The logouts were a session-security bug. The invented rule spread on Reddit and Hacker News and pushed customers to cancel before the cofounder confirmed 'we have no such policy' and committed to labeling AI support replies going forward.
NYC MyCity (2024) — a government bot telling businesses to break the law
New York City's official MyCity chatbot, built on Microsoft Azure AI to advise business owners, was found by The Markup to be giving illegal guidance: that employers could take workers' tips, that landlords could refuse tenants with housing vouchers, that stores need not accept cash. Each answer was confident, specific, and wrong under existing NYC law. Asked directly, the bot even claimed you could rely on it for professional business advice.
The numbers behind it
chance the same top agent resolved eight instances of one common support issue correctly (pass^8) on tau-bench's retail domain, roughly a 60 percent drop from its single-try score. Confident on any one reply, unreliable across a real queue.
of customers said they would prefer that companies did not use AI for customer service at all, and 53 percent said they would consider switching to a competitor if they learned a company was using it.
The pattern is identical every time: a fluent, specific answer, a policy or price that sounded real, and no check before it reached the customer. That is the gap TrueStandard closes. Before a support answer, macro, or help-center article goes out, paste it in and four to five models check every claim against each other in about a minute, so an invented refund window or device limit surfaces as a disagreement to resolve, not as a chargeback, a cancelled subscription, or a tribunal ruling later.
How to keep an AI support bot from inventing policy
Treat every bot answer that commits the company as a draft, not a decision. Before you let it reach customers unsupervised, put these in place.
-
01
Ground every answer in your real source of truth. The bot should answer only from your current help center, policy pages, and pricing, via retrieval, not from its own training. If it cannot find support for a claim, it should say so, not guess.
-
02
Forbid it from stating any policy, price, or refund it cannot cite. Constrain the bot to link or quote the specific article behind any commitment, and to escalate anything about money, eligibility, or exceptions to a human.
-
03
Test the same question many times, not once. Answers are non-deterministic; ask each important question repeatedly and across phrasings, and treat any variation in the policy it states as a defect.
-
04
Cross-check high-stakes answers before they send. For anything involving refunds, cancellations, legal, or safety, run the drafted answer past a second independent check and hold it if they disagree.
-
05
Keep your knowledge base current and dated. A bot grounded in stale docs will confidently quote a policy you already changed; review and timestamp the sources it reads.
-
06
Log every answer and audit for invented policy. Sample real transcripts weekly for claims that appear nowhere in your documentation, because that is where the next Air Canada is hiding.
-
07
Label AI replies and keep a human easy to reach. Customers act on what the bot says, so be clear when it is a bot and give a fast path to a person for anything it will not commit to.
How to make AI output reliable: check it across models
The fix is not to hunt for a single more accurate model. Every large language model predicts fluent, plausible text, so each one can be confidently wrong on its own. What changes the odds is agreement. When several independent models are asked the same thing and all land on the same answer, the chance they share the exact same hallucination drops sharply. When they disagree, you have found the precise claim to check by hand before it ships.
That is what TrueStandard does: it runs your draft through four to five frontier models at once and surfaces every disagreement in about a minute, with sources. See the AI fact checker for how the method works, or read why AI cites studies that do not exist for the mechanism behind the failures on this page.
Common questions
Can I use an AI chatbot for customer support at all?
Yes, for triage, drafting replies, and answering well-documented, low-stakes questions grounded in your help center. What it cannot do unsupervised is commit the company, quoting a refund window, a price, or an eligibility rule, because on realistic support benchmarks the best agents solve fewer than half of tasks, and in production they invent policies that sound exactly real. Keep a human on anything involving money or exceptions.
Why do support bots make up policies that do not exist?
Because they predict plausible-sounding text, not your actual rules. A policy answer has a predictable shape, a specific number, a firm rule, a confident tone, so when the model does not know, it generates something in that shape anyway. A ninety-day refund window or a one-device limit reads identically whether it is real or invented, which is why customers act on it.
Are we really liable for what the bot tells a customer?
A tribunal has already said yes. In Moffatt v. Air Canada, the British Columbia Civil Resolution Tribunal held the airline responsible for a bereavement-refund policy its chatbot invented, rejecting the argument that the bot was a separate legal entity. Treat anything your bot states as a commitment your company will be held to.
What is the safest way to run AI support?
Ground it strictly in your current documentation, forbid it from stating any policy or price it cannot cite, and verify high-stakes answers before they send. Running a drafted answer through several independent models and checking where they disagree catches the invented refund windows and device limits that any single bot, however fluent, will produce confidently.
Do not publish AI output on trust
Paste your draft. Four to five models check every claim in about 60 seconds, and you see exactly where they disagree before your name is on it.