Why AI Chatbots Make Things Up, and How to Stop Yours

AI chatbots make things up because a language model is built to produce a likely answer, and when the right answer isn't in front of it, a plausible one comes out instead. On a small-business website, almost every made-up answer I see traces back to one of four content problems: the answer is missing, the page is out of date, two pages disagree, or the question is vague enough that the bot grabs the wrong passage. Fix those, and most hallucinations stop.

I build ChatBento, an AI support chatbot for websites, so I spend a lot of time working out why an answer went wrong. For the legal side of a bot inventing a policy, see the Air Canada ruling post. This one is about causes and fixes.

What counts as a hallucination in customer support?

The US National Institute of Standards and Technology calls it "confabulation" in its Generative AI Profile (NIST AI 600-1, July 2024) and defines it as the production of "confidently stated but erroneous or false content". Hallucination is the everyday word for the same thing.

For a support bot, a working definition is simpler: any answer that states something your business never said. On real small-business sites that looks like:

• A plumber's bot saying "We offer free callouts" when the callout fee is $95.
• A bakery's bot saying "Yes, we have gluten-free options" when the site never mentions gluten.
• A salon's bot giving Sunday hours when the salon is closed on Sundays.
• A software company's bot saying "Yes, we integrate with Xero" because accounting tools usually do.
• A link to a page that doesn't exist, or a phone number that's slightly wrong.

There's also a quieter kind: a real fact attached to the wrong thing, like the price of your basic package when the visitor asked about the premium one. Every word came from your site, and the answer is still wrong.

What isn't a hallucination: "I don't have that information." That's the bot working correctly, and it never gets you into trouble.

Why do language models make things up at all?

A language model learns patterns from huge amounts of text. Ask it "Do you offer free quotes?" with nothing else to go on, and the pattern says lots of plumbers do, so "Yes, we offer free quotes!" is a likely-sounding answer. It isn't lying. It has no idea what your business does.

A September 2025 paper by researchers at OpenAI and Georgia Tech, Why Language Models Hallucinate, argues that training and evaluation reward this: models are scored like students on a test, where a confident guess earns more points than "I don't know". So a support bot needs two things on top of the model: your content, and rules that make "I don't know" an acceptable answer.

What does grounding mean, in plain words?

Grounding means the bot answers from documents you give it, not from what it picked up in training. The common technique is called retrieval-augmented generation, named in a 2020 research paper by Lewis and colleagues. Think of it as an open-book exam:

1. Your pages, Q&As and text are split into short passages and stored.
2. When a visitor asks a question, the system finds the passages most related to it.
3. The model gets the question, those passages, and a rule: answer only from these.
4. If the passages don't contain the answer, the model is meant to say so.

For a business owner, the point is that grounding moves most of the problem from the model to your content. The bot can only be as right as the passages it's handed, and it goes wrong in two places: step 2 finds the wrong passages, or the passages themselves are wrong. (More in the grounding glossary entry.)

ChatBento answers only from your website pages, Q&As and text. Its rules say never to invent prices, policies, dates, links, contact details or features, and when the answer isn't in your content it says so and offers a contact form. That removes most guessing. It doesn't fix a page that says the wrong thing. Only editing the page does.

The four causes on small-business sites

1. The answer is missing

A dog groomer's site lists breeds, prices and hours, but never says whether they groom cats. A poorly set-up bot fills the gap with "Yes, we groom cats and dogs!" A grounded bot says it doesn't have that information, which is safe but loses a quick answer.

Before: nothing on the site.

After: a short Q&A.

Q: Do you groom cats?

A: No. We only groom dogs. Cats aren't accepted, including for nail trims.

Write the "no" answers too. In my experience, missing answers are the most common cause, and the cheapest to fix.

2. The page is out of date

A yoga studio raised its drop-in price from $18 to $22. The pricing page was updated. A blog post from last spring, "New spring timetable: drop-in classes now $18", is still live, and the bot reads both.

Before: two prices on the site, one of them old.

After: the old post is deleted, removed from the bot's sources, or edited to say "Prices have changed since this post. See the pricing page." Better still, keep prices on one page and link to it instead of repeating the number.

Stale facts hide in blog posts, announcements and old promotions. When something changes, search your site for the old value, not just the page you think holds it.

3. Two current pages disagree

A café that does catering says "Free delivery on orders over $50" on its catering page, and "Free delivery within 5 miles" in its FAQ. Both are true: the real rule is over $50 and within 5 miles. The bot finds one or the other, or blends them into "free delivery on all orders", which isn't true at all.

Before: the rule split across two pages, each half-right.

After: one complete statement, used in one place:

Q: Do you charge for catering delivery?

A: Delivery is free for orders over $50 within 5 miles of the café. Beyond 5 miles, or under $50, delivery is $8.

Then the other page says "See delivery charges" and links to it. One source of truth per policy.

4. The question, or the page, is vague

A visitor types "how much is it?" on a site with three services. Or "do you do weekends?" on a site where the shop opens Saturdays but the repair service doesn't. The system finds a passage that looks related and the bot answers about the wrong thing.

Pages make this worse when the answer doesn't name its subject. A pricing section that says "From $95. Includes the first 30 minutes." under a heading "Pricing" loses its meaning once it's split into a passage: $95 for what?

Before:

Pricing

From $95. Includes the first 30 minutes.

After:

Emergency plumbing callout price

Our emergency callout fee is $95, which includes the first 30 minutes of work. Standard daytime visits are priced separately on the Services page.

Each answer should make sense on its own. For ambiguous questions, an instruction helps too. In ChatBento's Settings tab you can add your own Instructions, and a line like "If a question could apply to more than one service, ask which service they mean" handles "how much is it?" well.

How do you find hallucinations before customers do?

Ask the questions that expose each cause, and read the answers as a customer would:

Then keep checking after launch. Once a week for the first month, scan the conversations in your dashboard for answers with numbers, "free", "yes, we", links and phone numbers: the ones that cost money when they're wrong. More on that routine in how to train an AI chatbot on your website.

Fix the content, not the instructions

When a bot gets a fact wrong, the temptation is to add "Our callout fee is $95" to its instructions. It works until the fee changes and someone updates the website but not the instructions. Now you've built cause 3 yourself. A simple split:

• Facts go in your content: prices, hours, policies, service areas, what you do and don't do.
• Rules go in instructions: tone, when to ask a clarifying question, what to do with order problems. The dashboard's own example is a good model: "For order problems, ask for the order number first."

Instructions also can't override the safety rules. In ChatBento, your instructions apply unless they conflict with the built-in ones, so an instruction like "If unsure, give your best estimate" won't make the bot start guessing prices.

What if one slips through anyway?

Some will, especially early on. Decide now what you'll do:

1. Find the cause: the missing answer, the old page, the conflicting page.
2. Fix the content the same day, then ask the same question again.
3. If a customer relied on the wrong answer, the Air Canada case suggests honouring it is usually cheaper than arguing.
4. For anything that commits money, such as refunds, exceptions or custom quotes, have the bot pass the visitor to a person. I wrote the triggers and wording in when a chatbot should hand off to a human.

A checklist against made-up answers

• The bot answers only from your content and says when it doesn't know
• Your 20 most common questions have explicit answers, including the "no" ones
• Each price and policy lives on one page; other pages link to it
• Old posts and promotions with outdated facts are removed or updated
• Every answer names its subject and makes sense on its own
• Facts are in content, rules are in instructions
• You've run the four test questions above
• Someone reads conversations weekly for numbers, links and "yes, we"

Most of this is ordinary FAQ work, so it pays off twice: your visitors and your bot read the same pages. The FAQ automation guide shows how to turn repeat questions into answers a bot can use, and ChatBento's free plan lets you run the test questions above on your own site first.