How to Stop an AI Chatbot From Making Things Up (Hallucinations)

An AI chatbot that invents an answer is worse than no chatbot. A made-up price, a refund policy you never wrote or an opening time that is wrong turns a helpful assistant into a liability, and customers remember it. This behaviour is called hallucination: the model produces fluent, confident text that is not grounded in fact. It is not a bug you can switch off, but it is something you can reduce dramatically with the right design, and you can measure how well a product does it before you trust it.

This guide explains why hallucinations happen, the techniques that prevent them, how to test any chatbot for them in ten minutes, and how to keep your own assistant accurate over time. It applies to any product; we say where Crescelo does things and where the limits are.

Why chatbots make things up

A language model is trained to predict plausible text. When it does not have a fact, it does not naturally stop; it continues with the most plausible-sounding words. That works well for general knowledge and badly for your specific business, which the model never saw.

Notice that several causes are about your content and retrieval, not the model itself. That is good news: you can fix them.

The core defence: grounding

Grounding means every answer is built from passages of your own content, and the assistant is instructed to answer only from them. In practice, it works like this. The visitor's question is used to search your knowledge base. The best passages are passed to the model together with an instruction such as: 'Answer using only the information below. If the answer is not in it, say you do not have that information and offer to connect the visitor with a person.' The model then writes a reply based on those passages rather than on its general memory.

This approach is called retrieval-augmented generation, or RAG. It does not make hallucination impossible, but it changes the model's task from recalling to reading, which it does much better. The mechanics are described in what an AI website assistant is and hybrid search for RAG.

Seven techniques that reduce hallucinations

1. Retrieval that finds the right passage

If retrieval misses the passage that holds the answer, the model has nothing to read. Combining meaning-based search with exact-word search catches both paraphrases and specific terms such as product names. Splitting documents into sensible chunks also helps; see document chunking strategies.

2. A clear instruction to admit ignorance

The assistant needs explicit permission and a script for not knowing: 'I do not have that information. I can connect you with our team or take your details.' Without it, models tend to please. Test that the instruction holds.

3. Show and check sources

When you can see which passage an answer came from, you can verify it in seconds. Products that show sources make it easy to fix errors. Ask any vendor whether you can see the source behind an answer.

4. Confidence thresholds

If the retrieved passages are a weak match, a well-designed system declines to answer instead of guessing. It should offer a form or a person. This is one of the most effective safeguards.

5. Keep documents clean and current

Conflicting or outdated documents cause confident wrong answers. Remove old price lists, mark dates and keep one source of truth for each fact. This is the highest-value habit you can build.

6. Limit what the assistant may do

Decide what it must never say: legal, medical or financial advice, discounts, guarantees, delivery promises. Put those rules in its instructions and test them. Handover to a person is the safe fallback.

7. Review real conversations

No test list finds everything. Real visitors ask strange questions. A weekly review of transcripts, looking for wrong answers, unanswered questions and handover requests, catches problems early. See how to review your chatbot's conversations each week.

A ten-minute test you can run on any chatbot

Load the same documents into the product and ask these questions. Score each answer as correct, partly correct, wrong, or honestly unknown.

Run the test twice on different days. Answers that change wildly between runs signal weak grounding. You can try a grounded assistant with no account in the playground: upload a document and ask something the document does not cover.

Writing documents that a chatbot can use

• Use clear headings that match the questions people ask, such as 'How much does a colour treatment cost?'.
• Put the answer near the question, in full sentences.
• State facts with units and conditions: 'from $80, depending on length'.
• Date anything that changes, and remove old versions.
• Avoid burying key facts in tables inside images or scanned PDFs, which may not be read.
• Write policies plainly, including what happens in edge cases.
• Split large documents into topic pages.

Documents written for people to skim are usually good for retrieval too. See preparing your service menu and pricing documents.

What a good fallback looks like

The assistant will not always know. What it does next matters.

See handling obscure customer inquiries with fallbacks and live human handover flow design.

Guardrails for sensitive topics

In health, legal, financial and safety topics, a wrong answer can cause harm. Do not rely on the model's judgement. Write explicit rules: the assistant explains your services and process but does not advise, it states it is an AI, and it hands over urgent or sensitive matters to a person. Test those rules with realistic messages. See handling emergencies in automated chat and AI transparency and disclosure.

Measuring accuracy over time

• Correction rate: how many answers you had to fix each week; it should fall.
• Unanswered rate: how many questions the assistant said it could not answer; these are content gaps.
• Handover rate: how often visitors asked for a person, and why.
• Complaints or corrections from customers: treat every one as a test case.
• Spot checks: read twenty random conversations a month.

Crescelo lets you review conversations and correct answers, and it is designed to say when it does not know rather than guess. It cannot promise perfection, and neither can any product. A process of review is what keeps accuracy high.

Common mistakes

• Judging a chatbot only on questions it is expected to know.
• Leaving old documents in the knowledge base.
• Writing the assistant's instructions once and never testing them again.
• Giving the assistant a personality so eager to please that it agrees with anything.
• Ignoring unanswered questions.
• Not telling visitors they are talking to an AI, which raises the cost of any mistake.

Frequently asked questions

See how grounding feels in practice in the playground, or read how the AI assistant feature answers from your content. When you are ready, start a free trial.

Further reading

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al.): The original research paper behind retrieval-augmented generation.
• Google Search Central: creating helpful, people-first content: What Google looks for in useful pages.