A chatbot is not a machine you install and forget. It is more like a new team member: capable, fast, occasionally wrong, and much better after someone checks its work. The single habit that separates assistants that keep improving from those that quietly go stale is a short weekly review of real conversations. Fifteen minutes a week is enough for a small business. This guide gives you the routine: what to read, what to look for, how to score conversations, how to turn findings into fixes, and how to turn the same conversations into leads and content ideas.
Why weekly review matters
- Visitors ask things you did not expect. Only real conversations show you.
- Content goes out of date. Prices and policies change; the assistant does not know unless you tell it.
- Mistakes hide. A wrong answer given to one visitor is invisible until you look.
- Leads slip through. Interested visitors sometimes leave details that nobody follows up.
- Your website has gaps. Repeated questions show what your pages fail to explain.
A reviewed assistant becomes more accurate each week. An unreviewed one becomes less accurate as your business changes.
The 15-minute routine
| Minutes | Step | What to do |
|---|---|---|
| 0 to 3 | Scan the numbers | How many conversations, bookings, leads and handover requests this week |
| 3 to 8 | Read the flagged conversations | Unanswered questions, handovers, long conversations and any without a booking or lead |
| 8 to 11 | Spot check five random conversations | Read start to finish for tone and accuracy |
| 11 to 14 | Fix | Correct wrong content and add missing answers |
| 14 to 15 | Note | Write one line: what changed and what to watch next week |
Do it at the same time each week, ideally Monday morning, and put it in your calendar. In Crescelo, conversations are in Chat History and the numbers in Analytics.
What to look for
Wrong answers
Compare each answer with your documents. If it is wrong, find why: was the source outdated, did two documents conflict, or did the assistant ignore a correct passage? The cause tells you the fix.
Questions it could not answer
These are your content gaps. If the information should exist, write it down and add it. If it should not be shared (for example a private policy), make sure the assistant hands over politely.
Handover requests
Every time a visitor asks for a person, ask two things: was the request reasonable, and could the assistant have helped first? Repeated handovers on the same topic mean the assistant needs better content there.
Abandoned conversations
Where did visitors stop replying? A visitor who leaves after a price answer may be telling you the price was unexpected or unclear. See tracking in-chat funnel drop-off.
Leads and bookings that went nowhere
Check that each lead reached you and was followed up, and that each booking appears in your calendar and confirmations were sent.
Tone
Read a few replies aloud. If a human receptionist would not say it, adjust the instructions or add example replies.
A simple scoring sheet
| Conversation | Accuracy (0-2) | Helpfulness (0-2) | Tone (0-2) | Outcome | Action |
|---|---|---|---|---|---|
| Example: price question | 2 | 2 | 2 | Booked | None |
| Example: parking question | 0 | 0 | 1 | Left | Add parking information |
Score a sample of ten conversations for the first month, then five a week. Look at the totals over time. A rising average shows the review is working.
Turn findings into fixes
| Finding | Fix |
|---|---|
| Wrong price | Correct the price in the source document; remove old copies |
| Missing answer | Write a short document titled with the question |
| Answer too vague | Add specifics: figures, steps, conditions |
| Contradiction | Delete or reconcile the conflicting documents |
| Repeated handover on one topic | Add content or a clearer rule for that topic |
| Tone problem | Add three example replies in the right voice |
Retest the fixed question in the Testing Studio to confirm the change worked. See the questions every service business chatbot should answer well for a test list.
Use conversations to improve your website and marketing
- The most common questions belong on your pricing and FAQ pages.
- The words visitors use are the words to use in your copy.
- The objections they raise are what your pages should address.
- The pages where chats start show where visitors hesitate.
- Interest in a service you barely promote is a signal.
Many owners find the review teaches them more about their customers than analytics does. See unanswered chat questions as knowledge gaps and unanswered questions as lead opportunities.
Privacy in your review
- Limit who can read conversations to people who need to.
- Do not copy transcripts into unsecured places.
- Delete transcripts you no longer need, according to your retention policy.
- Do not use personal details from chats for marketing without a lawful basis and consent.
See the GDPR checklist for website chatbots. This is general information, not legal advice.
Team review versus solo review
In a team, share the load: one person reads, another fixes content, and a third checks bookings. Rotate the reader so different eyes catch different problems. Keep a shared log of changes so you know what was fixed and why. For a solo owner, keep the routine short and consistent rather than long and occasional.
Signals that need action right away
- A wrong price or policy stated to a customer: fix at once and consider contacting them.
- An emergency message that was not handed over: review your emergency rules today.
- A complaint that received a generic reply: follow up personally.
- A lead with no follow-up: reply now.
- A sudden rise in unanswered questions: a document may have been removed or a page changed.
A sample week, step by step
Here is what a Monday review might look like for a small clinic with 60 conversations last week.
| Finding | Detail | Fix |
|---|---|---|
| Three visitors asked about parking | The assistant said it did not know | Added a short parking page |
| One visitor was quoted last year's price for a check-up | An old price list was still uploaded | Deleted the old file and re-tested |
| Two visitors asked for a person about insurance | Reasonable; the assistant lacked detail | Wrote an insurance FAQ with the accepted providers |
| A lead left a phone number at 9 p.m. | Nobody had called back | Called the visitor first thing; set a morning routine for overnight leads |
| One reply sounded curt | Tone of the refund answer | Added three example replies in a friendlier voice |
The whole review took fifteen minutes and made five improvements. Do this consistently and the assistant becomes better at the things your customers actually ask, rather than the things you guessed they would.
Metrics worth tracking across weeks
| Metric | Direction you want |
|---|---|
| Conversations | Growing if traffic grows |
| Conversations that reach a booking or lead | Rising |
| Unanswered questions | Falling as you add content |
| Handover requests | Stable or falling, with the right ones staying |
| Wrong answers found in review | Falling |
| Time to fix a wrong answer | Under a day |
Chart them on a simple sheet. Trends matter more than any single week.
What not to do in a review
- Do not read every conversation when volume is high; sample.
- Do not blame the model before checking the content.
- Do not change many things at once, or you will not know what worked.
- Do not forget to retest after a fix.
- Do not skip a week because things seemed fine; that is when drift starts.
Setting up the habit
The routine only works if it survives busy weeks. Put a recurring fifteen-minute block in your calendar with a title that says exactly what to do, such as 'Read chatbot chats, fix three things'. Keep a running note of fixes so you can see the trend. If you miss a week, do a short version rather than skipping: read only the unanswered questions and handovers. Ask a colleague to cover when you are away, so the habit does not depend on one person, and celebrate small wins, such as the week the number of unanswered questions halves.
Frequently asked questions
How often should I review chatbot conversations?
Weekly for the first month at least, then weekly or fortnightly. Fifteen minutes is enough for most small businesses.
How many conversations should I read?
Read all flagged ones (unanswered, handovers, abandoned) and spot check five others. For high volume, sample.
What is the most important thing to fix first?
Wrong prices and policies, because they cost you money and trust.
How do I find what the assistant could not answer?
Look for replies where it said it did not know, and for handover requests. These are your content gaps.
Do I need special tools?
Not to start. Chat History and a simple spreadsheet are enough.
Should I tell customers I review conversations?
Yes, in your privacy notice: say that conversations are stored and used to answer and improve the service.
Start your routine by testing in the playground, see how the AI assistant works, and start a free trial when you want it on your site.
Further reading
- Google Search Central: creating helpful, people-first content: What Google looks for in useful pages.
ANSWER EVERY VISITOR. BOOK THE ONES WHO ARE READY.
Deploy Crescelo on your website in 15 minutes
Train your assistant on your price lists, documents, and service policies. Enable 24/7 grounded answers and conflict-free in-chat appointment booking.