The greeting and the starter buttons of your chatbot decide whether visitors start a conversation at all. Small changes, more specific wording, a different first question or a rearranged button, can move the share of visitors who open the chat. But guessing what works is unreliable, and testing properly on a small website is harder than the tools suggest: traffic is low, results are noisy and it is easy to mistake luck for a win. This guide gives a practical method for small sites: what to test first, how to run a fair comparison with or without special software, how long to run it, what to measure, how to avoid common traps and how to keep a simple log so that your learning accumulates. It also explains what Crescelo does and does not offer for testing, so you plan around it.
What is worth testing
| Element | Example change | Likely effect |
|---|---|---|
| Greeting text | Generic 'How can I help?' versus a specific offer | Chat open and reply rate |
| First question or prompt | 'Ask about prices' versus 'Book a time' | Direction of the conversation |
| Starter buttons | Two options versus four | Reply rate |
| Timing | Bubble after 10 seconds versus 30 | Open rate and annoyance |
| Placement or label | Icon versus 'Ask or book' | Discoverability |
| Tone | Formal versus friendly | Reply rate and satisfaction |
Test the biggest lever first: the greeting's clarity about what the assistant can do. Small wording tweaks come later.
Decide what success means
- Chat open rate: chats opened divided by visitors who saw the launcher.
- Reply rate: visitors who send a message divided by those who opened.
- Booking or lead rate: conversions per 100 visitors.
- Quality: are the conversations useful?
The final measure is bookings and leads per visitor. A greeting that gets more opens but fewer bookings is not a winner.
Two ways to test
| Method | How | Pros | Cons |
|---|---|---|---|
| Split test | Show version A to half the visitors and B to the other half at the same time | Fair; controls for seasonality | Needs a tool or code that can split visitors |
| Before and after | Run version A for a period, then version B | Simple; needs no special tool | Affected by seasonality and other changes |
Crescelo does not include a built-in A/B testing tool, so on a small site the practical method is before and after, done carefully: change one thing, keep everything else stable and compare periods of equal length. If you have a developer or use a testing tool that can show different greetings to different visitors, a split test is better.
Run a fair before-and-after test
- Write your hypothesis: 'A specific greeting will raise the share of visitors who reply.'
- Record the baseline for two to four weeks: visitors, chat opens, replies and bookings.
- Change one thing only.
- Run for the same length of time, ideally covering the same days of the week.
- Compare the same metrics.
- Check that traffic sources and promotions were similar in both periods.
- Decide, record and move on to the next test.
- Avoid testing during a holiday or campaign that changes traffic.
- Do not change the site, offers or prices at the same time.
- Keep the assistant's content stable.
How much data is enough
On a small site, a few dozen chat opens tell you very little. Random variation can produce a difference of several percentage points with no real cause. A rough rule: do not decide on fewer than a couple of hundred visitors per version, and treat small differences with suspicion. If your traffic is very low, test big changes only, because only big changes are visible, and rely on judgement plus reading conversations to refine the rest.
| Traffic per version | What you can conclude |
|---|---|
| Under 100 visitors | Almost nothing; use as a hunch |
| 100 to 500 | Only large differences are meaningful |
| 500 to 2,000 | Moderate differences may be real |
| Over 2,000 | Smaller differences become visible |
Ideas for greetings and prompts
- Lead with a specific outcome: 'Ask about prices or book a time in one minute.'
- Ask a concrete question: 'Looking for a haircut, colour or a beard trim?'
- Offer options as buttons: 'See prices', 'Book a time', 'Ask a question'.
- Mention out-of-hours availability: 'The team is offline, but I can help now.'
- Match the page: a pricing-page greeting versus a home-page greeting.
See chatbot welcome message copywriting for thirty examples and micro-copy for quick replies.
Read the conversations too
Numbers tell you what happened; conversations tell you why. After each test, read a sample of chats that started from each version. Did a greeting attract questions that the assistant could not answer? Did one bring in more off-topic messages? A version that gets fewer but better conversations may be the real winner.
A simple test log
| Päivämäärä | Muuta | Hypothesis | Baseline | Tulos | Decision |
|---|---|---|---|---|---|
| 1 June | Greeting: specific offer | Raises reply rate | Reply rate 9% | Reply rate 13% | Keep |
| 1 July | Bubble delay 10s to 30s | Fewer dismissals | Dismissals 40% | Dismissals 31% | Keep |
| 1 August | Four buttons to two | Raises replies | 13% | 12% | No change; revert |
A log turns scattered experiments into knowledge. Review it each quarter.
Pitfalls
- Changing several things at once.
- Stopping the test when the first numbers look good.
- Ignoring seasonality.
- Testing tiny changes on tiny traffic.
- Optimising open rate while bookings fall.
- Forgetting to record what you learned.
Worked example: a salon tests two greetings
A salon's chat greeted visitors with 'Hi, how can I help you today?' Over four weeks, 1,200 visitors saw the launcher, 96 opened the chat and 41 sent a message. The owner changed the greeting to 'Ask about prices or book a time in one minute' and added two buttons, 'See prices' and 'Book a time'. Over the next four weeks, with similar traffic and no other changes, 1,180 visitors saw the launcher, 128 opened the chat and 74 sent a message, and bookings from chat rose from six to eleven. The sample is small, so the owner treated the result as encouraging, not proven, kept the new greeting and planned to check again after the school holidays. The habit that mattered was recording the baseline before changing anything.
| Version | Visitors | Chats opened | Replies | Bookings from chat |
|---|---|---|---|---|
| Original greeting | 1,200 | 96 | 41 | 6 |
| Specific greeting with buttons | 1,180 | 128 | 74 | 11 |
Frequently asked questions
Can I A/B test my chatbot greeting?
Yes, with a tool that splits visitors, or with a careful before-and-after comparison. Crescelo does not have a built-in split test.
How long should a test run?
Long enough to collect a few hundred visitors per version and to cover a full week or two of traffic patterns.
What should I measure?
Chat open rate, reply rate and bookings or leads per visitor, plus a read of the conversations.
What should I test first?
The greeting's clarity about what the assistant can do.
Is a small difference meaningful?
Rarely on a small site. Treat small differences as noise unless you have plenty of data.
How do I keep track of tests?
Keep a simple log with the change, hypothesis, baseline, result and decision.
Try different greetings in the playground, see the website widget feature and read how to review your chatbot's conversations.
Further reading
- Google Search Central: creating helpful, people-first content: What Google looks for in useful pages.
VASTAA JOKAISELLE KÄVIJÄLLE. VARAA AIKA NIILLE, JOTKA OVAT VALMIITA.
Ota Crescelo käyttöön verkkosivustollasi 15 minuutissa
Opeta avustajasi hinnastoillasi, asiakirjoillasi ja palveluehdoillasi. Ota käyttöön lähteisiin perustuvat vastaukset ympäri vuorokauden ja päällekkäisyydettömät ajanvaraukset suoraan chatissa.