When a chatbot answers from your documents, the quality of the answer depends first on whether the right passage was found. That is the job of retrieval. Two families of search dominate: vector search, which matches meaning, and keyword search, which matches exact words. Each fails in ways the other does not. Hybrid search combines them, and for business content full of names, numbers and product terms, it is usually the most reliable approach. This guide explains how each method works, where each fails, how to merge their results, how to tune the mix and how to test retrieval on your own documents.
The article is aimed at developers and technically curious owners. We describe general practice and note how Crescelo approaches it.
Dense vector search: matching meaning
Dense retrieval converts text into a list of numbers, called an embedding, that captures meaning. Texts with similar meaning have similar embeddings. To search, the system embeds the question and finds the chunks whose embeddings are closest. Because it works on meaning, it can match different wording: 'how much does a haircut cost' finds a passage titled 'Cut prices'.
- Strengths: handles paraphrase, synonyms and natural-language questions; works across languages if the model supports them.
- Weaknesses: can miss exact identifiers, rare terms, product codes and numbers; can return passages that feel related but do not answer; quality depends on the embedding model.
Sparse keyword search: matching words
Sparse retrieval matches the actual words in the question against words in the documents, weighting rare words more heavily than common ones. The best-known method is BM25. It is fast, transparent and excellent at exact terms.
- Strengths: exact names, codes, numbers and rare terms; easy to explain why a result matched.
- Weaknesses: misses paraphrases and synonyms; sensitive to spelling and wording; treats 'cheap' and 'affordable' as unrelated.
Why each alone fails on business content
| Question | Vector only | Endast nyckelord |
|---|---|---|
| 'How much is a keratin treatment?' | Good if a passage discusses keratin pricing | Good if the word 'keratin' appears |
| 'Do you do a fade for curly hair?' | Finds related hair-cut passages | May miss if the page says 'taper' or 'skin fade' |
| 'Is item SKU-4471 in stock?' | May return similar-looking products | Finds the exact code |
| 'Can I cancel without a fee?' | Finds the cancellation policy despite different wording | Needs the exact words 'cancel' and 'fee' |
| 'What is the deposit for a two-hour colour?' | Finds deposit and colour passages | Finds passages with 'deposit' and 'colour' |
Real questions mix both kinds: some rely on meaning and some on exact terms. A retrieval system that uses both catches more of them.
How hybrid retrieval works
- The question is sent to both search methods.
- Each returns its own ranked list of chunks.
- A merging step combines the lists into one ranking.
- The top few chunks go to the language model as context.
The interesting part is the merge. The two methods produce scores on different scales, so you cannot simply add them. Two approaches are common.
Weighted score combination
Normalise the scores of each method to a common range and add them with weights, for example 0.6 for vectors and 0.4 for keywords. It is easy to understand but sensitive to how scores are normalised.
Reciprocal rank fusion (RRF)
Ignore raw scores and use only ranks. Each chunk gets a score of 1 divided by a constant plus its rank in each list, and the scores are added. A chunk that ranks well in both lists rises to the top; one that ranks well in only one still appears. RRF is robust, needs no tuning of score scales and works well in practice. See reciprocal rank fusion for RAG.
| Method | Pros | Cons |
|---|---|---|
| Weighted scores | Direct control over the balance | Needs score normalisation and tuning |
| Reciprocal rank fusion | Simple, robust, no score scaling | Ignores how much better one result is than the next |
Crescelo combines vector and keyword retrieval and merges them with reciprocal rank fusion, then can rerank the top results. Embeddings can be computed locally, which keeps your documents' text off third-party embedding services; see local embeddings on CPU.
Reranking: a second look at the shortlist
After merging, a reranker can re-score the top twenty or so candidates by reading the question and each passage together, which is more accurate than comparing embeddings. It costs more per query but only touches a small shortlist, so it is often worth it where accuracy matters. Reranking is optional; hybrid retrieval alone is a big step up.
Query preparation
Chat questions are often short or depend on earlier messages: 'and how much is it for kids?' is meaningless alone. Before searching, systems can rewrite a follow-up into a standalone question using the conversation history, and can expand a query with synonyms. See multi-turn query condensation for RAG and domain synonym clusters. These steps improve both vector and keyword retrieval.
Chunking and metadata still matter
Hybrid search cannot rescue badly cut chunks. Make sure chunks are self-contained, carry their titles and keep tables whole; see document chunking strategies. Store metadata such as source and date so you can show sources, filter and update.
Tuning the balance
Start with an equal mix and test. Some content leans toward one method.
| Content type | Lean toward |
|---|---|
| Product catalogues with SKUs and codes | More keyword weight |
| Conversational FAQs and policies | More vector weight |
| Legal or technical text with defined terms | Balanced, with keyword strong on terms |
| Multilingual content | More vector weight, with a multilingual embedding model |
| Short factual pages such as price lists | Balanced |
How to test retrieval on your own data
- Collect 30 to 50 real questions, and for each note which passage should answer it.
- Run each question through vector-only, keyword-only and hybrid retrieval.
- For each, record whether the right passage is in the top 3 and top 5 results.
- Compute the hit rate for each method and compare.
- Look at the misses: is the problem chunking, wording, or a missing document?
- Repeat after any change to chunking, documents or settings.
| Metric | Meaning |
|---|---|
| Hit rate at k (recall at k) | How often the right passage appears in the top k results |
| Mean reciprocal rank | How high the right passage typically ranks |
| Precision at k | How many of the top k results are relevant |
You do not need a large test set. Thirty carefully chosen questions reveal most problems. See benchmarking RAG retrieval metrics for business.
What this means if you are not a developer
If you run a business and use a chatbot product, you do not need to tune any of this. What you can do is ask the vendor whether it combines meaning-based and keyword search, whether it can show which passage produced an answer and how it behaves when nothing relevant is found. And you can help retrieval by writing clear documents with headings, exact product and service names and the plain words customers use. The guide to stopping chatbots from making things up explains what happens after retrieval.
Common mistakes
- Choosing vector-only search because it sounds modern, then missing exact codes.
- Choosing keyword-only search and missing paraphrases.
- Adding weighted scores without normalising them.
- Ignoring chunking quality.
- Not testing with real questions.
- Sending too many chunks to the model, which dilutes the answer.
A worked example of merging
Suppose a visitor asks: 'Do you take walk-ins for a skin fade on Saturdays?' Vector search returns these chunks in order: A (walk-in policy), B (services list mentioning fades), C (Saturday opening hours). Keyword search returns B (contains 'skin fade' and 'Saturday'), A (contains 'walk-ins'), D (an unrelated blog post about fades). Reciprocal rank fusion with a constant of 60 gives each chunk 1 divided by (60 + rank) from each list.
| Chunk | Vector rank | Keyword rank | Score | Final position |
|---|---|---|---|---|
| A (walk-in policy) | 1 | 2 | 1/61 + 1/62 = 0.0325 | 1 |
| B (services list) | 2 | 1 | 1/62 + 1/61 = 0.0325 | 1 (tie, broken by rank) |
| C (Saturday hours) | 3 | not returned | 1/63 = 0.0159 | 3 |
| D (fade blog post) | not returned | 3 | 1/63 = 0.0159 | 4 |
Chunks A and B, which both methods found, rise to the top, and C, a useful passage that only vector search found, still makes the shortlist. The unrelated post D drops to the bottom. Neither method alone would have ranked the top three so well.
Latency and cost of hybrid retrieval
Running two searches sounds slower, but in practice both are fast for the size of a small business knowledge base, often a few milliseconds each for thousands of chunks. The larger costs come from embedding the question and from any reranking step, so cache repeated queries and embeddings where you can. Watch the total time from question to first word of the reply, because visitors notice delays of more than a couple of seconds. Measure before optimising: most of the delay in a chat reply usually comes from the language model, not the search.
When keyword search saves the day
Businesses are full of terms that mean nothing to a meaning-based model: a treatment named after a brand, a service code, a postcode, a staff member's name. Keyword search finds these exactly. If your documents use unusual names, make sure the retrieval you rely on includes keyword matching, and add the plain-language words customers use alongside the internal name so both routes work.
Frequently asked questions
What is hybrid search in RAG?
It combines vector search, which matches meaning, with keyword search, which matches exact words, and merges the results so the best passages from both are used.
Is hybrid search better than vector search alone?
Usually for business content, because it handles both paraphrased questions and exact terms such as names, numbers and codes.
What is reciprocal rank fusion?
A simple method for merging ranked lists that uses ranks rather than scores. Chunks that rank well in more than one list rise to the top.
Do I need a reranker?
It is optional. It improves accuracy by re-scoring a shortlist, at extra cost per query.
How do I know if retrieval is good?
Test with real questions and check whether the correct passage appears in the top few results.
Does hybrid search reduce hallucinations?
It helps by finding the right passage more often, so the model has the facts to read. It should be combined with clear instructions to admit when it does not know.
You can see grounded answers on your own documents in the playground and read about the knowledge base feature.
Further reading
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al.): The original research paper behind retrieval-augmented generation.
- Google Search Central: creating helpful, people-first content: What Google looks for in useful pages.
SVARA ALLA BESÖKARE. BOKA DEM SOM ÄR REDO.
Lägg in Crescelo på din webbplats på 15 minuter
Träna din assistent på dina prislistor, dokument och tjänstevillkor. Aktivera välgrundade svar dygnet runt och bokning av tider direkt i chatten utan krockar.