Document Chunking for RAG: Strategies for PDFs, Word Files and Web Pages

Before a chatbot can answer from your documents, the documents have to be cut into pieces it can search. That step, called chunking, quietly decides how good the answers will be. Cut too small and each piece loses its context. Cut too large and the search returns a lot of irrelevant text alongside the useful part. Cut in the wrong place and a price ends up in one chunk and the service it belongs to in another. This guide explains chunking in practical terms: what it is, the main strategies, how to handle PDFs, Word files, web pages and tables, how to choose sizes and overlap, and how to test that yours works.

It is written for people building or tuning retrieval-augmented generation (RAG) systems, and for business owners who want to understand why the way they write documents affects the answers they get. We describe general practice and say where Crescelo's approach fits.

What chunking is and why it matters

A RAG system does not read your whole document for each question. It searches an index of small pieces, retrieves the few most relevant, and gives them to a language model to write the answer. Each piece is a chunk. Search quality depends on chunks being self-contained and focused enough to match a question, but large enough to carry meaning.

Main chunking strategies

1. Fixed-size chunks with overlap

Split text every N characters or tokens, with a small overlap between neighbours. It is simple and works acceptably for uniform text. Its weakness is that it ignores structure and may cut through a sentence or a table.

2. Sentence or paragraph-based chunks

Split on natural boundaries and group sentences until a size limit is reached. It preserves meaning better than fixed splits and is a good default for prose.

3. Structure-aware chunks

Use the document's own structure, its headings, sections, lists and tables, to decide chunk boundaries. A section titled 'Cancellation policy' becomes a chunk with that title attached. This is usually the best method for business documents because the structure reflects how the author organised the information.

4. Semantic chunking

Group sentences by similarity of meaning, starting a new chunk when the topic shifts. It can help with long, unstructured text but is more complex and can produce uneven sizes.

5. Parent and child chunks

Index small chunks for precise matching, but return the larger parent section for context. It combines precision with completeness and suits long documents.

Choosing a chunk size

There is no universal best size, but useful ranges exist. For question answering over business documents, chunks of roughly 200 to 500 words (about 300 to 700 tokens) with 10 to 15 percent overlap are a common starting point. FAQ entries and price lines should be kept whole, even if short. Long policies should be split at headings. Test sizes with real questions rather than trusting a number.

1. Assemble twenty real questions and the passages that answer them.
2. Try two or three chunk sizes.
3. For each question, check whether the right passage appears in the top three or five results.
4. Choose the size where the correct passage appears most often and stays coherent.

Overlap: when and how much

Overlap repeats a little text between adjacent chunks so that an idea spanning a boundary is not lost. Ten to fifteen percent is typical. Too much overlap wastes storage and can return near-duplicate chunks; none at all risks losing context at boundaries. Structure-aware chunking needs less overlap because the boundaries follow the meaning.

Handling different file types

PDFs

PDFs are display formats, not structure formats. Text may be in columns, headers and footers may repeat on every page, and scanned PDFs are images with no text at all. Extract the text carefully, remove repeated headers and footers, rejoin hyphenated words split across lines and use text recognition (OCR) for scanned pages. See OCR text extraction for RAG. Prefer PDFs exported from a text document over scans.

Word documents

Word files carry real structure: headings, lists and tables. Use the heading styles to define sections and keep each section's title with its text. This gives better chunks than treating the file as plain text.

Verkkosivut

Web pages include navigation, footers and cookie banners that are not content. Extract the main content, keep the page title and headings, and skip boilerplate. Crawling a sitemap lets you cover a whole site; see dynamic web scraping and sitemap ingestion.

Spreadsheets and CSV files

Tabular data does not chunk like prose. Turn each row into a short sentence or record that includes its column names, or group rows meaningfully. See tabular data ingestion for RAG.

Video and audio

Transcripts are long and unstructured. Split by time segments or topic shifts and keep timestamps for reference. See YouTube transcript ingestion.

Tables and lists

A table split across chunks is nearly useless: the rows lose their headers. Keep tables whole where possible, repeat header rows in each chunk if a table must be split, or convert rows to sentences such as 'Colour treatment, medium length: $95, 2 hours'. Keep lists together with the sentence that introduces them.

Add context to every chunk

A chunk that says 'the fee is $50' is ambiguous. Prepend the document title and section heading to each chunk, for example 'Pricing > Consultations: The fee is $50.' That small addition makes matching and answering much more reliable. It costs little and helps both keyword and meaning-based search.

Metadata

Store useful metadata with each chunk: the source file, page or section, the date and a category. It lets you show sources, filter by topic and remove outdated material. When a document changes, you can replace exactly the chunks that came from it.

How chunking interacts with search

Chunking and retrieval are partners. Meaning-based (vector) search finds paraphrases; keyword search finds exact terms. Combining them, called hybrid search, and merging the results improves recall. See hybrid retrieval with dense vectors and sparse keywords and reciprocal rank fusion. Good chunks make both work better.

Writing documents so they chunk well

• Use clear headings that state what follows, for example 'Cancellation policy'.
• Keep one topic per section.
• Put the question and its answer close together.
• Use full sentences for key facts.
• Avoid burying facts in images or complex layouts.
• Keep documents short and focused rather than one giant file.

These habits help human readers too, and they are the easiest way for a business owner to improve answers without touching any technology. See preparing your service menu and pricing documents.

Common chunking mistakes

• Using one fixed size for every kind of document.
• Ignoring headings and structure.
• Cutting tables in half.
• Leaving headers, footers and navigation text in the chunks.
• Not testing with real questions.
• Failing to re-chunk after documents change.
• Skipping OCR on scanned files and wondering why nothing is found.

How to test your chunking

Keep the questions and rerun the test whenever you change your chunking or add documents.

A worked example: chunking a price list

Imagine a salon price document with headings for Cuts, Colour and Treatments, and a table under each. A fixed-size splitter might cut the Colour table in half, leaving the row 'Full head, medium length: $95' in a chunk with no heading. A structure-aware approach keeps each table with its heading and prepends the document and section titles, so the chunk reads 'Price list > Colour: Full head, medium length: $95, about 2 hours.' A question such as 'how much for a full head of colour' now matches both by meaning and by the exact words 'colour' and 'full head'. The extra thought at chunking time is what makes the answer reliable later.

Re-chunking when documents change

Documents change: prices, policies, opening hours. Treat the chunks as derived data that you can rebuild. When a file is updated, remove the old chunks for that file and add the new ones, and check that nothing from the old version remains. Keep a record of when each source was last indexed, and review anything that has not changed in a long time, since old content is a common source of wrong answers.

Frequently asked questions

You can see how a grounded assistant uses your documents in the playground, and read how the knowledge base feature handles files, pages and text.

Further reading

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al.): The original research paper behind retrieval-augmented generation.
• Google Search Central: creating helpful, people-first content: What Google looks for in useful pages.