Chunking
Also called Text chunking, Document chunking
Chunking is splitting long documents into smaller passages, or chunks, before creating embeddings for retrieval. Chunks need to be small enough to match specific questions and fit in a model's context window, but large enough to keep their meaning. Many systems overlap neighboring chunks slightly so sentences aren't cut off.
Example
A 5,000-word policy page becomes about 25 chunks, each a few paragraphs long, so a question about refunds retrieves only the refund section.
Why it matters for a website chatbot
Good chunking means the chatbot retrieves the paragraph that answers the question, not a whole page. ChatBento splits content on paragraph and sentence boundaries into chunks of about 1,100 characters, with a small overlap between neighbors.
See it in practice: Docs and help-center search
Related terms
- EmbeddingsEmbeddings are lists of numbers that represent the meaning of a piece of text, image or other data, produced by a machine-learning model. Texts with similar meanings get similar numbers, even when they share no words, which lets software compare meaning mathematically for search, recommendations and grouping.
- Retrieval-augmented generation (RAG)Retrieval-augmented generation (RAG) is a technique in which an AI system first retrieves relevant passages from a chosen set of documents, then gives them to a language model to write its answer. It lets the model answer from current, specific information it was never trained on, and makes answers easier to check.
- Vector searchVector search finds items whose embeddings are closest to the embedding of a query, so results match by meaning rather than by exact words. The embeddings are stored in a vector index or vector database built to find the nearest matches quickly, even across millions of items.
- Context windowA context window is the maximum amount of text, measured in tokens, that a language model can take into account at once. It covers the instructions, the conversation so far, any retrieved documents and the reply being written. Anything outside the window is invisible to the model when it generates its answer.