Context window
A context window is the maximum amount of text, measured in tokens, that a language model can take into account at once. It covers the instructions, the conversation so far, any retrieved documents and the reply being written. Anything outside the window is invisible to the model when it generates its answer.
Example
A model with a 200,000-token window can read roughly a long novel's worth of text in one request; a long chat plus several documents can still fill it.
Why it matters for a website chatbot
A website's full content rarely fits in one request, and sending everything would be slow and costly. That's why chatbots retrieve only relevant passages. ChatBento sends the model the passages that best match each question plus the most recent turns of the conversation, not your whole site.
See it in practice: Docs and help-center search
Related terms
- TokenA token is the unit of text a language model reads and writes: a whole word, part of a word, a number or a punctuation mark. In English, one token averages about three-quarters of a word. AI providers measure context windows, speed and usage-based prices in tokens.
- Large language model (LLM)A large language model (LLM) is an AI model trained on very large amounts of text to predict and generate language. It can answer questions, summarize, translate and write. Its general knowledge comes from training data with a cutoff date, so it needs supplied documents to answer accurately about a specific business.
- Retrieval-augmented generation (RAG)Retrieval-augmented generation (RAG) is a technique in which an AI system first retrieves relevant passages from a chosen set of documents, then gives them to a language model to write its answer. It lets the model answer from current, specific information it was never trained on, and makes answers easier to check.
- ChunkingChunking is splitting long documents into smaller passages, or chunks, before creating embeddings for retrieval. Chunks need to be small enough to match specific questions and fit in a model's context window, but large enough to keep their meaning. Many systems overlap neighboring chunks slightly so sentences aren't cut off.