The context window is the maximum amount of text a language model can process in a single request. It covers everything: the overarching instruction, the context supplied, the conversation so far, and the answer to be generated. It is measured in tokens, not characters or words.
What a token is
A token is the smallest unit into which a model splits text — usually a part of a word. Common words are often a single token; rare or compound words break into several. That has a practical consequence particularly noticeable in German: compound nouns and umlauts produce more tokens than comparable English text. The same content therefore takes more room in the context window in German and costs more.
Why a large window does not make RAG redundant
As context windows grow, it is tempting to simply supply all documents and skip retrieval. That rarely works well. First, every token supplied costs money and time, on every single request. Second, reliability falls as more irrelevant text is supplied — relevant passages get lost in the noise. Third, every window remains finite while document sets grow. Deliberate selection therefore stays the better architecture even with large windows.
Position in the window matters
Models do not process every position in the context equally reliably. Information at the beginning and the end tends to be taken up better than material in the middle of a long context. In practice that means the most important instructions and the most relevant passages belong at the edges, not in the middle of a long compilation. Anyone supplying a lot of context should order it rather than concatenate it.
Conversation history eats the window
In conversational applications the history grows with every turn and consumes increasing room. Without a countermeasure, a longer conversation reaches the limit, and then either the beginning drops out or the request fails. Common strategies are summarising older parts rather than carrying them in full, or re-supplying only the parts relevant to the current question.
Strategies for long documents
Where a document exceeds the window, three routes are common. Split it and select the relevant parts deliberately — the usual RAG approach. Process it section by section and combine the partial results, which works well for summaries. Or work in stages: narrow down roughly first, then work precisely within the narrowed area. Which route fits depends on whether a specific passage is being sought or the whole document must be covered.
Practical consequence
The context window is a resource with a price, not a container to be filled. Supplying less but better-selected material generally yields more reliable answers at lower cost than exhausting the window.