Retrieval-augmented generation combines two steps: first, content relevant to a question is retrieved from your own knowledge base; then a language model writes an answer based on that content. This lets a model talk about documents, manuals or tickets it has never seen and back the answer with sources. The idea is simple. Quality is decided almost entirely in the first step, the retrieval.
Preparing documents
RAG starts with preparing the sources. PDFs, Word files and web pages must be turned into clean text without headers, footers, page numbers or duplicated navigation. Tables are a common weak spot: extracted as running text, rows and columns lose their relationship. It pays to read samples of the extracted text before optimising retrieval — many quality problems originate here.
Chunking sensibly
Documents are split into sections that are searched individually. Sections that are too large dilute matches; sections that are too small tear context apart. Splitting along the document's structure — headings, paragraphs, lists — works better than a fixed character count, with a small overlap. Each section should carry metadata: document title, chapter, date, tenant and permissions. Prefixing the section text with the document title often improves retrieval noticeably, because sections on their own frequently do not reveal what they are about.
Hybrid search
Pure vector search finds content with similar meaning but often misses exact terms: product names, article numbers, legal paragraphs, abbreviations. Classic full-text search is strong here but does not understand paraphrase. Hybrid search combines both and merges the result lists. For German text this matters especially, because compound words and technical terms are not always matched reliably by semantic search.
Reranking
Retrieval returns a list of candidates, usually more sections than should go into the model. A reranking step scores these candidates again specifically against the question and selects the best. It costs a little time but improves precision considerably, particularly in large knowledge bases with many similar documents. Fewer but more relevant sections lead to better answers than an overloaded context.
Enforcing permissions
In SaaS products this is the most critical point. A user may only receive answers from documents they could open themselves. This check must happen during retrieval, not afterwards in the prompt. Asking the model «not to use» confidential sections is not a security measure. If vectors live in PostgreSQL, row-level security enforces this boundary automatically; with a separate vector database, tenant and permission must be passed as filters on every query.
Answers with sources
The model should be instructed to answer only on the basis of the supplied sections and to say openly when the information is insufficient. Linking each statement to a source has two benefits: users can verify it, and errors become visible. An answer without a matching source is a warning sign that the model filled in from general knowledge. An honest «I can't find anything about this in your documents» builds more trust than a plausible invention.
Freshness and deletion
Knowledge bases change. When a document is updated or deleted, its sections and vectors must be updated or removed too — otherwise the system quotes outdated or deleted content. For data protection erasure requests this is mandatory. Processing that regenerates affected sections on every change, plus a regular reconciliation between source and index, prevents gradual drift.
Making quality measurable
Without measurement, RAG is optimised by feel. An evaluation set of typical questions and the documents a good answer should draw on allows two measurements: does retrieval find the right sections, and does the model give a correct, sourced answer? Every change to chunking, retrieval or prompt is tested against this set. Fifty well-chosen questions from real user queries are worth more than any generic metric.
Common mistakes
The most frequent problems in RAG projects are badly extracted text, rigid chunking by character count, pure vector search without a full-text component, permissions checked only in the prompt, and no measurement. None of these require exotic technology; all require care with the fundamentals.
Where RAG fits
For most enterprise applications RAG is the right way to connect a model with your own knowledge — cheaper and more current than fine-tuning, and with traceable sources. Innopulse builds RAG features on PostgreSQL with pgvector and hybrid search, with permissions at the database level and an evaluation set from the start.