RAG or fine-tuning? An honest guide to deciding
Updated: 2026-09
RAG supplies a model with current, verifiable knowledge at run time. Fine-tuning shapes its behaviour and form. The question is rarely either-or — but sequence matters: RAG and careful prompting first, fine-tuning only if the form still does not hold.
Anyone aligning an AI application to their own content meets this question early. It is usually posed as a choice between two routes and is really a question about the problem: does the model lack knowledge, or does it lack the right form?
That distinction determines everything else. Knowledge belongs in a queryable source, because it changes. Behaviour belongs in the model, because it should stay stable. Mixing the two builds a system that is expensive to maintain and hard to correct.
Head to head
| Criterion | RAG | Fine-tuning |
|---|---|---|
| Currency of knowledge | Always current, because retrieved at run time | Frozen at the point of training |
| Traceability | A source can be named per statement | No origin of a statement is visible |
| Cost of change | Swapping a document suffices | Retraining required |
| Behaviour and format | Steered via the prompt, less stable | Anchored firmly in the model |
| Cost per request | Higher, because context is supplied | Lower, because shorter instructions suffice |
| Deletability of data | Individual content can be removed | Trained content cannot be selectively deleted |
When RAG wins
- —The model needs knowledge from your documents that changes.
- —Statements must be evidenced and traceable to a source.
- —Confidential or tenant-specific data is involved and must remain deletable.
When Fine-tuning wins
- —The model deviates from the desired format despite careful instructions.
- —A professional register must be hit consistently.
- —A narrowly bounded task needs high, unvarying reliability.
Our take
Our view: in most projects RAG is the right first step and fine-tuning the later, optional second. The reason is not that RAG is superior but that it is cheaper, faster to change and usually sufficient in substance — and that fine-tuning creates a lasting dependency that must be maintained.
Where both make sense, they complement each other cleanly: RAG supplies the knowledge, fine-tuning the form. What does not work is using fine-tuning to keep facts current — they are outdated at the next change.
Parent service: Digital Transformation
Matching offers
AI integration strategy
An AI integration strategy establishes where artificial intelligence creates genuine value in your organisation — and where it does not. It identifies use cases, assesses feasibility and benefit, prioritises, and delivers a plan that builds in AI Act conformity from the start rather than retrofitting it.
EU AI Act gap assessment
A gap assessment answers three questions: which AI systems do we use, how are they classified under the EU AI Act, and what is missing for conformity? The output is a prioritised action plan, with an eye on the 2 August 2026 enforcement window.
FAQ
Can we combine both?
Yes, and in demanding applications that is the norm: RAG for knowledge, fine-tuning for consistent form. The sensible order is RAG first, then assess whether fine-tuning is still needed.
What about confidential data?
It belongs in RAG, not in training. What has been trained into a model cannot be selectively deleted — with personal data that is a serious problem.
Is fine-tuning not cheaper to run?
Per request often yes, because shorter instructions suffice. Over the lifetime, maintenance, evaluation and rebuilding on model changes are added, which regularly consumes the advantage.