RAG vs Fine-Tuning: Which One Your Project Needs
Clients often ask us to "train the model on our data". Nine times out of ten, what they need is retrieval, not training. Here is how to tell which is which.
- Published
The rule of thumb
Use RAG when the model needs to know things — facts, policies, documents. Use fine-tuning when the model needs to behave differently — a format, a style, a narrow classification task at high volume.
Side by side
| RAG | Fine-tuning | |
|---|---|---|
| Teaches | Knowledge, looked up at query time | Behaviour, baked into the model |
| Updating content | Re-index in minutes | Retrain and re-evaluate |
| Citations | Natural — sources are retrieved | Not available |
| Upfront effort | Ingestion pipeline + evals | Curated training set + evals |
| Best for | Q&A over documents, support | Consistent formats, classification, cost reduction at scale |
When you need both
High-volume systems sometimes fine-tune a smaller model to follow a specific output format or tone cheaply, and use RAG to feed it current facts. That combination is an optimisation, not a starting point: begin with RAG and a strong model, measure, then fine-tune if cost or latency demands it.
Common mistakes
- Fine-tuning to add facts — the model still guesses and cannot cite.
- Fine-tuning before prompts and retrieval are properly engineered.
- Skipping evals — without them, nobody knows if the fine-tune helped.
Frequently asked questions
Is fine-tuning expensive?
Training itself is often affordable; the real cost is building a high-quality training set and evaluating it.
Can we fine-tune on customer conversations?
Only with appropriate consent and after removing personal data.
What do you recommend for a first project?
RAG with a capable model and a proper eval set. Fine-tune later only if the numbers justify it.
