Fine-tuning vs. RAG? When to fine-tune a model vs. use documents

Fine-tuning is training an off-the-shelf model on your examples to change how it responds: style, format, vocabulary. RAG feeds your documents to the model at the time of the query, without changing the model itself. In short: fine-tuning teaches how to speak, RAG provides what to speak about.

What is the difference?

Knowledge that changes (price lists, terms of service, news): RAG. Document updates take effect immediately, without training.
Style and format (brand tone, fixed response template, narrow task): fine-tuning.
Sources: RAG can point out where it got the answer. A fine-tuned model cannot tell which example it learned from.
Cost: RAG requires a vector database and some extra tokens per query. Fine-tuning requires data preparation, training, and re-training with every major change.

How does our lab do it?

We chose RAG and did not fine-tune any model. The lab processes approx. 2k new texts weekly (week 12–18.10.2026: 2,197). A model fine-tuned on Monday would be outdated by Wednesday.
New text = new vector, not new training. The hourly process adds vectors for new texts and the chat sees them immediately.
Behavior is set by instructions and code, not training: response with a source, refusal for out-of-context queries, output guardrails. When something needs to change, we update the prompt and run tests.
Cost: 0 PLN for training, because there is no training.

When to use fine-tuning instead?

When a small, cheap model needs to perform one narrow task very well, e.g., classify tickets.
When responses must always follow the exact same format and instructions are not enough.
When you have hundreds of good examples and a stable task that does not change every week.
Often a combination works best: a model fine-tuned for format + RAG for knowledge.

Questions and answers

Does fine-tuning „upload” my documents into the model?

Not in a reliable way. The model learns patterns, and facts from training can be distorted. RAG is used for knowledge.

What is cheaper to start with?

Almost always RAG. It does not require training data preparation or training.

Can both be combined?

Yes. Fine-tuning sets the style and format, RAG provides up-to-date knowledge.

Sources

Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv:2005.11401
Hu et al.: LoRA: Low-Rank Adaptation of Large Language Models, arXiv:2106.09685
Lab case study (week 12–18.10.2026), chat vector pipeline (23.10.2026)

Fine-tuning vs RAG