Fine-Tuning vs Continued Pretraining vs RAG: A Decision Guide
Three ways to adapt a model answer different problems. Read the distinction, a decision test, and why RAG is usually first.
Teams often ask which adaptation method is best, as if the three were competing products. They solve different problems: retrieval gives the model information, fine-tuning teaches it a behaviour or format, and continued pretraining teaches it a domain. Choosing the wrong one produces an expensive system that still fails.
Direct answer: Use RAG when the answer depends on information that can be retrieved and changes; use fine-tuning when the task is stable but the model's behaviour or output format needs teaching; use continued pretraining when the domain's language and concepts are unfamiliar. The RAG article, the LoRA article, and the domain-adaptation article cover the mechanisms. The boundary: many real systems need more than one — retrieval for facts, fine-tuning for behaviour — and the choice is about the dominant failure, not a single label.

What each method changes
RAG changes what the model reads at answer time. It can use current, verifiable information and cite sources, but it does not change the model's behaviour or reasoning, as the retrieval article explains.
Fine-tuning changes what the model does — its output format, tone, and task behaviour. It does not add knowledge reliably, which the distillation article's caveats about transferred behaviour illustrate.
Continued pretraining changes what the model knows about a domain by exposing it to more of the domain's text. It is the most expensive and the least frequently justified, as Continued Pretraining for Domain Adaptation explains.
Three questions separate them:
- Is the problem missing information, wrong behaviour, or unfamiliar language?
- Does the information change often? Retrieval handles change better than weights.
- Can the desired behaviour be described by examples? Fine-tuning needs them.
A worked decision
The following is hypothetical and illustrative.
A legal team wants an assistant that answers questions about their contracts and produces summaries in a specific internal format. Three failures are observed:
- Answers cite the wrong contract version. This is an information problem — the correct document was not retrieved. RAG with better indexing and version control is the fix, not fine-tuning, as the knowledge-graphs article explains for structured sources.
- Summaries omit required sections. This is a behaviour problem — the model does not follow the format consistently. A fine-tuned model with examples of good summaries can help, or a structured output schema, as the structured-output article describes.
- The model misreads domain terms. If the terms are unfamiliar, continued pretraining on legal text may be justified — but only after retrieval and prompting have been tried, as Continued Pretraining for Domain Adaptation argues.
Three conclusions follow:
- The three failures need three different responses.
- RAG is first because the most serious failure — wrong version — is an information problem, and retrieval is where provenance lives, as the data-governance article explains.
- Fine-tuning is second for the format, because that behaviour is stable and teachable by example.
A single method would fix one and leave the others.
A decision sequence
- Classify the failure — information, behaviour, or domain knowledge.
- Try prompting and retrieval first, because they are cheapest and most reversible, as the prompt-engineering article argues.
- Add fine-tuning when behaviour must be consistent across many cases and examples exist.
- Consider continued pretraining only when language itself is the barrier and a substantial corpus exists.
- Evaluate the full system after each addition, since methods interact, as the LLM evaluation article explains.
- Record why the method was chosen, so a future team can revisit the decision, which the decision-memo article covers.
A well-designed system often combines retrieval for facts with a small behavioural adaptation.
Combining methods deliberately
Once the dominant failure is identified, combinations follow a pattern:
Retrieval plus prompting is the default starting point for most knowledge tasks. It is reversible, updatable, and makes provenance visible, as the retrieval article explains. Many teams never need more.
Retrieval plus fine-tuning is common when the answer depends on changing information and the output must follow a stable format. The retrieval supplies facts; the fine-tuning teaches the format, which the structured-output article covers.
Continued pretraining plus retrieval suits domains where the language itself is unfamiliar but facts still change. Pretraining teaches the vocabulary; retrieval supplies current detail, as Continued Pretraining for Domain Adaptation describes.
These combinations are not stacked at once. Each addition should follow evidence that the previous stage left a specific failure, which the evaluation article explains how to measure.
What the guide cannot decide
It cannot rank methods universally. The right choice depends on your failure, data, and budget, which the cost-estimator article quantifies.
It cannot replace evaluation. Only a labelled set can show which failure dominates and whether a change helped, as the evaluation-rubric article explains.
It is not static. A system that starts with retrieval may need fine-tuning later when the task stabilises, which the monitoring article covers.
A decision test
- Name the dominant failure in observable terms.
- Check whether retrieval can fix it with better data or indexing.
- Check whether examples can teach the behaviour through fine-tuning.
- Check whether domain language is the barrier before considering continued pretraining.
- Start with the cheapest reversible option.
- Evaluate the combined system after each step.
- Revisit when the failure pattern changes.
For the retrieval method, see Retrieval-Augmented Generation; for the training methods, see LoRA and PEFT and Continued Pretraining.
References & Further Reading
- Geoffrey Hinton, Oriol Vinyals, and Jeff Dean, 2015. Distilling the Knowledge in a Neural Network — Soft-label distillation, the basis of most model compression work.
- Anthropic, 2026. Models overview - Claude Platform Docs — The current Claude model lineup (Opus 5, Fable 5.2, Sonnet 5.5, Haiku 5.5), supported inputs and the official model-selection guidance.
- Nils Reimers and Iryna Gurevych, 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks — The bi-encoder recipe behind modern semantic search and RAG retrieval.