Retrieval or fine-tuning: how to choose for your data
Most teams pick between retrieval and fine-tuning without writing down what each would change. A practical way to decide, and what to check first.
Most teams that ask whether to use retrieval or fine-tuning have not yet written down what each option would change. The question sounds technical, so it often goes to the engineers, and the answer comes back as a preference for whichever tool the team already knows. That is the wrong order. The choice turns on three things a leader can answer without a laboratory: how often the underlying material changes, whether answers must show where they came from, and whether the organisation can change the model at all. Settle those first, and the technical choice becomes much narrower.
How it works
In plain terms, retrieval means that when a question arrives, the system first looks up passages from a body of documents the organisation controls, then places those passages in front of the model so it can write an answer from them. The model itself does not change. Update the documents and the answers can change the same day. Fine-tuning works the other way. The organisation supplies many examples of questions and good answers, and the model's internal settings, its weights, are adjusted to fit them. The knowledge, or the style, becomes part of the model. Changing it means running the adjustment again.
That difference has a practical consequence that is easy to miss. Fine-tuning needs a model whose weights you are allowed to change, and it needs a training process that someone in the organisation can run and repeat. A model you only reach through a vendor's interface may not offer that. Stanford HAI's glossary defines an open-source model by its weights, source code and training data, or by an auditable account of that data. The definition is cut short in the source, so read the full text before relying on it. The point for a leader is simpler: check early whether the model you plan to use lets you change it, and on what terms.
Who it affects
Companies that provide or deploy generative AI in the European Union face specific duties. According to the European Commission's page on the AI Act, providers of generative AI must ensure that AI-generated content is identifiable, and certain AI-generated content must be clearly and visibly labelled. The Commission says the Act's transparency rules come into effect in August 2026. Whether those duties reach your particular system depends on what it does and where it is used. That is a question for your legal and compliance advisers, not for the engineering team alone.
Explainability is the second concern. The Commission notes that it is often not possible to find out why an AI system made a decision, which matters for anyone who must audit outputs. In our judgement, a retrieval design gives a practical audit trail that a fine-tuned model does not: you can record which passages were supplied with each question. That record shows what the model was given. It does not prove why the model wrote what it wrote, so treat it as evidence to examine, not a guarantee.
What leaders get wrong
- Treating fine-tuning as a way to load facts. It is a poor place to store material that changes every month. Each update means another training run, and the old facts may linger in the answers.
- Assuming retrieval solves accuracy. A system that fetches the wrong passage writes a confident answer from the wrong passage. Retrieval moves the quality problem to search and to the document library.
- Choosing before naming the owner. Someone must maintain the documents or the training data. If no one is named, the system drifts within a year.
- Skipping the rights question. Whether you may use particular documents, or change a particular model, is decided by contracts and licences, not by the technology.
What to do about it
Start with a short written test. Ask how often the material your answers depend on changes; if the answer is weekly or faster, retrieval is usually the better fit, because you update documents rather than the model. Ask whether a user must be shown the source of an answer; if so, retrieval gives you something to show. Ask whether you can change the model's weights and whether you can repeat the process; if not, fine-tuning is off the table regardless of the other answers. Where the answers point in different directions, the fine-tuning case needs to be stronger, because it carries the higher cost of revision.
This draft is not yet ready to publish. The fact sheet supplied for it contains no source on retrieval, vector search, grounding or model adaptation, so the technical explanations above are our judgement rather than sourced fact. Before this piece goes out, a researcher should obtain the published research papers on retrieval-augmented generation and model adaptation, check each technical claim against them, and add the figures they support. The article is also shorter than the length the format calls for, for the same reason.