Fine Tuning

What Fine Tuning means

Fine tuning is adjusting an AI model by training it further on a set of examples, so that it writes, formats or classifies in a consistent way afterwards. Nothing is looked up at answer time. Whatever the model has learned is now part of the model itself.

The RAG and fine-tuning comparison offers a picture that holds up. Fine-tuning is an apprenticeship: after months of examples, the trainee writes and responds in the house style without opening a manual. The alternative, retrieval-augmented generation, is an open-book exam. The candidate has not memorised the textbook but is allowed to look things up, and can point to the page.

What it changes, and what it does not

The comparison puts the distinction in one sentence. RAG changes what the model can see. Fine-tuning changes what the model is. That is why the two are not two versions of the same thing, and why choosing between them as if they were is, in the comparison’s words, the most common mistake in this area.

A large language model knows what it learned in training and nothing else. It has never seen a business’s price list, contracts or staff handbook. Fine-tuning does not fix that gap well. It shapes how the model behaves: its tone, its format, the way it sorts items into categories. It is a good way to shape behaviour and a poor way to load facts.

Behaviour problem or Knowledge problem?

The question that settles most cases is whether the gap is something the model does not know, or something it does not do the way you need.

A knowledge problem is staff asking about policies, contracts or product details. The model needs to see the documents, which is RAG. A behaviour problem is every customer reply needing an exact structure, or thousands of support tickets needing the same categories every time. That is a change in behaviour, and it is where fine-tuning starts to earn its place. The comparison says most businesses have the first kind.

Where it is Strong

Fine-tuning is strong where the job is narrow, repetitive and needs identical behaviour every time: a fixed output format, a consistent tone, a classification applied the same way across a large volume of cases. The signs are volume and sameness. There are many similar items, there is one correct format, and there is a measurable definition of good.

The definition on this page says the same thing from the other side. It solves a behaviour problem and not a knowledge problem, and it earns its place when one narrow, repetitive task needs identical behaviour thousands of times.

Where it fails

The comparison lists the failures. It goes stale, because what the model learned is frozen until it is retrained. It depends on good examples, and preparing them is usually the slowest part. It can make a model worse at things it did well before. And it is hard to audit, because there is no passage to point to, only a behaviour.

It also does not stop a model making things up. The comparison cites a 2023 study that compared the two methods for teaching a model new information and found that RAG consistently outperformed fine-tuning, for knowledge the model had met in training and for entirely new knowledge. A fine-tuned model can still produce a confident wrong answer, which is the problem described under hallucination. The comparison says RAG reduces it by tying answers to a checkable source, and does not remove it, because retrieval can pull the wrong document.

Why it is rarely the first fix

The usual order is clearer instructions first, then RAG, then fine-tuning only for a narrow, repetitive task that RAG cannot reach. The comparison explains why. Many problems that look like a need for fine-tuning are a need for a clearer request, a worked example in the prompt, or saved custom instructions that set the tone and format. That is quick to test and settles a surprising share of cases.

The next step is RAG, if the answers need a business’s own material. Fine-tuning comes third, and only if the instructions and RAG still miss the task and there are clean examples the business is allowed to use. The comparison’s summary is that it is rarely the first tool and almost never the answer to “make the AI know our business”.

What it means for Data Protection

The sharpest difference between the two routes is what happens to data. With RAG, documents stay in a store the business manages, and removing one is a direct action. With fine-tuning, the data used for training is absorbed into the model. There is no single step that takes one person’s information back out, and the comparison notes that a model can reproduce fragments of its own training data.

The comparison reports the EU data protection board’s opinion of December 2024: whether a model built on personal data counts as anonymous has to be assessed case by case, and the bar for showing it is high. In practice that means personal data should not go into fine-tuning without a clear legal basis and a data protection check first. The GDPR duties attach to the personal data however it is used. The EU AI Act scales its obligations to the risk of the system and not to the technique underneath, so a fine-tuned model and a RAG system face the same question about the use. This page is not legal advice.

Cost and Ownership

The comparison says to compare cost by shape and not by size. RAG is quicker to stand up and has running costs for storing, indexing and refreshing documents. Fine-tuning front-loads its cost into preparing examples and running the training, and repeats it whenever the behaviour has to change.

The hidden cost in both is the same and rarely budgeted: someone has to own the material. A tuned model nobody re-tests decays in exactly the way the article on what happens when nobody owns an AI tool describes. Because a fine-tuned model goes stale until retrained, the owner’s job includes deciding when the behaviour needs retraining. See AI ownership.

Using it with RAG

The two can be combined, and some mature systems do. The comparison describes a common pattern: fine-tune for the manner, such as tone, format or a narrow classification, and use RAG for the facts, so answers stay current and traceable. It is a sensible destination and a poor starting point, because each part adds something that has to be maintained. Start with RAG, see where the model’s behaviour still falls short, and fine-tune only that gap.

A worked example

This is an illustration, not a real client. A small firm receives a large volume of support messages every week and wants each one sorted into the same handful of categories before a person sees it. The categories are fixed, the volume is high and the definition of a correct sort is easy to measure.

The firm starts with clearer instructions and a few worked examples in the prompt, and tests the result against a sample. If that settles it, nothing more is needed. If the sorting still drifts, and the firm has a set of past messages it is allowed to use as examples, with personal data removed or a clear basis for using it, fine-tuning is worth testing on that one task. The firm keeps answers about its policies out of the tuned model entirely. Those come from RAG, so they can be updated by changing a document.

Mix-ups worth avoiding

Four confusions come up often enough to name. The first is treating fine-tuning as the way to teach a model about the business. It is a weak way to add facts, and the comparison’s cited study found RAG stronger. The second is treating it as a cure for made-up answers. It is not a reliable one.

The third is assuming training data can be taken back out. A document can be removed from a RAG store in one checkable step, and a fine-tuned model has no equivalent. The fourth is treating it as a one-off job. A tuned model is frozen until retrained, so it needs an owner and a review date like any other tool.

Where it goes wrong

One failure is starting here because it sounds more serious than writing a clearer instruction. Another is training on examples that were never checked, so the model learns the mistakes along with the style. A third is putting personal data into training with no legal basis and no data protection check, when it cannot be reliably removed later.

The last is training once and never testing again. A model fine-tuned on last year’s examples keeps writing last year’s way, and nobody notices until a customer does.

Where it appears in an Audit

Whether a process needs fine-tuning, RAG or neither is decided by what that process handles, who is accountable for it and what happens when it is wrong, not by the technique. An audit weighs the choice against the actual work and not in the abstract, and it usually finds that better instructions or a tool already connected to the business’s files settle the need first.

If you want that question answered for a specific process, a discovery call is the quickest way to find out whether it is a knowledge problem, a behaviour problem or neither.

Sources

• RAG vs fine-tuning for business: the behaviour and knowledge distinction, where each route is strong and fails, the order of instructions then RAG then fine-tuning, the 2023 study it cites, the data protection difference, the EU data protection board’s December 2024 opinion as it reports it, the cost shapes and combining the two.
• Retrieval-augmented generation: the alternative route.
• What happens when nobody owns the AI you have rolled out: why unowned material decays.
• How we audit: weighing the choice against the actual process.