When a model is not doing what you want, there are three levers people pull: better prompting, retrieval, or fine-tuning. They are wildly different in cost and effort, and teams reach for the expensive one far too early. Here is how to pick the right lever without burning your budget on the wrong one.
Prompting: start here, always
Ninety percent of "the model is not good enough" turns out to be "the prompt was not good enough." Prompting costs nothing, changes in seconds, and fixes more problems than people expect. Before anything else, write a clear brief, add the right context, show an example. If that gets you there, and it usually does, stop. You are done.
RAG: when the model needs knowledge it does not have
If the problem is that the model does not know your documents, your data, your current facts, that is a retrieval problem, not a training problem. Give it the right information at question time. RAG is the answer when the gap is knowledge, and it is far cheaper and more flexible than baking that knowledge into the model.
Prompting changes what you ask. RAG changes what it knows. Fine-tuning changes how it behaves. Diagnose which one is actually broken before you spend a cent.
Fine-tuning: when you need consistent behavior, not knowledge
Fine-tuning is for style and behavior, not facts. Use it when you need the model to reliably respond in a specific format, tone, or pattern that prompting cannot hold consistently at scale. It is the most expensive option in time, data, and maintenance, and it goes stale as base models improve. It is a real tool for a narrow job, and a common mistake for a broad one.
The decision, in one line each
- Wrong output but the info is there? Prompt better.
- Missing knowledge or private data? RAG.
- Need a consistent behavior at scale that prompts cannot hold? Fine-tune, reluctantly.
Work the ladder from cheap to expensive, not the other way around. Most teams that fine-tuned would have gotten there with a better prompt and a little retrieval, for a fraction of the cost and none of the maintenance.