A general-purpose AI model knows a lot about the world but nothing about your company, your customers or your house style. Teams building AI features usually reach for one of three techniques to close that gap: prompting, retrieval-augmented generation (RAG) and fine-tuning. They solve different problems.
1. Prompting: tell the model what you want
Prompting means giving the model instructions, context and examples in the input. This includes system prompts that set behavior for every conversation, and few-shot examples that show the desired output format.
Prompting suits tone, format, rules, task instructions and small amounts of reference information.
It’s instant and cheap to change, and you don’t need any special infrastructure. OpenAI’s guide to optimizing accuracy calls prompt engineering “typically the best place to start.”
The catch is space. Everything has to fit in the context window, and long prompts add cost to every request.
2. RAG: give the model the right information
RAG retrieves relevant documents at question time and adds them to the prompt. The 2020 research paper that introduced the term highlighted a key advantage: the model’s knowledge can be updated by changing the documents, without retraining. We explain the mechanics in our RAG explainer.
RAG is the right tool for facts that change, large knowledge bases, answers that need citations, and access to documents that has to respect permissions.
Answers stay up to date and can be traced back to their sources, with no retraining.
But quality depends heavily on your search and on the documents themselves, and you need infrastructure to index and retrieve content.
3. Fine-tuning: change the model’s behavior
Fine-tuning continues training a model on your own examples, adjusting its internal weights. Providers offer fine-tuning for some hosted models, and open-weight models can be fine-tuned fully.
Fine-tuning makes sense for consistent specialized formats, a distinctive style, narrow classification tasks, or getting a smaller, cheaper model to do one specific task as well as a larger one.
Once it’s done, prompts get shorter, behavior gets more consistent and costs can drop at scale.
It does need good training examples, time and expertise. You’ll have to redo it when the base model changes, and it’s a poor way to teach facts that change.
Quick decision guide
| If you need… | Start with |
|---|---|
| A particular tone or format | Prompting |
| Answers from your documents | RAG |
| Up-to-date information | RAG |
| Citations to sources | RAG |
| Highly consistent output at huge volume | Fine-tuning, after prompting |
| A small, cheap model for a narrow task | Fine-tuning |
| To teach the model new facts | RAG, not fine-tuning |
A common mistake Teams often jump to fine-tuning to teach a model about their products. That usually disappoints: facts go stale and the model may still guess. Fine-tuning shapes behavior; RAG supplies knowledge.
Combining approaches
Mature systems usually layer all three: a well-crafted prompt defines behavior, RAG supplies current knowledge, and, where volume justifies it, a fine-tuned model delivers consistent formatting cheaply.
Measure before and after
Whichever path you choose, build an evaluation set: a few dozen realistic questions with good answers. Test every change against it. Without measurement, it is easy to spend weeks on techniques that feel sophisticated but do not improve results. OpenAI’s guide makes the same point, starting every project with “a good prompt with an evaluation set of questions and ground truth answers.”
Start simple, add complexity when needed
Start with prompting, add RAG when you need your own knowledge, and consider fine-tuning only when you have clear evidence it will improve consistency or cost. Most successful projects follow exactly that order.
Sources
- Optimizing LLM Accuracy, OpenAI API documentation
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv, 2020
