A general-purpose AI model knows a lot about the world but nothing about your company, your customers or your house style. Teams building AI features usually reach for one of three techniques to close that gap: prompting, retrieval-augmented generation (RAG) and fine-tuning. They solve different problems.
Updated September 2026: we added links to OpenAI’s guidance on choosing between these techniques and the original RAG research paper.
1. Prompting: tell the model what you want
Prompting means giving the model instructions, context and examples in the input. This includes system prompts that set behavior for every conversation, and few-shot examples that show the desired output format.
Best for: tone, format, rules, task instructions and small amounts of reference information.
Pros: instant, cheap to change, no special infrastructure. OpenAI’s guide to optimizing accuracy calls prompt engineering “typically the best place to start.”
Cons: everything must fit in the context window, and long prompts add cost to every request.
2. RAG: give the model the right information
RAG retrieves relevant documents at question time and adds them to the prompt. The 2020 research paper that introduced the term highlighted a key advantage: the model’s knowledge can be updated by changing the documents, without retraining. We explain the mechanics in our RAG explainer.
Best for: facts that change, large knowledge bases, answers that need citations and permissions-aware access to documents.
Pros: up to date, auditable, no retraining.
Cons: quality depends heavily on search and document quality; requires infrastructure to index and retrieve content.
3. Fine-tuning: change the model’s behavior
Fine-tuning continues training a model on your own examples, adjusting its internal weights. Providers offer fine-tuning for some hosted models, and open-weight models can be fine-tuned fully.
Best for: consistent specialized formats, a distinctive style, narrow classification tasks, or getting a smaller, cheaper model to perform a specific task as well as a larger one.
Pros: shorter prompts, consistent behavior, potential cost savings at scale.
Cons: needs good training examples, takes time and expertise, must be redone when base models change, and is a poor way to teach facts that change.
Quick decision guide
| If you need… | Start with |
|---|---|
| A particular tone or format | Prompting |
| Answers from your documents | RAG |
| Up-to-date information | RAG |
| Citations to sources | RAG |
| Highly consistent output at huge volume | Fine-tuning, after prompting |
| A small, cheap model for a narrow task | Fine-tuning |
| To teach the model new facts | RAG, not fine-tuning |
A common mistake Teams often jump to fine-tuning to teach a model about their products. That usually disappoints: facts go stale and the model may still guess. Fine-tuning shapes behavior; RAG supplies knowledge.
Combining approaches
Mature systems usually layer all three: a well-crafted prompt defines behavior, RAG supplies current knowledge, and, where volume justifies it, a fine-tuned model delivers consistent formatting cheaply.
Measure before and after
Whichever path you choose, build an evaluation set: a few dozen realistic questions with good answers. Test every change against it. Without measurement, it is easy to spend weeks on techniques that feel sophisticated but do not improve results. OpenAI’s guide makes the same point, starting every project with “a good prompt with an evaluation set of questions and ground truth answers.”
Start simple, add complexity when needed
Start with prompting, add RAG when you need your own knowledge, and consider fine-tuning only when you have clear evidence it will improve consistency or cost. Most successful projects follow exactly that order.
Sources
- Optimizing LLM Accuracy, OpenAI API documentation
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv, 2020
