Ask an AI assistant for five books on a niche topic and you may get a tidy list, complete with authors and publication years, that includes a title no one ever wrote. This is what people call an AI hallucination: a confident answer that is partly or entirely made up.
Updated September 2026: we added OpenAI’s research on why models hallucinate, a study of fabricated citations and the details of the best-known court sanctions case.
Hallucinations are not a bug that will be patched next month. OpenAI researchers argued in 2025 that they arise partly because training and evaluation reward guessing over admitting uncertainty. They are a side effect of how language models work, as we explain in our plain-English guide to LLMs. The good news is that they follow patterns, and a little skepticism goes a long way.
Why AI makes things up
A language model generates text that is likely to follow your question. Most of the time, likely text and true text are the same thing. But when the model has weak or conflicting information, it can still produce something fluent and specific, because fluency is what it was trained to deliver. It has no built-in sense of embarrassment about being wrong.
The warning signs
Be most careful when an answer includes:
- Precise numbers such as percentages, prices or statistics with no source
- Citations and references, especially academic papers, court cases and page numbers; a 2023 study found that 55% of the citations GPT-3.5 generated, and 18% of GPT-4’s, were fabricated
- Quotes attributed to real people
- Niche facts about small companies, local rules or obscure people
- Recent events, which may be past the model’s knowledge cutoff
- Confident answers to ambiguous questions where a careful expert would say “it depends”
A five-minute fact-check routine
- Ask for sources. If the assistant has web search, ask it to cite where each key claim comes from.
- Open the sources. Check that each link exists and actually says what the summary claims. This step catches most problems.
- Check one claim independently. Search for the most important fact yourself, using an official or primary source.
- Ask the model to challenge itself. A prompt like “Which parts of your answer are you least certain about?” often surfaces weak spots.
- Match effort to stakes. A dinner recipe needs less checking than a medication question or a contract clause.
A simple rule If you would be embarrassed to repeat a claim to your boss without knowing where it came from, do not repeat what the AI said without checking it.
How to reduce hallucinations in the first place
- Give the AI the source material. Summaries of documents you upload are far more reliable than answers from memory.
- Turn on web search for anything current.
- Allow uncertainty. Add “If you are not sure, say so” to your prompt. Our prompting guide covers more techniques.
- Ask narrower questions. Broad questions invite the model to fill gaps.
Where the stakes are highest
Lawyers have faced court sanctions for filing briefs that cited cases invented by AI tools; in the best-known example, a New York federal judge fined two lawyers and their firm $5,000 in June 2023 over fake cases produced by ChatGPT, and similar embarrassments have happened in journalism and academia. The pattern is always the same: someone trusted a fluent answer without opening the source. Medical, legal and financial questions deserve particular care, and a qualified professional should make the final call.
Trust, but verify
Treat AI answers like a well-informed friend’s opinion: useful, often right, and worth checking before you act on anything important. Verification takes minutes. Cleaning up after a confident mistake can take much longer.
Sources
- Why language models hallucinate, OpenAI, September 2025
- Fabrication and errors in the bibliographic citations generated by ChatGPT, Scientific Reports, 2023
- Update on the ChatGPT case: counsel who submitted fake cases are sanctioned, Seyfarth Shaw, June 2023
