AI Hallucinations Explained: Why Chatbots Make Things Up and How to Get Accurate Answers

What are AI hallucinations?

An AI hallucination is an answer from a chatbot or other generative AI system that is false, invented or inconsistent, yet delivered as if it were true. The US National Institute of Standards and Technology (NIST) uses the term confabulation and defines it as a phenomenon in which generative AI systems generate and confidently present erroneous or false content ^[1]. NIST includes outputs that diverge from the prompt or contradict the system's own earlier statements, and notes these are colloquially called 'hallucinations' or 'fabrications' ^[1].

Examples range from invented court cases with fake quotations, as in a 2023 US lawsuit ^[3], to hallucinated details and outdated facts in news answers ^[5]. NIST warns that systems can also produce confabulated logic or citations that appear to justify an answer, which may lead people to trust the output more than they should ^[1].

How do you avoid them, in short?

1. Give the chatbot the source material and ask it to answer only from that material ^[7].
2. Explicitly allow it to say 'I don't know' ^[7].
3. Ask for direct quotes and citations for each claim, then open the sources yourself ^[7].
4. Check anything important against an independent, authoritative source, because no technique removes the problem entirely ^[7].

Why do AI chatbots make things up?

Large language models (LLMs) generate text by predicting likely next words. NIST explains that confabulations are a natural result of this design: models produce outputs that approximate the statistical distribution of their training data, which can be accurate but can also be factually wrong or internally inconsistent ^[1]. NIST says the problem is particularly relevant for open-ended, long-form answers and for questions that need specialist knowledge ^[1].

Why don't models just say 'I don't know'?

A 2025 paper by Adam Tauman Kalai and colleagues, three of them at OpenAI and one at Georgia Tech, argues that hallucinations need not be mysterious: they originate as errors in binary classification ^[2]. If false statements cannot be told apart from facts, statistical pressures during training produce them naturally ^[2]. The authors add that hallucinations persist because most benchmarks score answers in a way that rewards a confident guess over an honest 'I don't know', so models are, in their words, optimised to be good test-takers ^[2]. Their proposed fix is to change how widely used benchmarks are scored ^[2].

Why do they sound so sure?

Fluent, confident wording is part of what models learn to produce. NIST notes that risks arise when users believe false content, often because of the confident nature of the response ^[1]. Confidence in tone is therefore not evidence of accuracy.

How often do AI chatbots hallucinate?

There is no single hallucination rate. It depends on the model, the task, the language and how errors are counted. Independent studies give a sense of scale for specific tasks.

News questions

A study coordinated by the European Broadcasting Union (EBU) and led by the BBC involved 22 public service media organisations in 18 countries working in 14 languages ^[5]. Journalists assessed more than 3,000 responses from ChatGPT, Copilot, Gemini and Perplexity ^[5]. They found that 45% of answers had at least one significant issue, 31% had serious sourcing problems and 20% contained major accuracy issues, including hallucinated details and outdated information ^[5]. Gemini had significant issues in 76% of its responses ^[5]. The EBU says these distortions were consistent across languages and territories, which matters for readers using AI in languages other than English ^[5].

Professional legal tools

Researchers at Stanford tested AI legal research tools from LexisNexis and Thomson Reuters and found that each hallucinated between 17% and 33% of the time ^[6]. Those tools use retrieval-augmented generation (RAG), which fetches documents before answering, and the authors concluded that providers' claims of eliminating hallucinations were overstated ^[6].

These figures describe particular systems at particular dates; newer versions may do better or worse. They show that even specialised tools grounded in documents still make errors ^[6].

What happens when people trust AI hallucinations?

Fake cases in a US court

In Mata v. Avianca, a federal judge in New York sanctioned two lawyers and their firm on 22 June 2023 after they submitted non-existent judicial opinions with fake quotes and citations created by ChatGPT ^[3]. The lawyers later acknowledged that six of the cited decisions had been generated by ChatGPT and did not exist ^[3]. When one lawyer asked ChatGPT whether a case was real, it replied that the case 'does indeed exist' and could be found on Westlaw and LexisNexis ^[3]. Judge P. Kevin Castel wrote that there is nothing inherently improper about using a reliable AI tool, but that existing rules give lawyers a gatekeeping role to ensure their filings are accurate, and imposed a $5,000 penalty ^[3].

The problem has grown since. A database maintained by researcher Damien Charlotin lists court and tribunal decisions in which a party was found to have relied on hallucinated content; it had identified 2,262 cases as of its update on 2 November 2026 ^[4].

A chatbot's wrong promise to a customer

In a 2024 decision (Moffatt v. Air Canada, 2024 BCCRT 149), British Columbia's Civil Resolution Tribunal held Air Canada liable for negligent misrepresentation after the airline's website chatbot wrongly suggested that a customer could apply for a bereavement fare retroactively, which another page of its website contradicted ^[9]. The tribunal rejected the argument that the chatbot should be treated as a separate entity, stating that it makes no difference whether the information comes from a static page or a chatbot ^[9].

How can you get more accurate answers from AI chatbots?

Anthropic's developer guide for its Claude models sets out techniques that apply to most chatbots ^[7]:

1. Allow uncertainty. Tell the model it may say it does not know or lacks enough information; Anthropic says this can drastically reduce false information ^[7].
2. Ground answers in quotes. For long documents, ask the model to extract word-for-word quotes first and then base its answer only on them ^[7].
3. Require citations and retractions. Ask for a supporting quote or source for every claim, and tell the model to remove any claim it cannot support ^[7].
4. Restrict its knowledge. Instruct it to use only the documents you provide, not its general knowledge ^[7].
5. Compare several answers. Run the same question more than once; inconsistencies between outputs can signal a hallucination ^[7].
6. Ask follow-up checks. Feed the answer back and ask the model to verify or expand on specific statements ^[7].

Checks you should do yourself

• Open every citation. Confirm that the source exists and says what the answer claims; 31% of answers in the EBU study had serious sourcing problems ^[5].
• Do not ask the chatbot to vouch for itself. The lawyers in Mata v. Avianca did exactly that and were told a fake case was real ^[3].
• Use primary sources for high-stakes facts, such as official legal databases, regulators or medical professionals. NIST flags healthcare as a domain where confabulated summaries could lead to wrong diagnoses or treatment ^[1].
• Check the date. Outdated information was among the major accuracy issues the EBU found ^[5].

Anthropic's own guide stresses that these techniques significantly reduce hallucinations but do not eliminate them, and advises validating critical information ^[7].

Can technology detect hallucinations automatically?

Researchers are building automatic warning signs. One approach, published in Nature in 2024 by Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn and Yarin Gal at the University of Oxford, is called semantic entropy ^[8]. It samples several answers to the same question, often about five, groups them by meaning, and measures how scattered those meanings are; high scatter suggests the model is likely to answer wrongly ^[8].

The method has limits. Its authors say it will not catch cases where a model was trained into a wrong set of facts or style of reasoning, cases where training does not transfer to a new context, or deliberate deception ^[8]. A model that confidently repeats the same wrong answer every time will look consistent. This is why the simple user technique of asking twice, which follows the same logic, is a hint rather than a guarantee ^[7][8].

Retrieval-augmented generation, which lets a model look up documents, helps but is not a cure: the Stanford study found double-digit hallucination rates in RAG-based legal tools ^[6].

Which prompting techniques work best?

No technique is perfect, and they work best combined. The table below compares common approaches, what they target and where they fall short. Techniques that tie answers to documents you supply address the root problem that models otherwise rely on statistical patterns ^[1][7], while consistency checks catch only some kinds of error ^[8].

What are the benefits and limits of using AI chatbots for facts?

Benefits

• NIST notes that statistical prediction can produce factually accurate and consistent outputs, not only errors ^[1].
• When given source documents, models can extract and cite exact quotes, which makes their answers auditable ^[7].

Limits

• Errors are common on some tasks: 45% of news answers had a significant issue in the EBU study ^[5].
• Even specialised legal tools hallucinated 17% to 33% of the time in the Stanford tests ^[6].
• Errors can come with convincing but false explanations and citations ^[1].
• Responsibility stays with the user or business: courts have sanctioned lawyers ^[3], and a tribunal held a company liable for its chatbot's statements ^[9].

Will AI hallucinations ever be solved?

How far hallucinations can be reduced is an open question, but some of the explanations above point to causes that can be worked on. If, as Kalai and colleagues argue, benchmarks reward guessing, changing how they are scored could encourage models to express uncertainty ^[2]. Detection methods such as semantic entropy can flag some likely errors ^[8]. NIST, however, describes confabulation as a natural result of the way generative models are designed ^[1], which suggests some error rate will remain.

What is still uncertain

• Real-world error rates for current models. Published studies test specific versions and tasks, and results change with each release ^[5][6].
• How courts will treat repeated cases. Charlotin's database had already logged 2,262 such decisions by 2 November 2026, and each new one adds to the case law ^[4].
• Whether benchmark reforms will be adopted widely, which the Kalai paper calls a socio-technical change ^[2].

As of 3 November 2026, the safest working assumption is that any chatbot can be wrong in a confident voice, and that checking the source is the user's job.