How to Fact-Check an AI Answer
Chatbots sound exactly as confident when they are right as when they are making it up. This five-step routine takes about five minutes and catches most of the damage. It includes the prompt that makes the model show its sources and say how sure it is.
Why the confident tone means nothing
A hallucination is a chatbot answer that is stated confidently and is not true. It is not a rare glitch; it is a side effect of how these tools work. A language model (the program behind ChatGPT, Gemini, Claude and Copilot) predicts the most plausible next words, and a plausible guess looks the same as a fact. OpenAI's research paper from September 2025 argues that standard training and testing reward guessing over saying "I don't know", the way a student guesses on an exam rather than leaving it blank. Anthropic's help center says Claude can "occasionally produce responses that are incorrect or misleading". Google's Gemini help says plainly that Gemini "can make mistakes" and tells you to double-check responses. The makers agree. Checking is your job.
Six hallucination patterns to recognize
- The perfect citation. A paper with a title, authors, year and journal that all sound right and does not exist. Anthropic has a whole help page on Claude producing links that do not work.
- The suspiciously precise number. "Adoption grew 37.4 percent" with no source. Real statistics come with a who and a when.
- The confident quote. A real person, invented words. Quotes are the easiest thing to fabricate and the most embarrassing to repeat.
- The claimed action. "I've emailed that to you" or "I checked the website". Unless the tool is visibly connected to email or search, it did not.
- Agreeing with your premise. Ask "why did X cause Y" and it will explain, even if X never caused Y. Ask "did X cause Y?" instead.
- Stale facts as current. Prices, software versions, laws, who holds a job. Every model has a training cutoff date, and without search turned on it does not know what changed since.
The five-step routine
1Pull out the claims
Read the answer and mark every statement that could be true or false: names, numbers, dates, quotes, "studies show", "the official rule is". Everything else is opinion or padding, and you do not check padding. Most answers contain fewer checkable claims than they appear to.
2Ask for sources and confidence
Paste the prompt below into the same chat. If the tool has a search mode, turn it on first: OpenAI's help center says ChatGPT search shows inline citations you can click, with a Sources button under the reply; Claude's web search includes citations in every response, per Anthropic's help pages. Gemini has offered a double-check button that highlights statements it finds supported in Google Search, though its availability has varied, so if you do not see it, ask Gemini to search.
3Open the sources
Click every link. Does the page exist? Does it say what the model said it says? A real page that says something slightly different is the most common failure, more common than a fake link. If the "source" is a homepage rather than a specific page, count it as no source.
4Cross-check the ones that matter
For anything involving money, health, law or your reputation, find a second source that did not come from the chatbot: the official website, a government page, the original paper. Or open a fresh chat in a different chatbot with search on and ask the same question, without hinting at the first answer. Two independent sources that agree are evidence. Two chatbots that agree are not, since they may share the same wrong training data.
5Decide and record
Keep, fix or drop each claim. Sourced facts go into your work. Unsourced ones get cut, or labelled "according to the AI, unverified" if you must keep them. Note where you checked, so the next person (often future you) does not repeat the work.
Three power moves
- Make it argue against itself: "What is the strongest case that your answer is wrong?" Models are good at finding holes in text, including their own.
- Give it permission to not know: Anthropic's developer guidance says explicitly telling the model it may say "I don't know" reduces false information. Add "if you are not sure, say so" to any prompt where facts matter.
- Calibrate on what you already know: ask about your own town, employer or hobby. You will see how often it slips, and in what ways, before you rely on it for something you cannot check.
Try it now: ask any free chatbot for five facts about a subject you know well, then paste the prompt above. Open every source it gives. Count how many claims were rated high confidence and were still wrong or unsourced. That number is your reason to keep this routine.
Quick answers
Why do AI chatbots make things up?
A language model predicts plausible text. When it does not know something, it often produces a plausible guess instead. OpenAI's 2025 research paper argues that training and testing methods reward guessing over admitting uncertainty, so models learn to answer confidently even when they should not.
Does asking for sources stop hallucinations?
No, it exposes them. A model can invent a source as easily as a fact. The value is in opening the source and reading it. A citation you never click is worth nothing, and a real page that says something different from the summary is the most common failure.
Which AI chatbot is the most accurate?
None is reliable enough to skip checking, and OpenAI, Anthropic and Google all say so in their own help pages. Search-enabled modes such as ChatGPT search and Claude web search reduce out-of-date errors because they read live pages and cite them, but a cited page can still be misread.
