Identify high-risk claims in an LLM answer (facts, citations, numbers, code) and apply concrete techniques to catch hallucinations.
Why models hallucinate — and where to look
Why models hallucinate — and where to look
As you learned in Unit 1, an LLM generates text by predicting the next token from patterns in its training data — it does not look up facts or check a database. This means it can produce confident, well-written text that is simply false. Researchers at OpenAI (Kalai, Nachum, Vempala, and Zhang, 2025) argue that hallucinations persist because training rewards confident-sounding answers, much like a student who guesses on an exam rather than admitting they do not know.
Hallucinations cluster in predictable places. Watch for: specific statistics and percentages (the model may invent a plausible-sounding number); named sources, papers, or URLs (citations are frequently fabricated — the title and author sound real but the paper does not exist); quotes attributed to real people; recent events (the model's knowledge has a cutoff); and arithmetic (models without a calculator tool are unreliable on computation).
Four practical techniques help you catch them. First, ask the model for its sources and then check those sources yourself — do not assume a convincing citation is real. Second, ask the model to flag its own uncertainty: «Mark any claim you are not confident about». Third, cross-check independently using a search engine or authoritative database for any claim that matters. Fourth, treat code output as untested until you have run it — the model can produce syntactically correct code that does the wrong thing.
The goal is not paranoia but calibrated trust. Conversational text and brainstormed lists carry low hallucination risk. Specific facts, citations, numbers, and code carry high risk and always deserve a second look.
Lesson notes
Why models hallucinate — and where to look
As you learned in Unit 1, an LLM generates text by predicting the next token from patterns in its training data — it does not look up facts or check a database. This means it can produce confident, well-written text that is simply false. Researchers at OpenAI (Kalai, Nachum, Vempala, and Zhang, 2025) argue that hallucinations persist because training rewards confident-sounding answers, much like a student who guesses on an exam rather than admitting they do not know.
Hallucinations cluster in predictable places. Watch for: specific statistics and percentages (the model may invent a plausible-sounding number); named sources, papers, or URLs (citations are frequently fabricated — the title and author sound real but the paper does not exist); quotes attributed to real people; recent events (the model's knowledge has a cutoff); and arithmetic (models without a calculator tool are unreliable on computation).
Four practical techniques help you catch them. First, ask the model for its sources and then check those sources yourself — do not assume a convincing citation is real. Second, ask the model to flag its own uncertainty: «Mark any claim you are not confident about». Third, cross-check independently using a search engine or authoritative database for any claim that matters. Fourth, treat code output as untested until you have run it — the model can produce syntactically correct code that does the wrong thing.
The goal is not paranoia but calibrated trust. Conversational text and brainstormed lists carry low hallucination risk. Specific facts, citations, numbers, and code carry high risk and always deserve a second look.