Grounding & Retrieval: Why Citations Do Not Make an AI Answer True
Retrieval gives a relevance signal, not a truth signal. Adwait Changan walks through three citation checks, shows a four-answer test where a verbatim quote still supports a false claim, and argues for putting quote and claim side by side for a human.
An AI answer with a citation attached looks trustworthy, but the video by Adwait Changan argues that a citation does much less than people assume. The key distinction is between an agent finding a document and an agent actually using it. Those are two different events, and only one of them can be checked automatically.
Three checks a citation must survive
For a citation to mean anything, it has to pass three checks. Does the source actually exist? Does the quoted text appear in it word for word? And does that quote actually support the claim being made? The first two are string comparisons that code can do. The third is a judgment about meaning, and no amount of engineering turns it into a string comparison.
Relevance is not truth
The video also looks at what retrieval alone gives you. It returns a document that scored well against the question, which is a relevance signal and not a truth signal. Retrieval answers whether anything looked related. Grounding asks whether a specific sentence follows from a specific passage. The speaker's complaint is that most systems ship the first and call it the second.
A four-answer test
To make this concrete, the video shows four answers, each with a citation, and runs the automatic checks. One cites a document that does not exist. One quotes text that is not in the document it names. The other two pass cleanly. But when you look at whether the claims are true, one of those two passing answers is false, and it is false in the most dangerous way, because everything a reviewer would check by habit comes back clean.
The 14 versus 30 days case
The failing claim says that travel is reimbursed within 14 days. The cited document is real, and the quoted sentence appears in it word for word. That sentence says 30 days. Every automatic check passes, yet the answer is wrong, and no stricter string matching would have caught it. The video then shows an honestly labeled 12-line checker whose docstring says it checks only whether the source is missing and whether the quote is not in the source. It never decides whether the quote supports the claim, and pretending otherwise would be the real bug.
What a citation buys you
The video lists what a citation proves: that the source exists, and that the text was not invented. It does not prove the quote is relevant, that the claim follows from it, or that the answer is true. Four of these six are things people assume they get for free.
From there it suggests three things to build, in order of value. Check that the source exists, which is nearly free. Check that the quote is verbatim, which catches invented text. Then put the quote beside the claim where a human can see both at once, because that is the only check that catches 14 against 30. In this view a citation is not there to convince the reader but to let the reader disagree quickly. A source link nobody can open next to the sentence it supports is decoration.
Key takeaways
- Finding a document and using it are different events, and only one can be checked automatically.
- A citation should pass three checks: source exists, quote is verbatim, quote supports the claim.
- Only the first two checks can be automated with string matching.
- Retrieval returns a relevance signal, not a truth signal.
- In the four-answer test, two citations passed every automatic check and one was still false.
- The failure to watch for is a real, verbatim quote that the claim goes further than.
- Interfaces should show the quote beside the claim so a human can disagree fast.
Try it yourself
Take three answers your system produced with citations attached. For each one, quote the exact sentence from the source that supports it, then ask whether that sentence actually says what the answer claims or only something nearby. Flag every case where the citation is real but the claim goes further than the quote. The video notes that this next topic, guardrails and evaluation, is covered in the following episode.
Chapters
- 0:00Finding vs using a document: Relevance signals vs truth signals
- 0:15The three citation checks: Source existence, verbatim quotes, and claim support
- 0:30The 14 vs 30 days test: Why automatic string checks miss false claims
- 0:50What citations actually prove: Catching invented text vs verifying truth
- 1:10Building for human evaluation: Spotting claims that go further than the quote
Full transcript(auto-generated, with timestamps)
Finding vs using a document: Relevance signals vs truth signals
[0:00]Hi, I am at wait Changan. In this video is about why citations do not make an AI answer true. Today we are going to learn the difference between an agent finding a document and an agent actually using it. Those are two different events and only one of them can be checked automatically. A citation has to survive
The three citation checks: Source existence, verbatim quotes, and claim support
[0:16]Three checks before it means anything. Does the source actually exist? Does the quoted text appear in it word for word? And does that quote actually support the claim being made? The first two are string comparisons. The third is a judgment about meaning and no amount of
The 14 vs 30 days test: Why automatic string checks miss false claims
[0:30]Engineering turns it into one. And notice what retrieval alone gives you. It hands back a document that scored well against the question. That is a relevant signal, not a truth signal. Retrieval answers did anything look related? Grounding answers does this specific sentence follow from that specific passage? Most systems ship the first and call it the second. So here
What citations actually prove: Catching invented text vs verifying truth
[0:50]Are four answers, each one with a citation attached. Run the automatic checks. One sites a document that does not exist. One quotes text that is not in the document it names. And two of them pass cleanly. Now look at the right-hand column, which is whether the claim is actually true. Two passed. One of those two is false. And it is false in the most dangerous way available
Building for human evaluation: Spotting claims that go further than the quote
[1:10]Because everything a reviewer would check by habit comes back clean. The claim says travel is reimbursed within 14 days. The cited document is real. The quoted sentence is in it word for word. And that sentence says 30 days. Every automatic check passes. The answer is still wrong and no stricter string matching anywhere would have caught it. And here is the checker honestly labeled. 12 lines, source missing, quote not in source, otherwise the citation checks out. Read the doc string because it says what the function does not do. It never attempts to decide whether the quote supports the claim and pretending otherwise would be the actual bug. So be precise about what a citation buys you. It proves the source exists. It proves the text was not invented. It does not prove the quote is relevant. It does not prove the claim follows from it. And it absolutely does not prove the answer is true. Four of those six are things people assume they got
For free, which gives three things worth building in order of how much they buy you. Check the source exists because it is nearly free. Check the quote is verbatim because that catches invented text. And then put the quote next to the claim where a human can see both at once because that is the only check that catches the 14 against 30, which reframes what a citation is for. It is not there to convince the reader. It is there to let the reader disagree quickly. If your interface shows a source link that nobody can open beside the sentence it supports, you have built decoration. So, the verdict retrieval finds what looks related. That is a relevant signal, not a truth signal, and shipping the first as the second is the most common mistake in this whole area. A citation survives three checks. The source exists, the quote is verbatim, and the quote supports the claim. Only the first two can be automated. In four test
Answers, two citations passed every automatic check, and one of those two was still false because a real document was quoted word for word to support a number it does not say. Build for the third check. It is the only one that decides whether the answer is true. Your turn. Here is the prompt. Take three answers my system produced with citations attached. For each one, quote the exact sentence from the source that supports it, then tell me whether that sentence actually says what the answer claims or only something nearby. Flag every case where the citation is real, but the claim goes further than the quote. Run that against your own output. The phrase to watch for is goes further than the quote because that is the failure that survives every automatic check you will ever write. That was grounding in retrieval. Next time, guardrails and evaluation and how you catch an agent failing when you cannot watch every run. I'm Wait Chang Gan for Humanitarian's AI.
More from Humanitarians AI Fellows
4:50Two-Week Progress Review: Turning Tracker Data into an AI Dashboard | HAI Project Manager
2:02Mycroft's Schema Drift: Unplanned Structural Changes & Silent Pipeline Failures
5:16AI Effort Estimation: Stop Guessing and Estimate from History
3:44What Happens Inside a Single Transformer Layer?
3:42Planning & Decomposition: When AI Agent Plans Are Worth the Cost
1:51