Why Self-Checking Is Not Independent Verification

Someone asks whether Claude rechecking its own answer already counts as verification. Liam explains the difference: that is self-checking, not independent verification, and the distinction matters because of what a self-check structurally cannot catch.

An agent fabricates two competitors, silently

Consider an agent pulling together a competitive analysis from five industry reports. Two competitors named in the final output never appear in any of the source documents. No error message shows up anywhere. What actually happened: the agent read two of the five reports, and for the rest, it drafted from training data instead, then ran an internal consistency check. That check found no contradictions, because every part of the output came from the same source, the model's own reasoning. It reported the task complete.

Why a self-check passes on fabricated data

The question worth asking is why a self-check would pass on work that fabricated two data points. The answer is that self-checking and independent verification are not the same thing. A system reviewing its own output works from the same context as the step that produced it: the same reading of the sources, the same assumptions, the same blind spots. A check built this way cannot catch an error the generation step already baked in, because the same system cannot independently verify itself.

The anchor case: 15% misread as 50%

Take a concrete example. An agent misread 15 percent as 50 percent in a source table. It then ran a consistency check comparing its own summary against its own recalled version of that table. The recall also said 50 percent, so of course the check passed, the same wrong number was being compared against itself. Opening the actual source document showed the real figure: 15 percent.

The fix: compare to the source, not to recall

Independent verification means comparing the output to the real source, not to the agent's memory of that source. Open the cited document. Find the sentence the agent claims to quote. If the source says something different from what the agent reported, this kind of check catches it, something the agent's own recall never could, since recall and self-check draw from the same flawed memory.

The practical takeaway

After any agent task, open at least two of the cited sources and check the claims made about them personally. If the agent skipped a file entirely, that file is exactly where a contradiction is most likely hiding, since nothing generated from it was ever checked against the real content. A self-check that passes is not proof the work is correct. The check cannot catch what the generation step already got wrong. Independent verification means going to the source, never to the agent's memory of it.

Key takeaways

• A self-check reviews output using the same context, assumptions, and blind spots that produced the output in the first place, so it cannot catch errors baked in during generation.
• An agent can fabricate data from training knowledge instead of reading a source, then pass its own consistency check because everything in the output agrees with itself.
• In the anchor example, an agent misread 15% as 50%, and its self-check passed because its recall also said 50%, the same error compared against itself.
• Genuine independent verification means comparing the agent's claims against the actual cited source document, not against the agent's memory or recall of it.
• A practical habit is opening at least two cited sources after any agent task and checking claims personally, especially any source the agent may have skipped reading.

Who this is for

Anyone building or relying on AI agent pipelines that include a self-review or self-check step, who needs to understand why that step alone is not sufficient and what an actually independent verification step requires.