How Do You Know It Worked? Grading Claims Over Stories

Why fluent agent explanations are post-hoc narrations, not process logs

[0:00]Hi, I'm Divge Power. Last time we ended on an uncomfortable fact, an agent that stuck can look identical to one that succeeded right up until someone checks. Today, three real mechanisms that turn the agent says it worked into something you can actually stand behind and exactly where they stop working.

Parsing prose into four checkable claims: Citations, numbers, hedges, and causes

[0:16]Here's the uncomfortable starting point. When an agent explains its reasoning, I checked the numbers, cross-referenced the filing, and concluded X, that explanation was generated by the same model that generated the conclusion. It's not a transcript of what happened. It's a plausible-sounding narration

Verifying citations against raw 10-K filings with a 1% tolerance window

[0:31]Produced after the fact. A model that hallucinated a number can write a perfectly coherent paragraph about how it derived that number. Fluency and accuracy are different properties and a language model optimizes hard for the first one. The real question isn't whether the explanation sounds reasonable, it's whether any part of it can be checked against something outside the model

Intentional self-disagreement: Detecting numerical drift across parallel runs

[0:50]Itself. The log is evidence of output. It is not evidence of process. The first move is to stop treating the reasoning as one continuous story and start treating it as a pile of individual checkable assertions. Run the text through a parser that pulls out four kinds of claims. Citations, wherever the text points to a source. Quantitative claims, dollar figures,

Why verification catches errors and drift without proving absolute truth

[1:10]Percentages, multiples, basis points. Hedge language, words like approximately or assumed, flagging the model's own uncertainty. And causal claims. Sentences using because or resulting in, where the model asserts one thing caused another. Take a real line, revenue grew 34% year over year, driven by international expansion, source the 10K. The parser tags it three ways at once, quantitative, causal, and citation. None of this requires understanding what the text means. It's pattern matching, regular expressions hunting for percent signs, source brackets, and a fixed list of hedge words. You're not asking a second AI to judge the reasoning. You're mechanically converting prose into a checklist of falsifiable pieces. Extraction gives you a list of claims. It doesn't tell you if any of them are true, that's the next step and it actually leaves the model's head entirely. Take that citation, source the 10K, and the system fetches it for real. If it's a government filing, it parses the structured data directly, real reported numbers, not text scraped off a page.

Otherwise, it searches the raw text for numbers. Then it checks, does a matching number show up in that source? Within about 1% to allow for rounding, a citation lands in one of three states. The source is reachable and a number matches. The source is reachable and nothing matches, a real red flag. Or the fetch itself fails, unattainable, no judgment possible either way. Roll all of that up into one number, the fraction of citations actually confirmed, and a human reviewer has something to act on that has nothing to do with how convincing the writing sounds. Say our 34% claim checks out against the real filing. Across a whole reasoning trace, that might land at six of nine citations confirmed, 67%. One honest caveat, the check isn't perfectly tight. It pulls every number in the whole reasoning trace and asks whether any of them shows up in the source. So, a citation can get marked confirmed on the strength of a completely unrelated number. That's a real gap,

Not a rounding error, and it's worth knowing before you trust the number too much. Verification catches claims that point at real sources, but plenty of reasoning doesn't cite anything external at all. It's synthesis judgment based on everything above, I conclude X. You can't fetch a source for that, so you use a different trick, ask the same question twice. Run the identical query through the full reasoning process a second time, completely independently. Then compare the two conclusions, how much do the actual words overlap? And weighted more heavily, how much do the specific numbers overlap? A model reasoning from real evidence should land in roughly the same place twice. A model confabulating a plausible story tends to drift, especially on the numbers, because a fabricated number is much harder to reproduce identically than a fabricated vibe. Score it, classify it, high agreement, medium, or low, and raise one specific hard flag whenever a number shows up in one run, but not the other at all. Run our example

Twice. If 34% shows up both times, that's real signal, high agreement score something like 0.82. If the second run says 41% instead with no 34% anywhere, that's a flag, not just a lower score. A number that only exists in one of two otherwise similar runs is one of the strongest signals available that it was invented, not retrieved. Here's the part that's tempting to skip and the part that actually matters most. None of these three mechanisms proves the agent reason correctly. High consistency is weak positive evidence, not a guarantee. Two runs can agree with each other while both being confidently, identically wrong. A model doesn't need real evidence to be consistent. It just needs to draw on the same internalized pattern twice. Verification only checks that a number exists somewhere in the cited source. It can't confirm the agent's causal story about what that number means. And a citation coming back unattainable tells you nothing either way. What these mechanisms are actually good at is the other

Direction, catching disagreement, catching absence, catching drift. Low consistency is strong negative evidence. It's very hard to accidentally look inconsistent while being right. A citation where the source is real but the claim number simply isn't there is a real structural problem, full stop. These tools reliably catch when something is wrong. They were never built to certify that something is right. Treating them as if they could just relocates the trust problem one layer down instead of solving it. So, here's what actually holds up. One, don't grade the story, grade the claims. Break reasoning into small, checkable pieces before you evaluate any of it. Two, check what can be checked against the outside world and log honestly when something can't be checked at all instead of quietly assuming pass. Three, when nothing external exists to check against, make the system disagree with itself on purpose. Geniune drift under repetition is one of the few signals a fabrication can't easily fake. None of these three prove truth. All three narrow

What's plausible. So, don't grade the story, grade the claims. Check what can be checked against the outside world and log it honestly when something can't be. And when nothing external exists to check, make the system disagree with itself on purpose because genuine drift is one of the few things a fabrication can't fake. None of this proves truth. All of it narrows what's plausible. Your turn. Take a real reasoning trace, one of your own agent's outputs, or a sample one. First, tag it by hand. Mark every citation, every number, every hedge word, every causal claim. Then pick one citation and actually go check it against the real source. Did the number show up? If there's no external source to check, run the same prompt twice and see whether the specific number survived the second pass. That gap between checking and proving is exactly where the next problem lies. What happens when you've got more than one agent and they don't agree at all? Signing off, Divya Pawar.