Can AI Catch Its Own Mistakes? Auditing the Self-Reference Myth

A sentence sitting in a real video script read like settled fact: a system cannot reliably audit its own output. Stated flatly, structurally, as if it were self-evident. Tanmay Kulkarni went looking for the number behind that claim, the tally, the trial count, anything that would justify stating it as fact. There wasn't one. Not in that sentence, not anywhere in the script it came from. So the only thing left to do was go test it directly.

The question and the expectation

The question is simple to state: when an AI gets something wrong and you ask it to check its own work, does it catch the mistake, or does it just agree with itself? Going in, the expectation was to confirm the claim. The honest starting assumption was that the model would wave its own mistake straight through, producing a tidy demonstration of the self-reference problem. That is not what happened. Thirty-three wrong answers, three model setups, and a result that didn't match the prediction at all.

A four-question framework for testing any AI claim

Before getting to the result, the more broadly useful part of this project is the method. Any time someone claims a system fails because of some specific cause, four questions decide whether that's a real finding or just a good story: what's the claimed cause; what else could produce exactly the same observation; is there a version of the test where only that one cause changes and nothing else; and, the question learned the hard way here, will that test actually collect any data. The first two questions are cheap to ask. The third is the one people skip most often. The fourth is the one that got skipped in this project's own first attempt.

Where the original test design failed

The claim under test names its cause as self-reference: a model checking its own work, using the same weights, the same activations, the same direction of error. But there's an obvious rival explanation: the task itself is simply hard, and a model that gets arithmetic wrong and stays wrong isn't protecting its ego, it just can't do the arithmetic. Those two explanations predict the exact same outcome. The original experiment design had the model check itself, then verify that check in Python, but both of those arms changed multiple things at once, so the design couldn't actually distinguish between the two explanations. That's not a wrong conclusion. It's no conclusion at all.

The corrupted operator method

The fix was small but decisive: take a wrong answer, show it to the model twice. Once inside its own conversation, where it looks like the model's own work. Once in a completely fresh conversation, presented as somebody else's submission. Same content, same question, with the only variable being who the answer appears to belong to. If self-reference really is the cause, that gap between the two conditions should be large. If it isn't, the gap should be nothing.

Building that test hit its own snag first: the models were simply too good. Testing arithmetic errors, Sonnet 5 solved nine out of ten problems, then ten out of twenty, meaning there weren't enough actual mistakes to measure how the model handled catching them. The intended experimental arm had zero trials in it. That's exactly what question four in the framework is meant to catch, and it's the one that got missed on the first pass.

Forcing real mistakes to test

The actual fix, matching what published research on this topic does, was to stop waiting for mistakes and manufacture them instead: take a correct answer, corrupt exactly one operator, and now there's a wrong answer with a controlled degree of wrongness, subtle, moderate, or obvious. Every trial in the resulting test had something real to catch, and critically, the version shown to the model as its own work and the version shown as a stranger's submission were now character-for-character identical except for that framing.

The result: 33 out of 33, and a null gap

Across 33 wrong expressions, 99 verdicts, and three model configurations, the model caught every single mistake, zero missed, not low, zero. And the gap between catching its own supposed mistake and catching a stranger's was zero as well. Provenance made no measurable difference. The grading was run twice out of disbelief at the result, and it held both times.

Why the honest statistical read matters

The instinct to overclaim here would be to say the self-reference effect is zero. That's not quite right, and stating it that way would repeat the same mistake being investigated. Zero out of 33 doesn't mean a true rate of exactly zero; running the actual statistics on that sample size puts the true rate somewhere below about 9 percent. That's a meaningfully narrow finding: one task, basic arithmetic on four numbers, checkable by a Python function in a millisecond. Nothing here says anything about code review, factual claims, or judgment calls, and the video is explicit about not extending the result beyond what it actually tested.

Two claims, and only one survives

The original script that sparked this investigation actually made two claims that sound like one: don't let a model be both judge and defendant, put a real check in place, and, separately, a model can't audit itself. The second claim didn't hold up; it was watched auditing itself successfully 33 times out of 33. But the first claim survives, and the reason it survives is exactly what made this experiment possible in the first place: a Python evaluator sitting outside the model, independently grading every single answer. Remove that external check, and there's no finding left, just a model telling you it checked.

Why publishing a null result matters

Null results tend to get quietly shelved in favor of topics that produce a cleaner story. That instinct is precisely how claims like the original one stay in circulation unchallenged: nobody publishes the run where the predicted effect didn't show up. A null isn't a failed experiment. It's an experiment that answered the question it was designed to ask.

Key takeaways

• A claim stated as fact with no supporting numbers is a hypothesis, not a finding, no matter how confidently it's phrased.
• Use four questions to test any causal claim about AI behavior: the claimed cause, the rival explanation, a test that isolates only that cause, and whether the test will actually collect data.
• The corrupted operator method creates guaranteed, controlled-severity mistakes so a self-audit experiment has something real to measure.
• A controlled test found the model caught 33 out of 33 deliberately corrupted answers, with no measurable difference between self-attributed and stranger-attributed submissions.
• The honest statistical conclusion is that the true miss rate is below roughly 9 percent for this narrow task, not proven to be exactly zero, and doesn't generalize to harder judgment calls.
• The external, independent evaluator is the actual instrument that makes any of these results trustworthy, not an optional safety net.

Try it yourself

Take a claim you currently believe, something with a "because" in it, a system fails because of X. Write the cause down in one sentence, then write a rival explanation that would look identical from the outside. Describe a test where only the cause changes, and then ask the last question: would that test actually collect any data, or are you waiting on a failure that almost never happens? This kind of rigorous, self-correcting inquiry is the standard behind Humanitarians AI Fellows projects like this one.