An Agent That Finishes First Can Be Worse Than One That Stops

Liam, in for Professor Bear, breaks down a failure mode that does not look like a failure at all: an AI agent that finishes a job, reports it done in a confident tone, and never mentions the files it could not reach.

The assumption that breaks

The natural read of a confident "done" is that the agent went through everything in scope. Liam opens with the case that breaks that assumption. An agent drafted a six-bullet brief with a clear recommendation, and it shipped to leadership. Two days later, someone found three dissenting documents sitting in a subfolder the agent never opened, one of which said the opposite of the recommendation. No error ever appeared in the report.

Why the omission stays silent

An agent works through a job one operation at a time, and each step only reports on itself: open a file, success; summarize it, success; move to the next, success. What the agent does not know is what it could not reach in the first place, whether that is a subfolder it lacked permission for, a scanned page that would not parse as text, or a file that scrolled past when a long listing got cut off. A crash surfaces a visible failure. A silent omission does not, because the completion report is built from successful operations, not a census of everything that existed.

The one flag worth checking

Some tools do log every skipped or unreadable file in a separate section, but a confident "done" alone does not tell you which kind of tool you are looking at. Liam frames this as the one thing worth checking before trusting a summary: does this tool surface skips at all, or does it only tally what succeeded.

The Myra case

Liam walks through a concrete example: Myra ran an agent over twelve client PDFs to draft a digest. The agent read nine, could not parse three that were scanned images, and reported the digest done. Those three documents held the revised targets. The digest shipped with the old numbers, and nothing in the report said anything was missing.

Asking for the inventory, not the summary

The fix is to ask for an inventory instead of a summary: documents in scope, documents opened, documents skipped. A mismatch, like three skipped against twenty-six in scope, is what catches the gap. Liam is careful about what this does and does not prove in both directions. A matching inventory does not prove every document was read correctly, only that none went silently missing. And one skipped file does not mean the whole brief is wrong, just that one document needs a second pass.

Key takeaways

• A confident "done" only reports what finished, never what got skipped.
• Silent omissions happen because each operation reports on itself, not on the full scope of the job.
• Some tools log skipped files separately, but you cannot tell which kind you have from the report alone.
• Ask for an inventory: in scope, processed, and skipped, rather than accepting a summary at face value.
• A matching inventory rules out silent loss; it does not guarantee every file was read correctly.

Who this is for

This is for anyone relying on an AI agent to process a batch of files, documents, or records, who wants a concrete habit for catching what got silently left out before a summary gets acted on.