Novia's Textbook Fact-Checking: Building the Chapter 14 Workbook

A fact check that only tells you "this looks accurate" is not a fact check, it is a vibe. This walkthrough builds something more rigorous: an audit-ready workbook for Chapter 14 of a textbook, one that classifies every single sentence, writes down the reasoning behind each classification, and catches internal contradictions a casual read would miss entirely.

A fact check is a workbook, not a vibe

The starting principle is simple to state and hard to skip: take every sentence in the chapter, classify it, and write the decision down. For Chapter 14, that means working through 138 sentences and producing three separate sheets of output, not a single green checkmark. The whole point of building a workbook instead of writing a summary judgment is that the reasoning behind every claim becomes visible and checkable by someone else.

Four moves: count, split, editorial, reread

The process breaks into four distinct moves. First, count every sentence in the chapter. Second, send statistics, guidelines, approvals, evidence, specialist claims, and dated facts out to the web for verification, flagging them as web-checkable; everything else gets marked AI-only, each with a stated reason it wasn't sent to be Googled. Third, run an editorial pass that checks the chapter against itself for internal consistency. Fourth, reread the AI-only pile specifically looking for inverted mechanisms, cases where a process or relationship got described backwards. Skipping any one of these four steps means the result isn't a real review, just a partial one.

Reconstructing the ledger

Working from Chapter 14's counts, the ledger comes out to 138 total sentences, 46 of which get flagged to go to the web, and 92 that are AI-only. Those 92 get split further by the six chapter files in reading order, and the numbers are checked to make sure they sum correctly, so no sentence can silently disappear from the count during processing. Components turns out to be the largest single file in that AI-only pile, at 33 sentences. A first pass that prints only the 46 web-flagged sentences is explicitly called out as a partial report: that's steps one and two done, with the sheet covering the AI-only 92 still missing.

Sheet A versus Sheet B: auditing the AI-only classifiers

A report that only publishes the 46 web-verifiable sentences has effectively dropped the AI-only classifiers from the audit entirely, which is exactly the gap the workbook is designed to close. Two more moves are still needed at this point: an editorial sheet, which turns up eight findings where the chapter contradicts itself, and a reread of the 92 AI-only sentences, which turns up five hallucination flags. Once the ledger is updated, the second pass prints all three sheets together: Sheet A holds the 46 that went to the web, Sheet B holds the 92 AI-only sentences each with a stated reason, and Sheet C holds the eight editorial findings.

Sheet C: contradictions and inverted mechanisms

The editorial and reread sheets are where the real risk lives, because these are errors a purely web-verifiable check would never catch. Examples include a drug being described as still in current use when it isn't, and a uniqueness claim that contradicts the chapter's own figure elsewhere in the text. The reread catches biological errors too: dendritic cells described incorrectly as killer cells, and a cytokine axis, specifically IL-17 and IL-23, written backwards. None of those five hallucination flags were ever sent to the web for checking, because they aren't factual claims a search engine would flag as wrong; they're internal errors only a careful reread of the AI-only sentences would surface.

Key takeaways

• A real fact check produces a workbook with a written decision for every sentence, not a single overall judgment.
• The four moves are: count every sentence, split web-verifiable claims from AI-only claims, run an editorial self-consistency pass, and reread the AI-only pile for inverted mechanisms.
• Chapter 14 broke down to 138 total sentences: 46 flagged for web verification and 92 classified as AI-only.
• The editorial pass found eight internal contradictions; the hallucination reread found five flags, including an inverted IL-17/IL-23 mechanism.
• Publishing only the web-verified sentences (Sheet A) without the AI-only sheet (Sheet B) or the editorial sheet (Sheet C) leaves the classifier and the chapter's internal contradictions unaudited.

Try it yourself

Paste this method onto another chapter and build the three sheets before arguing with any single sentence in it: every sentence counted, an AI-only sheet with a reason on every row, and at least one same-file contradiction check on the editorial sheet. If any of those three pieces is missing, the report isn't finished. This workbook approach comes out of the Medhavy track within the Humanitarians AI Fellows program, which focuses on rigorous, verifiable methods for reviewing AI-assisted content.