Process Supervision & Multi-Hop Reasoning Explained | AI Reasoning and Validation | Uday Sonawane

[0:00]I'm Uday S. Anawane, process supervision and multi-hop agents are two different things. One is how you grade, the other is what you asked. They belong in the same sentence for exactly one reason, and it is a measured one. On the benchmarks built to test multi-step reasoning, you can score well without doing the reasoning. First, what the phrase actually joins. Process supervision is a way of grading. You score every intermediate step instead of only the final answer. Multi-hop is a kind of question, one that cannot be answered without chaining several facts together. Those are not the same category. The reason they get said together is a failure mode, a model

[0:33]Producing the right answer from a chain that was never actually followed. Outcome grading, by construction, cannot tell the difference. Three questions for any claim that grading the steps works better. One, what counts as a step? Because in arithmetic, a step is a checkable object, and in an agent trajectory, it is whatever you decided to cut on. Two, who labeled the steps? A human or a process that inferred the labels from how things turned out. Three, is the advantage fundamental or practical? Because those have different consequences for whether it will hold in your setting. Here is the measurement that makes this concrete. There is a probe called

[1:05]Disconnected reasoning. You split the supporting evidence so no single piece can answer the question, and you see how well a model still does. On HotpotQA, a model scores 68.8 answer F1 while bypassing the reasoning steps. On TwoWikis, 63.4. On MuSiQue, a dataset built specifically to resist shortcuts, 37.8. Lower is harder to cheat. There's a second tell in the same paper, and it is subtler. Their best model scores 57.9 on two-hop questions, 47.9 on three-hop, and 28.1 on four-hop. It degrades as the chain gets longer, which is what actually doing the chain looks like. Another model stays at roughly 42% no matter how many hops the question needs.

[1:48]A flatline across hop count is not robustness. It is a model that is not hopping. So, that is the problem. Process supervision is the proposed answer. Grade every intermediate step rather than final answer alone and stop at the first step that is wrong. The paper everyone cites reports a process supervised model solving 78% of problems from a representative subset of the math test set and says process supervision significantly outperforms outcome supervision. What that took is the number worth remembering, 800,000 step level labels written by humans. Now the part that keeps this honest because process supervision is not a law. An earlier study on grade school maths found

[2:30]Outcome and process supervision reaching similar final performance, so the advantage is not universal across difficulty. And a theoretical result goes further. Under standard assumptions about data coverage, learning from outcomes is no harder statistically than learning from steps up to polynomial factors. If that holds, the gain is not information. It is practical, which is still worth having and is a different claim. Which brings us to the question that decides whether any of this transfers. In arithmetic, a step is a checkable object. A human can look at one line and say whether it follows. In an agent trajectory, a step is whatever you decided to cut on, a tool call, a

[3:07]Thought, a retrieval, and whether it was correct often is not knowable until the end. 800,000 human labels existed for maths. There is no comparable public corpus for agent steps, and when labels are inferred from how the episode turned out, outcome information has come back in under a new name. Scored what counts as a step clear in maths, a design decision in an agent. Who labeled them? Humans 800,000 times for the result everyone quotes. Fundamental or practical? The theory says the information is not the bottleneck. So be excited that there is now a measured way to catch a right answer built on a broken chain and that the

[3:43]Benchmarks have started being rebuilt around it. Be skeptical of process supervision claimed for an agent without saying what a step is and who decided it was correct. Your turn. Next time you read that an agent was trained or evaluated with process supervision, ask three things. What one step was, who labeled it correct, and whether that label could have been known before the episode ended. If the answer to the third is no, you are reading outcome supervision with extra steps. Right answer, wrong chain.