How to Engineer an AI Agent: Pipelines vs. Autonomous Loops

The truth table test: Is it deterministic code or a model's job?

[0:00]Hi, I'm Divage Pow. This is part four of the STEM agent series. Today, how do you actually design an agentic AI workflow? How to find the one piece that's really the model's job, how to choose your orchestration pattern, and how to derive guardrails instead of copying someone else's. Traditional software writes an if then rule for every case you can imagine because the real world doesn't fit into a false statements. Agentic engineering flips that. Instead of scripting the whole problem, you isolate the one part that's genuinely unstructured and hand only that part to a model. Here's the test. Can you write a truth table for this decision? If yes, it's deterministic code. Write the false. If the boundary itself is fuzzy, a visual judgment call. An ambiguous sentence, an incomplete instruction, that's the model's job, and everything else stays engineered. The boundaries,

Autonomous loops vs. Fixed deterministic pipelines

[0:45]The tools, the state. Once you know which piece is the model's job, there's a second decision. Who decides what happens next? One option is an autonomous agent loop. The model itself decides which tool to call in what order and when to stop. It's flexible and it's how a lot of demos get built. The other option is a fixed deterministic pipeline. The engineer decides the order in advance and the model only reasons inside each step. It's less flexible and it's what most production systems touching real consequences actually use. The case study we're about to walk through picks the boring option on purpose. Perception, then location, then grounding, then synthesis. Always in that order. Predictability beats flexibility the moment you're touching legal or government systems. Here's a real case study. An open-source Android app called Pothole Reporter. It handles

Case Study: Road damage assessment using a vision language model

[1:30]Three kinds of civic reports. Potholes, garbage, and open manholes across several Indian states and every mapped national highway. But only one of those three actually uses AI. Garbage and manhole reports are just what the user says they are. A person looked and confirmed it. Potholes are different. Judging whether road damage is bad enough to report is the fuzzy call the app hands to a vision model. A photo isn't an agent. An agent bridges raw perception and contextualized action. Let's reverse engineer how this one does it. Step one, when you approach a problem as an agentic engineer, isolate the unstructured variable, the part you genuinely can't hardcode. Here it's the physical environment. A pothole looks different in rain at night from any angle. So a vision language model handles perception. Find it, size it, call it clear, probable, uncertain, or

The 4 pipeline steps: Perception, Tool Use, Grounding, and Synthesis

[2:15]Absent. Notice how narrow that job is. The model's blast radius here is just, "Does this look like reportable damage?" Never who's liable for it. The narrower the question you hand the model, the easier it is to catch it being wrong. But perception alone isn't enough. The model has no local context. It doesn't know whose road this actually is or who's supposed to fix it. Guess at an authority from a photo alone, and you risk naming the wrong office, which is worse than naming none. that context gap gets closed with tools and retrieval, not more prompting. So instead of letting the model decide what to do next, this workflow runs a strict deterministic pipeline. The engineer enforces the order. First, the photo drive mode runs a continuous background scan through the phone's camera, and nearby repeat sightings of the same pothole get folded into one report instead of 10. The frame goes to the vision model, which judges the damage and reports it as clear, probable, uncertain, or absent, never a fake, precise percentage. Second tool use. The agent grabs the exact GPS coordinate and

Three questions to derive your own guardrails

[3:12]Makes a deterministic call to open street map, reverse geocoding it into a street address. Third, knowledge grounding. In this step only runs for supported Kronautica routes. Nothing is invented for the other states. The system ranks the coordinate against a downloadable pack of 13,577 contracts, itself filtered down from a public procurement snapshot of over 42,000 rows and short lists up to 25 candidates. A model may then pick one above a confidence threshold. And most of those contracts don't even have a known contractor name on file. So the app leaves that field blank rather than invent one. Fourth, action synthesis. The photo, the coordinate, the road ownership check, and any contract match feed back into the model with a strict prompt template. Out comes one complaint addressed to the right recipient, but the app never sends it. It opens your own email filled in and stops there. A

Why "usually right" is the ultimate trap for AI safety

[4:01]Good aentic engineer designs for failure, not just success. But don't just copy someone else's guardrails. Derive your own by asking three questions. Where can your retrieval be wrong or stale? Hedge the language there. Where is the action irreversible or high stakes? Put a human in front of it. And is there a category of input where refusing outright is safer than guessing? Build a failsafe branch for it. This case study answers all three the same way. Fail closed. Every downloaded data set is checked bite for bite against a pin checksum before it's trusted. If a file is missing or altered, that route refuses rather than guesses. On a national highway, the app refuses to name a maintaining authority itself and hands off to the official highway channel instead.

A contract match is only ever labeled probable because the actual warranty term isn't in the data. The app only infers it from how old the tender is. And nothing is ever sent automatically. The app doesn't call a complaint API and doesn't log in anywhere. It opens your own email filled in and stops. Here's the mistake that undoes all of this. Trusting the model to grade its own homework. It's tempting to skip the human review step because the model is usually right, but usually right is exactly the failure mode you're designing guardrails against. The one case in a 100 where it's confidently wrong is the one that reaches a real commissioner's inbox or a real bank account. If a guardrail only fires when you remember to check it,

It isn't a guardrail, it's a suggestion. So isolate the one genuinely unstructured piece and run the truth table test to find it. Choose a fixed pipeline over an autonomous loop wherever consequences are real. Derive your guardrails from three questions. What's stale? What's irreversible? What should refuse outright? And never let usually right replace a human check. That's the exact failure mode guardrails exist for. Your turn. Pick one AI feature you're building or one you use every day. Run the truth table test on what the model actually decides. Then ask whether it should run as a fixed pipeline or a looser agent loop given what's actually at stake. Then derive one guardrail from what's stale, what's irreversible, or what should refuse outright before you ship. Signing off. Divage power.