今日已更新 165 条资讯 | 累计 27340 条内容
关于我们

How Do You Contain an AI Agent Failure You Can't Prevent?

Sara Mo 2026年07月26日 11:16 7 次阅读 来源:Dev.to

Every part of this series has quietly agreed on one thing: the agent will be wrong sometimes. Part 1 set the bar at "acceptably wrong." Part 3 measured how often. So the last question is not how to stop it from ever failing. It is the one that actually decides whether you can ship: when it is wrong, what is the worst that can happen? That worst case is not fixed. It is a design choice, and it is the one most teams never make on purpose. Blast radius is something you choose Two agents give the same wrong answer. One drafted an email for a human to send. The other sent it. One suggested a refund. The other issued it. Identical mistake, completely different consequence, because someone decided how much power the agent had when it was wrong. You set the blast radius by choosing what the agent is allowed to do, not by hoping it does the right thing. Guardrails: match capability to proven trust Give an agent the least authority the job allows. Let it read before it writes, propose before it executes. An action more dangerous than the agent's measured reliability has earned is a liability you chose. If Part 3 told you a step is right eighty percent of the time, that step does not get to move money unsupervised. Capability should track trust, and trust is a number you now have. Put a human on the expensive failures, and only those Human-in-the-loop is not "approve everything," which kills the speed that made an agent worth building. It is a gate on the small set of actions where a wrong one is irreversible or costly: the disqualifying failures you named in Part 1 (the known abuse modes are catalogued in the OWASP LLM Top 10 ). Everything reversible and cheap runs on its own. Everything that cannot be taken back waits for a person. Make failures reversible and visible Prefer actions you can undo, and log enough to undo them. A dry-run mode, a soft delete, a confirmation step: these turn an incident back into a mistake. And you cannot contain what you cannot see, so trace eve

本文内容来源于互联网,版权归原作者所有
查看原文