今日已更新 143 条资讯 | 累计 40588 条内容
关于我们

When an AI Agent Makes a Mistake in Production, Which Layer Should Stop It?

Hossein Hezami 2026年09月05日 23:28 1 次阅读 来源:Dev.to

A familiar production failure looks like this: an AI support agent reads a ticket, decides the customer deserves compensation, calls the refund tool, and refunds the full annual subscription instead of the $12 add-on. The model did not crash. The API did not throw an exception. The tool worked exactly as designed. The postmortem usually starts with the wrong question: “How do we stop the model from making bad decisions?” The better question is: which layer should have stopped the mistake before it became damage? AI agents fail in many different ways. They misunderstand intent. They create dangerous plans. They pass malformed arguments. They exceed permissions. They loop. They leak data. They take irreversible actions. Each failure mode belongs to a different layer, and each layer has a different job. If your only defense is a prompt that says, “Be careful,” you do not have a safety architecture. You have a hope. TL;DR: AI agent mistakes should not be stopped by the model alone. Use layered defense: intent classification stops wrong missions, plan validation stops forbidden sequences, tool schemas stop invalid arguments, authorization stops unauthorized actions, execution controls limit blast radius, output validation catches harmful results, runtime monitors stop loops, and human approval guards asymmetric risk. The best stopping layer is the earliest deterministic layer that can prevent harm, with the final brake closest to irreversible side effects. 📋 Table of Contents The Mistake Is Not One Failure Mode 1. The Prompt Layer Should Persuade, Not Enforce 2. The Intent Layer Should Catch the Wrong Mission 3. The Planning Layer Should Reject Forbidden Paths 4. The Tool Contract Layer Should Make Invalid Actions Unrepresentable 5. The Authorization Layer Should Veto Even Correct-Looking Actions 6. The Execution Layer Should Make Side Effects Boring 7. The Output Layer Should Catch Harmful Results Before They Ship 8. The Runtime Monitor Should Stop Slow-Motion Failures

本文内容来源于互联网,版权归原作者所有
查看原文