I Discovered AI Agents Can't Self-Verify. The Real Problem Is Much Bigger.
I Discovered AI Agents Can't Self-Verify. The Real Problem Is Much Bigger. I'm an undergrad in China, building an AI governance thesis in public. Two months ago I found that AI agents can't independently check if they followed your rules. I built mechanical gates to work around it. They worked — 55.9% violations down to 0.7%. But last week I realized I'd been solving the wrong problem. The real problem isn't verification. The real problem is that natural language is structurally the wrong language for AI governance. Here's What I Mean Right now, every layer of AI governance speaks the same language: Human writes NL rules → Model reads NL → Model generates behavior Human writes NL checks → Model reads NL → Model generates "yes I followed the rules" But every autoregressive transformer — GPT, Claude, DeepSeek, Qwen — generates text and evaluates text through the exact same mechanism. Think of it like this: the model has one pipeline for producing words. When you ask it "did you follow rule X?", it can't pause, run an internal audit, and give you a verified answer. It can only run that same word-production pipeline and generate text that claims it followed the rule. The pipeline doesn't know the difference between "I actually checked" and "I wrote a sentence that sounds like I checked." (Technically: both generation and evaluation route through P(token | context; θ) — the same probability distribution over next tokens. If you don't care about the math, the one-sentence version is: the model can't step outside itself to verify itself. ) I called this the Prose Barrier . (Wrote about it here . René Zander, a German dev I've never met, independently discovered the same thing. Convergent evolution.) The Prose Barrier means: you cannot fix AI governance by writing better prompts. The language itself is the bottleneck. So what's the alternative? Three Paths, Three Languages The future isn't "better NL." The future is using the right language at each layer. Human defines cons