The Model Passed Your Benchmark. Now Stop Merging Its Code Blindly

A few weeks ago I wrote about building a reproducible test harness for comparing free AI coding models before you commit . That harness answers one question: which model should I use? It does not answer the harder follow-up: once a model generates a patch for my real codebase, when is it safe to merge? This week there was a great discussion on DEV about "understanding over origin" — the idea that it doesn't matter whether code came from a human or a model, only whether someone actually understands it. I agree with the principle, but principles don't survive contact with a busy afternoon. What survives is a checklist with teeth. So here is the pipeline I bolted onto my model harness: every AI-generated patch has to pass through a scripted review gate before I even read it, and the script produces a scorecard that tells me how carefully I need to read it. The problem with eyeballing diffs When a model produces a 40-line diff that looks idiomatic, my brain does a dangerous thing: it pattern-matches on style and skips semantics. The code reads like something I'd write, so I approve it like something I'd write. The failures I've actually shipped from AI-generated code were never syntax errors — the tests even passed. They were things like: A retry loop that retried on the wrong exception type, so real errors got swallowed. A query filter that was subtly wider than the one it replaced (tests passed because fixtures were too small to notice). A dependency added for a one-liner the standard library already covers. All three would have been caught by asking four boring questions before reading the code. So I scripted the questions. The review gate: a reproducible artifact The gate is a small shell script. It takes a patch file, applies it to a throwaway worktree, and runs four checks. It never touches my working branch, and it prints a one-line verdict at the end. #!/usr/bin/env bash # review-gate.sh <patch-file> <base-branch> set -euo pipefail PATCH = " $1 " BASE = " ${ 2 :