My LLM drift tracker flagged four regressions this week. All four were wrong.

I run a public board that probes 16 LLMs on a frozen 35-task suite, once a day, and keeps every score. When a model drops against its previous run, it opens a GitHub issue by itself and writes me a draft post. Between 21 and 24 July it did that four times: 23 Jul Gemini 3.5 Flash -11.4 pts 24 Jul Gemini 3.1 Pro -2.9 pts 21 Jul Grok 4.3 -5.7 pts 22 Jul Llama 3.3 70B -2.9 pts Four regressions in four days, across three labs. That's a post that writes itself, and it would have been fast, legible, and wrong. None of those models got worse. Here's how I know, because the how is the only part worth reading. Two of them weren't the model Every point on the board carries a second number next to accuracy: reliability , the share of probe calls that actually came back. Look at the two Google alerts with that column showing: gemini-3.5-flash 22 Jul acc 1.000 reliability 1.000 23 Jul acc 0.886 reliability 0.914 <- "-11.4 pts" gemini-3.1-pro 22 Jul acc 0.914 reliability 0.943 23 Jul acc 0.886 reliability 0.914 <- "-2.9 pts" 24 Jul acc 0.971 reliability 1.000 <- next clean run Accuracy and reliability fell together. That's the signature of calls that never returned, not answers that got worse — a failed call has no answer to grade, and an ungraded task scores the same as a wrong one. I know this signature well because this board already published the lesson. On 20 July, Llama 3.3 70B appeared to fall 66 points overnight: api.groq.com -> 429: Rate limit reached for model ` llama-3.3-70b-versatile ` service tier `on_demand` ... requests per minute (RPM): Limit 30, Used 30 34 of 35 calls were rate-limited. The model didn't get dumber; a 429 scored as a zero. A rate limit scoring as a 0% is the single most misleading thing a drift tracker can do, because it looks exactly like the thing the tracker exists to catch. Gemini 3.1 Pro settles its own case: the next clean run came back at 97.1%, higher than before the "regression." The other two were one question The remaining two alerts ar