Design AI Productivity Metrics That Survive Goodhart’s Law

Suppose leadership rewards teams for increasing the percentage of “AI-assisted pull requests.” The dashboard rises. Did productivity improve, or did people learn which box to tick? Before launching that metric, I would run a consequence-mapping session: Intended behavior Plausible adaptation Counter-metric try useful assistance label trivial PRs as assisted retained task outcome ship faster split work into tiny PRs lead time per task share adoption avoid difficult non-AI work task-mix distribution accept suggestions reduce review scrutiny rollback and defect rate The metric card should make disagreement possible: name : ai_assisted_pr_share purpose : detect workflow adoption, not productivity owner : developer-experience known_game : self-label inflation counter_metrics : [ task_mix , review_minutes , rollback_rate ] review_date : 2026-08-19 retire_when : classification cannot be audited Then interview both high and low scorers without treating the score as performance. Ask what work disappeared, what new verification appeared, and what behavior the dashboard encouraged. Include an anonymous channel: a metric cannot reveal pressure if challenging it carries career risk. The SPACE framework argues that developer productivity cannot be captured by one dimension. That is especially relevant when AI telemetry is easy to count but verification and rework are harder to observe. My launch gate is not “the metric is accurate.” It is: teams can inspect its definition, challenge its interpretation, and show where it changes behavior. If the counter-metrics diverge, pause incentives before refining the chart. What behavior would your current AI dashboard accidentally reward?