What a Kubernetes controller actually does when you break something

⚡ TL;DR Four things about controller mechanics are widely half-understood: what Reconcile receives, where its work comes from, what a periodic resync is, and what a predicate turns off. I built an operator, broke it five ways, and measured each mechanism directly. The reconcile function runs in 2.71ms mean, 77/77 under 25ms , a short resync period costs zero additional API requests , and GenerationChangedPredicate cut steady-state reconciles by 48.5% without touching live repair at all. That last combination is the one that matters at scale. Repo, raw data, and harness: kirPoNik/k8s-drift-operator . 🧩 The four barriers Everyone who runs Kubernetes knows the platform repairs itself. Delete a pod, it comes back. Scale a Deployment by accident, something puts it back. Almost nobody who relies on that property can say how it works, and the gaps are specific and consequential. I keep meeting the same four: People think a controller is told what changed. It is not, and the reason it is not is the single most important design decision in Kubernetes. People think a controller polls the API server. It does not, and knowing what it does instead tells you where your API load actually comes from. People think a resync is a re-check against the cluster. It is not, which is why a short resync period is nearly free — and why the number that is expensive sits somewhere else entirely. People treat a predicate as a pure optimisation. It is a filter with a silent cost, and the cost is not the one the documentation warns you about first. So I built the smallest system that has the self-healing property, broke it on purpose ten times per failure mode, and instrumented each of those four mechanisms until I could state what it does rather than what it is said to do. What I built. One CRD called Echo , holding an image, a replica count, and a greeting. A controller keeps three child objects in sync with it — a Deployment, a Service, and a ConfigMap holding the greeting — with owner referen