Two Kubernetes Decisions Nobody Writes About Honestly

1. Node Group Sizing Fewer large nodes vs. many small ones. Textbooks don't cover this. We ran 10 nodes, 32 CPU each. Seemed efficient. Problem: One node dies, 320 CPU worth of workloads need to reschedule. Cluster autoscaler couldn't handle it. Pods sat pending for 10 minutes. We switched to 20 nodes, 16 CPU each. Same total capacity. One node dies now? 160 CPU to reschedule. Autoscaler catches it in 90 seconds. Scheduling is tighter, but failures are isolated. Cost stayed the same. Blast radius halved. Why nobody writes about this: The tradeoff isn't obvious. Large nodes are "more efficient." Smaller nodes are "more resilient." Both are true. It depends on whether you'd rather have one big problem or many small ones. We picked smaller nodes because a node failure was our actual failure mode. Not resource efficiency. 2. Readiness vs Liveness Probes Misconfigure these and your cluster looks like it's melting. Readiness probe: "Can this pod take traffic?" Liveness probe: "Is this pod alive? Restart it if not." One team set readiness = liveness. Same probe checked both. Probe logic: "If I can reach the database, I'm ready." Database gets slow. Probe fails. Pod becomes "not ready." Load balancer removes it from rotation (correct). But liveness also failed. Kubernetes killed the pod and restarted it. New pod starts. Probe fails immediately (database still slow). Gets killed. Restarted. This cascaded across 30 pods. 30 restarts/minute. New pods spent 100% of time restarting. Looks like an application bug for the first 20 minutes. Actually a probe configuration bug. Fix: Readiness checks "can I take traffic right now?" Liveness checks "am I fundamentally broken?" Use separate probes. Readiness: Check database connection with short timeout. Fail if slow. This is reasonable - don't send traffic to slow pods. Liveness: Check if the process is responding at all. Much stricter threshold. Only kill if truly hung. Same database slowness? Pods become unready. Traffic reroutes. Cl