Decision Trees Aren't Trained. They're Grown.

Classic Machine Learning Through the Eyes of an SRE — Part 2 The second algorithm I studied broke everything I'd just learned from the first. Logistic regression taught me that training means gradient descent: guess, measure error, adjust the weights, repeat until convergence. So when I opened decision trees, I went looking for the optimizer. There wasn't one. A decision tree isn't optimized the way I expected. It's grown. At each step it finds the locally best split, commits to it, and recursively repeats the process. No backtracking. No second chances. There is optimization happening — each split minimizes impurity — but only locally, one step at a time. Finding the globally optimal tree is NP-hard, so the algorithm doesn't even try. That felt surprisingly familiar. In incident response or capacity planning, we rarely know the perfect answer. We make the best decision with the information we have, knowing a different first choice might have led somewhere else. Decision trees simply turn that idea into an algorithm. The bet a tree makes Every machine learning algorithm makes a different bet about the world. Logistic regression assumes relationships are smooth. Risk gradually increases as signals change. Decision trees make the opposite assumption. They assume the world is made of boxes. A project isn't slightly riskier because velocity drops. It's risky when several conditions happen together: a fixed-price contract, a new account manager, and a month-end delivery. Inside that box, projects fail. Outside it, they're usually fine. This is exactly how many operational systems work. Severity matrices, routing rules, escalation policies, approval workflows — they're all collections of decision boxes. That's why trees immediately felt intuitive to me. The hidden cost of flexibility Trees make very few assumptions about the data. That sounds like an advantage. The price is instability. Change a small part of the training data and the first split can change. Since every l