Random Forest Is Horizontal Scaling for Predictions
Classic Machine Learning Through the Eyes of an SRE — Part 3 The random forest is the first ML algorithm that made me feel at home. Not because of the math — because it's an SRE idea wearing a stats costume. Many independent workers. No single point of failure. Majority vote. If one worker goes weird, the fleet absorbs it. We've been building systems this way for decades; the forest just applies it to prediction. The problem it exists to fix Last article: a single decision tree is readable but unstable — small data change, whole tree flips, explanation rewrites itself. That instability is variance, and it's exactly what scared me about trusting one tree in production. The forest's move: grow hundreds of trees, each on a random resample of the data, and — this is the part that matters — force each split to choose from only a random subset of features. That second randomization is the whole difference between a random forest and plain bagging. Bagging alone gives you many trees on resampled data, but if one feature is strongly predictive, every tree grabs it first and they all end up looking alike. Starving each split of features is what makes the trees genuinely different from each other. The randomness isn't sloppiness. It's manufactured disagreement. The instability doesn't get fixed. It gets CANCELLED. Each tree is still jumpy, but they're jumpy in different directions, and the average is calm. What surprised me No new loss function. Each tree still minimizes impurity exactly like a lone tree. The forest adds zero new objectives. The entire gain is a bias-variance bargain: variance drops hard, bias barely moves. You give up readability and get back trustworthiness. Embarrassingly parallel. Trees are independent, so training scales horizontally — throw cores at it. Boosting, its sequential cousin, is the opposite: each model depends on the last. Map-reduce versus a pipeline. The smoothness illusion. A forest's decision boundary looks smooth, almost like regression'