Token Bucket vs. Sliding Window: Building Rate Limiters That Actually Hold Under Load

Rate limiting sounds like a solved problem until you actually implement one and watch it fail in a way your load test didn't predict: legitimate bursts getting rejected, or a limiter that lets through 2x its stated limit at window boundaries. The failure modes are specific enough that it's worth working through the two dominant algorithms — token bucket and sliding window — with actual code, not just the diagrams. The problem with fixed windows The naive approach almost everyone reaches for first is a fixed window counter: pick a window size (say, 60 seconds), count requests in that window, reset the counter when the window rolls over. import time class FixedWindowLimiter : def __init__ ( self , limit : int , window_seconds : int ): self . limit = limit self . window_seconds = window_seconds self . count = 0 self . window_start = time . time () def allow ( self ) -> bool : now = time . time () if now - self . window_start >= self . window_seconds : self . window_start = now self . count = 0 if self . count < self . limit : self . count += 1 return True return False This is simple and cheap, and it's also broken in a specific, exploitable way. Say the limit is 100 requests/minute. A client can send 100 requests in the last second of window N, then another 100 in the first second of window N+1. That's 200 requests in roughly two seconds, well within the letter of "100/minute" as the code enforces it, but nowhere near the spirit of it. This is the classic boundary-burst problem, and it's the reason fixed windows get replaced once traffic is adversarial or bursty enough to find the seam. Sliding window: smoothing the boundary A sliding window log fixes this by tracking actual timestamps instead of a single counter, and counting how many fall within the trailing window at the moment of the request: from collections import deque import time class SlidingWindowLogLimiter : def __init__ ( self , limit : int , window_seconds : float ): self . limit = limit self . window_seco