Too Many Req: A Bucket List Guide to Building a Rate Limiter

Hello, I'm Maneshwar. I'm building git-lrc, a Micro AI code reviewer that runs on every commit. It is free and source-available on Github. Star git-lrc to help devs discover the project. Do give it a try and share your feedback. Every serious API will eventually tell you to sit down and be quiet. Hammer GitHub, Stripe, or AWS a little too eagerly and your requests start bouncing back with a polite but firm 429 . I always found that fascinating, so let's build the thing that says no. By the end of this post we'll have designed a rate limiter that actually holds up when you put it in front of real traffic, and I promise to only make a reasonable number of bucket puns along the way. A rate limiter does one job: it decides how many requests a client is allowed to make in a given window of time. It protects your system from getting flattened, and it keeps one greedy user from eating everyone else's lunch. Simple idea. Surprisingly spicy implementation. Let's build it up piece by piece, the way you'd actually reason through it in an interview or a design doc. First, what are we even building? Before writing a single line, let's agree on what "good" looks like. Here's my wishlist: Configurable limits. Something like "100 requests per minute per user." The rules should not be hardcoded, because free users and premium users deserve different amounts of pain. Honest rejections. When someone goes over, we return HTTP 429 Too Many Requests and include helpful headers telling them how many requests they have left and when the window resets. No mystery. Barely-there latency. This check runs on every single request , so it has to be fast. Let's aim for under 3ms at P95. If your rate limiter is slow, congratulations, you built a second bottleneck. Highly available and shared. Multiple servers need to agree on the same counts. More on why that word "shared" is doing a lot of heavy lifting later. Cool. Now let's start naive and let reality punch us in the face a few times. Attempt 1: