Token Bucket vs. the rest: choosing a rate-limiting algorithm
Halt ships four algorithms because no single one fits every endpoint. Here is how they actually behave under traffic, and a simple decision guide for picking per route.
Choosing a rate-limiting algorithm
"100 requests per minute" sounds like one rule, but there are at least four meaningfully different ways to enforce it and they behave very differently when traffic gets spiky. Halt implements all four so you can pick per policy instead of bending one algorithm to every job.
Fixed Window: the simple baseline
Count requests in a fixed interval (12:00:00–12:00:59), reset to zero when the next interval starts.
- Pros: trivially cheap one counter and one timestamp per key, O(1) everywhere.
- Cons: the boundary problem. A client can send 100 requests at 12:00:59 and 100 more at 12:01:00 200 requests in two seconds, all "within the limit."
Use it for internal services and admin endpoints where the worst case of a boundary burst is acceptable and you want the lowest possible overhead.
iSliding Window: fixing the boundary
Instead of hard resets, the window slides with the clock: the count at any instant covers the previous 60 seconds, weighted across the boundary. The 200-requests-in-two-seconds trick stops working.
- Pros: the most accurate enforcement of "N per minute" as humans understand it.
- Cons: slightly more state and arithmetic per check than a fixed window.
Use it for public APIs where fairness between clients matters and you've been bitten by boundary bursts before.
Token Bucket: bursts are a feature
Each key owns a bucket of tokens. Every request spends one; tokens refill at a constant rate. A full bucket lets a client burst, then the refill rate caps their sustained throughput.
- Pros: matches real client behavior apps naturally fire a handful of calls on page load, then go quiet. Steady average, friendly bursts.
- Cons: "limit" is now two numbers (capacity + refill rate), which takes a moment to reason about.
This is Halt's recommended default for user-facing APIs, and it's what presets like presets.PUBLIC_API build on.
fLeaky Bucket: smoothing, not just limiting
Requests enter a queue (the bucket) and drain at a constant rate. Where Token Bucket lets bursts through, Leaky Bucket flattens them into a steady drip.
- Pros: the output rate is constant no matter how spiky the input ideal when a fragile downstream (a legacy system, a paid third-party API) must never see a burst.
- Cons: under sustained load, requests queue and latency grows before requests are rejected.
Use it for traffic shaping in front of systems that care about rate more than count.
A decision guide
| Endpoint | Pick | Why |
|---|---|---|
| Public REST API | Token Bucket | Bursty clients, steady average |
| Login / OTP | Sliding Window | Strict, accurate, no boundary tricks |
| Internal microservice | Fixed Window | Cheapest check, boundary risk acceptable |
| Proxy to a fragile downstream | Leaky Bucket | Constant output rate |
Because Halt policies are per-limiter (and can be overridden per request), mixing algorithms in one service is normal: Token Bucket on /api/*, Sliding Window on /auth/*, done.
Try them side by side
Every algorithm runs on every store, so you can benchmark with InMemoryStore locally and switch the backend later without touching policy code.
nRead the full algorithms reference, or start from a runnable setup in the examples.