A customer writes a script with a tight loop and no sleep. Within seconds your API is serving ten thousand requests a minute from a single key, the database connection pool is exhausted, and every other customer sees timeouts. Nothing was hacked and nothing was broken. You simply had no policy for how much one caller is allowed to take.

Rate limiting is that policy. It protects shared resources, enforces fairness between tenants, and puts a ceiling on the cost of a misbehaving client. The hard part is not deciding to have one; it is choosing an algorithm whose behavior matches what you actually want to allow. Burst or no burst? Smooth output or smooth input? Exact counts or cheap approximations? This article walks through the four algorithms you will meet in practice, with code you can run.

What a Rate Limiter Must Decide

Every limiter answers one question for each incoming request: given who is calling and what they did recently, should this request pass? Three design choices shape the answer.

  • The key. Per API key, per user, per IP address, per route, or a combination. IP addresses are weak identifiers behind shared NATs and proxies; authenticated identities are better when you have them.
  • The limit and the window. “100 requests per minute” sounds precise, but the algorithms below interpret it very differently around window edges.
  • The response. Reject with HTTP 429 and a Retry-After header, queue the request, or degrade the response. Rejecting is the common default.

Fixed Window Counter

The simplest approach: divide time into fixed windows (for example, each minute), keep a counter per key per window, and reject when the counter exceeds the limit.

class FixedWindowLimiter {
  private counters = new Map<string, { windowStart: number; count: number }>();

  constructor(
    private readonly limit: number,
    private readonly windowMs: number,
  ) {}

  allow(key: string, now = Date.now()): boolean {
    const windowStart = Math.floor(now / this.windowMs) * this.windowMs;
    const entry = this.counters.get(key);

    if (!entry || entry.windowStart !== windowStart) {
      this.counters.set(key, { windowStart, count: 1 });
      return true;
    }
    if (entry.count >= this.limit) return false;
    entry.count += 1;
    return true;
  }
}

It uses one integer per key and is trivial to implement in Redis with INCR and EXPIRE. The flaw is the boundary burst. With a limit of 100 per minute, a client can send 100 requests at 12:00:59 and another 100 at 12:01:00. Each window is within its limit, yet the backend saw 200 requests in two seconds. If the limit exists to protect a fragile dependency, a fixed window can allow up to twice the intended rate.

Use it when approximate protection is fine and simplicity matters: coarse daily quotas, internal tools, billing-style counters.

Sliding Window Log

To remove the boundary problem, store the timestamp of every accepted request and, on each new request, count how many fall within the last windowMs.

class SlidingWindowLogLimiter {
  private logs = new Map<string, number[]>();

  constructor(
    private readonly limit: number,
    private readonly windowMs: number,
  ) {}

  allow(key: string, now = Date.now()): boolean {
    const cutoff = now - this.windowMs;
    const log = (this.logs.get(key) ?? []).filter((t) => t > cutoff);

    if (log.length >= this.limit) {
      this.logs.set(key, log);
      return false;
    }
    log.push(now);
    this.logs.set(key, log);
    return true;
  }
}

This is exact: at any instant, no more than limit requests exist in the trailing window. The cost is memory proportional to the limit for every active key, and in Redis, a sorted set per key with ZREMRANGEBYSCORE and ZCARD on every call. For a limit of 10 per minute that is fine. For a limit of 10,000 per minute across many keys, it is expensive.

Sliding Window Counter

A well-known compromise keeps two fixed-window counters, the current one and the previous one, and weights the previous by how much of it still overlaps the sliding window.

class SlidingWindowCounterLimiter {
  private windows = new Map<string, Map<number, number>>();

  constructor(
    private readonly limit: number,
    private readonly windowMs: number,
  ) {}

  allow(key: string, now = Date.now()): boolean {
    const current = Math.floor(now / this.windowMs);
    const elapsedFraction = (now % this.windowMs) / this.windowMs;

    const counts = this.windows.get(key) ?? new Map<number, number>();
    const prevCount = counts.get(current - 1) ?? 0;
    const currCount = counts.get(current) ?? 0;

    const estimated = prevCount * (1 - elapsedFraction) + currCount;
    if (estimated >= this.limit) return false;

    counts.set(current, currCount + 1);
    // keep only the two windows we need
    for (const w of counts.keys()) if (w < current - 1) counts.delete(w);
    this.windows.set(key, counts);
    return true;
  }
}

Two integers per key, no boundary doubling. The estimate assumes requests in the previous window were evenly spread, so it is an approximation, but it errs in a bounded and usually acceptable way. This is a strong default for public APIs where you want “about N per minute” without burst loopholes and without paying for a full log.

Token Bucket

The token bucket flips the model. Instead of counting what happened, it tracks a budget. A bucket holds up to capacity tokens. Tokens are added at a steady refillPerSecond rate. Each request consumes one token; if the bucket is empty, the request is rejected.

class TokenBucket {
  private tokens: number;
  private lastRefill: number;

  constructor(
    private readonly capacity: number,
    private readonly refillPerSecond: number,
    now = Date.now(),
  ) {
    this.tokens = capacity;
    this.lastRefill = now;
  }

  tryConsume(cost = 1, now = Date.now()): boolean {
    const elapsedSec = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(this.capacity, this.tokens + elapsedSec * this.refillPerSecond);
    this.lastRefill = now;

    if (this.tokens < cost) return false;
    this.tokens -= cost;
    return true;
  }
}

// Sustained 5 requests/second, bursts up to 20
const bucket = new TokenBucket(20, 5);

Notice there is no timer. Refill is computed lazily from elapsed time on each call, so an idle key costs nothing. Two parameters give you two independent knobs: refillPerSecond is the long-run average rate, capacity is how large a burst you tolerate. That separation is why token bucket is the most widely deployed algorithm; it matches how real clients behave, quiet for a while and then a flurry of calls when a page loads.

It also supports variable cost naturally. An expensive search endpoint can consume 5 tokens while a cheap read consumes 1, all from the same budget.

Leaky Bucket

The leaky bucket is the mirror image, aimed at smoothing output. Requests enter a queue of fixed size; the queue drains at a constant rate. If the queue is full, new requests are dropped.

class LeakyBucketQueue<T> {
  private queue: T[] = [];
  private timer: ReturnType<typeof setInterval>;

  constructor(
    private readonly maxQueue: number,
    drainPerSecond: number,
    private readonly handler: (item: T) => void,
  ) {
    this.timer = setInterval(() => {
      const next = this.queue.shift();
      if (next !== undefined) this.handler(next);
    }, 1000 / drainPerSecond);
  }

  offer(item: T): boolean {
    if (this.queue.length >= this.maxQueue) return false;
    this.queue.push(item);
    return true;
  }

  stop(): void {
    clearInterval(this.timer);
  }
}

The key difference from the token bucket: a token bucket lets a burst through immediately; a leaky bucket never emits faster than the drain rate. Downstream systems see a perfectly even flow. That is exactly what you want when calling a third-party API with a strict throughput contract, or feeding a worker that cannot absorb spikes. The trade-off is latency: queued requests wait, so it suits background work better than interactive endpoints.

A quick mental model: the token bucket polices the input (how fast callers may arrive), the leaky bucket shapes the output (how fast work leaves).

Doing It Across Servers With Redis

In-memory limiters are per-process. Behind three replicas, a limit of 100 per minute silently becomes 300. For a shared limit, the state must live in a shared store, and the read-modify-write must be atomic. Doing GET then SET from the application loses updates under concurrency. A Lua script runs atomically inside Redis:

import Redis from "ioredis";

const redis = new Redis();

const TOKEN_BUCKET_LUA = `
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refill = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])

local data = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(data[1]) or capacity
local ts = tonumber(data[2]) or now

tokens = math.min(capacity, tokens + (now - ts) / 1000 * refill)
local allowed = 0
if tokens >= cost then
  tokens = tokens - cost
  allowed = 1
end

redis.call('HSET', key, 'tokens', tokens, 'ts', now)
redis.call('PEXPIRE', key, math.ceil(capacity / refill * 1000) * 2)
return { allowed, math.floor(tokens) }
`;

export async function allow(key: string, capacity: number, refillPerSec: number, cost = 1) {
  const [allowed, remaining] = (await redis.eval(
    TOKEN_BUCKET_LUA, 1, `rl:${key}`, capacity, refillPerSec, Date.now(), cost,
  )) as [number, number];
  return { allowed: allowed === 1, remaining };
}

Two details matter. The PEXPIRE lets idle keys disappear so the keyspace does not grow forever. And the timestamp is passed in from the application, which means clock skew between app servers can distort refill; if that worries you, call redis.call('TIME') inside the script instead.

Python follows the same shape. With redis-py you register the script once and call it:

import time
import redis

r = redis.Redis()
script = r.register_script(TOKEN_BUCKET_LUA)  # same Lua as above

def allow(key: str, capacity: int, refill_per_sec: float, cost: int = 1) -> bool:
    allowed, _ = script(keys=[f"rl:{key}"], args=[capacity, refill_per_sec, int(time.time() * 1000), cost])
    return allowed == 1

Telling Clients What Happened

A limiter that rejects silently produces confused clients and retry storms. Return 429 Too Many Requests with a Retry-After header giving the seconds to wait. Many APIs also send RateLimit-* or X-RateLimit-* headers exposing the limit, remaining budget, and reset time. The exact header names vary between providers, so pick one convention and document it.

import type { Request, Response, NextFunction } from "express";

export async function rateLimit(req: Request, res: Response, next: NextFunction) {
  const key = req.header("x-api-key") ?? req.ip ?? "anonymous";
  const { allowed, remaining } = await allow(key, 20, 5);

  res.setHeader("X-RateLimit-Remaining", String(remaining));
  if (!allowed) {
    res.setHeader("Retry-After", "1");
    return res.status(429).json({ error: "rate_limited" });
  }
  next();
}

Well-behaved clients should answer a 429 by waiting rather than hammering again: exponential backoff with jitter, honoring Retry-After when the server sends it.

Choosing an Algorithm

NeedGood fit
Simple quota, approximate is fineFixed window
Exact limit, small volumeSliding window log
Public API, cheap and fairly accurateSliding window counter
Allow bursts, cap average rateToken bucket
Perfectly smooth downstream flowLeaky bucket

Common Mistakes

  • Limiting only by IP. Offices, schools and mobile carriers share addresses. Prefer authenticated identity, and keep an IP limit as a coarse outer layer.
  • Check-then-set races. Non-atomic read-modify-write across servers lets bursts slip through. Use Lua scripts or atomic commands.
  • Failing closed by accident. If Redis is down, decide deliberately: allow traffic (fail open) or reject it. For most public APIs, failing open with an alert beats taking the whole API offline.
  • One global limit for everything. A cheap health check and an expensive export should not share a budget. Use per-route costs or separate buckets.
  • Ignoring distributed retries. If every rejected client retries at the same instant, the limiter becomes a synchronized spike generator.

When Not to Use It

A rate limiter limits arrival rate, not concurrency or load. A single slow request can still saturate a worker even at one request per second. For that, pair rate limits with concurrency limits, timeouts and queue bounds. And if all your callers are internal and trusted, load shedding and backpressure may serve you better than per-caller quotas.

Key Takeaways

  • Fixed windows are cheap but allow up to double the limit at window boundaries.
  • Sliding window counters give a good accuracy-to-cost ratio for most public APIs.
  • Token bucket separates average rate from burst size; leaky bucket enforces a smooth output rate at the price of queueing latency.
  • Shared limits across servers need atomic operations, typically a Redis Lua script.
  • Always return 429 with Retry-After, and decide explicitly whether the limiter fails open or closed.