Redis Rate Limiting and Distributed Locks
On this page
Locks and rate limits need shared state
Ten workers share a queue, and a nightly report must be generated once, not ten times. Separately, your API must allow each user at most 100 requests per minute, across twenty API servers. Both need shared state that every server sees and updates atomically. Redis is a common place for it, and both are easy to get almost right.
Why SET NX plus DEL is not a safe lock
The first attempt: SET lock:report 1 NX EX 30 to take the lock, DEL lock:report to release it. Here's the failure:
- Worker A takes the lock with a 30 second expiry.
- A pauses for 35 seconds: a long garbage collection, a slow query.
- The lock expires. Worker B takes it.
- A wakes up, finishes, and runs
DEL lock:report, deleting B's lock. - Worker C takes the lock while B is still running.
The fix has two parts. Store a unique token per attempt, and release only if the lock still holds your token. The check and the delete must happen as one step, so they go in a Lua script, which Redis runs without interleaving other commands.
A safe Redis lock and a sliding-window rate limiter
# limits.py: a safe lock release and a sliding-window rate limiter, both as Lua scripts
import time, uuid
import redis
r = redis.Redis(decode_responses=True)
RELEASE = r.register_script("""
if redis.call('GET', KEYS[1]) == ARGV[1] then
return redis.call('DEL', KEYS[1])
end
return 0
""")
SLIDING_WINDOW = r.register_script("""
-- KEYS[1] limiter key; ARGV: now_ms, window_ms, limit, unique request id
local now, window, limit = tonumber(ARGV[1]), tonumber(ARGV[2]), tonumber(ARGV[3])
redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window)
if redis.call('ZCARD', KEYS[1]) < limit then
redis.call('ZADD', KEYS[1], now, ARGV[4])
redis.call('PEXPIRE', KEYS[1], window)
return 1
end
return 0
""")
def acquire(name: str, ttl_ms: int = 10_000) -> str | None:
token = str(uuid.uuid4()) # unique per attempt, so we only release our own lock
return token if r.set(f"lock:{name}", token, nx=True, px=ttl_ms) else None
def release(name: str, token: str) -> bool:
return RELEASE(keys=[f"lock:{name}"], args=[token]) == 1
def allow(user_id: str, limit: int = 100, window_ms: int = 60_000) -> bool:
now = int(time.time() * 1000)
return SLIDING_WINDOW(keys=[f"rate:{user_id}"], args=[now, window_ms, limit, str(uuid.uuid4())]) == 1A quick test against a local Redis:
A got lock: True | B got lock: False
B release with wrong token: False | A release: True
allowed: 5 of 8 (limit 5 per second, 8 rapid requests)The rate limiter keeps one sorted set per user, with a timestamp as each member's score (Day 3). Each call drops entries older than the window, counts the rest, and adds the new request only if there's room. Because it all runs in one script, two servers can't both see 99 and both let a request through. The member is a unique request id rather than the timestamp, so two requests in the same millisecond both count.
A sliding window is exact but stores one entry per request. For high limits, a fixed-window counter (INCR plus EXPIRE on a key per minute) uses far less memory, at the cost of allowing a burst at window edges.
If a job can run longer than the lock's TTL, don't just pick a huge TTL, which leaves the lock stuck for that long when a worker dies. Renew it instead: every few seconds, the holder runs a script that extends the TTL only if the key still holds its token, the same compare-then-act shape as RELEASE:
RENEW = r.register_script("""
if redis.call('GET', KEYS[1]) == ARGV[1] then
return redis.call('PEXPIRE', KEYS[1], ARGV[2])
end
return 0
""")
# called every ttl_ms / 3 while the job runs
if RENEW(keys=[f"lock:{name}"], args=[token, ttl_ms]) != 1:
raise RuntimeError("lost the lock, stop working") # someone else holds it nowA failed renewal is a signal, not an error to retry: the lock expired and may already belong to another worker, so stop.
What a Redis lock can't guarantee: fencing tokens and Redlock
Even with tokens, step 2 of the failure story can still happen: a paused worker may keep working after its lock expired, while another worker holds it. The token stops the paused worker from deleting the new lock. It doesn't stop the paused worker from also writing.
So treat Redis locks as efficiency locks: they prevent duplicate work most of the time, and an occasional duplicate is harmless, like generating a report twice. If a duplicate would corrupt data, such as charging a card twice, you need either a consensus system like etcd or ZooKeeper, or a fencing token: an increasing number issued with each lock that the storage system checks, rejecting writes from older holders.
Here's how that works in practice. Take the lock and a number from a counter in one script, so the numbers follow the order in which the lock was granted:
-- KEYS: lock:{report}, fence:{report}; ARGV: token, ttl_ms
if redis.call('SET', KEYS[1], ARGV[1], 'NX', 'PX', ARGV[2]) then
return redis.call('INCR', KEYS[2]) -- the fencing token: 41, 42, 43...
end
return 0The worker passes its number with every write, and the database rejects anything older than what it has already seen:
UPDATE reports SET body = @body, fence = @token
WHERE id = @id AND fence < @token; -- 0 rows: a newer lock holder already wroteTested: worker B wrote with token 42, then the paused worker A woke up and wrote with 41, and A's update changed nothing. The check has to live in the system being protected, because that's the one place both workers' writes meet. The hash tag {report} keeps both keys in one slot, so the script also works on Redis Cluster (Day 8).
Redlock, an algorithm that takes the lock on a majority of several independent Redis nodes, survives one node failing. It doesn't solve the pause problem. Martin Kleppmann's 2016 post "How to do distributed locking" and Salvatore Sanfilippo's reply "Is Redlock safe?" lay out both sides and are worth reading before relying on it.
Check yourself
With a separate GET and DEL, the lock can expire and be taken by another client between the two commands, and you'd delete their lock. The script runs as one uninterruptible step.
No. A paused process can act after its lock expired. Use an idempotency key in the payment system, a fencing token, or a consensus-based lock.
Switch from a sorted-set sliding window, which stores every request, to a fixed-window counter or a sliding-window approximation built from two counters.
The client sends 100 requests at the end of one minute and 100 at the start of the next, and each window sees exactly 100. Sliding-window approaches, either a sorted set of timestamps or an approximation that weights the previous window's count, smooth out this burst at the boundary.
If the process holding the lock crashes, nobody would ever release it, and every other process would wait forever. The expiry makes the lock release itself eventually. The trade-off is that a slow but alive holder can lose the lock while still working, which is why fencing tokens or idempotency are needed for critical work.