Catalog My learning Blog
Redis Tutorial: Caching, Persistence and Scaling · Phase 3 · Lesson 12 · 5 min read

Redis Transactions, Pipelining and Lua Scripts

On this page
  1. Pipelining vs MULTI/EXEC vs Lua scripts
  2. Pipelining: fewer round trips, no atomicity
  3. Redis transactions: MULTI, EXEC and WATCH
  4. Lua scripts and Redis Functions
  5. Why a slow script blocks Redis
  6. Check yourself

Pipelining vs MULTI/EXEC vs Lua scripts

Moving points from one user to another needs three things: read the sender's balance, decide whether there's enough, then update both balances, with no other client changing anything in between. Redis has three features that sound like they might do this. Only one does it cleanly.

ToolSaves round tripsAtomicCan decide based on a value it read
PipeliningYesNoNo
MULTI / EXEC (with WATCH)PartlyYes, for the queued commandsNo, the app decides and retries
Lua script or FunctionYes, one callYes, the whole scriptYes

Pipelining: fewer round trips, no atomicity

Normally a client sends a command and waits for the reply before sending the next. With 1 ms of network latency, 10,000 commands take at least 10 seconds of waiting. A pipeline sends many commands in one go and reads all the replies together.

Code
with r.pipeline(transaction=False) as pipe:
    for i in range(10_000):
        pipe.set(f"k:{i}", i)
    pipe.execute()

Against a local Redis, 10,000 SETs took 1.00 second one by one and 0.05 seconds pipelined. Over a real network the gap is larger. But other clients' commands can still run between yours: pipelining is purely about latency.

Redis transactions: MULTI, EXEC and WATCH

Commands between MULTI and EXEC are queued and then run back to back, with no other client's command in between. Two things surprise people coming from SQL:

  1. No rollback. If one command fails at run time, the others still run. An error Redis catches while queuing, such as a wrong number of arguments, is different: EXEC then returns EXECABORT and runs nothing.
Redis
SET name sam
MULTI
SET a 1
INCR name          # will fail: "sam" isn't a number
SET b 2
EXEC
# 1) OK
# 2) ERR value is not an integer or out of range
# 3) OK            <- a and b were both set anyway
  1. No decisions inside. Every command is queued before any runs, so you can't read the balance and branch on it within the transaction.

WATCH adds optimistic locking. Watch the key, read it, decide in your app, then MULTI/EXEC. If anyone changed the watched key in the meantime, EXEC does nothing and returns null, and you retry:

Code
pipe = r.pipeline()
pipe.watch("user:42:points")
balance = int(pipe.get("user:42:points"))
# ... another client changes user:42:points here ...
pipe.multi()
pipe.decrby("user:42:points", 30)
pipe.incrby("user:99:points", 30)
pipe.execute()          # raises redis.WatchError: nothing applied, retry from watch()

It works, but under contention on a busy key you retry repeatedly, and every attempt costs several round trips.

In .NET, StackExchange.Redis doesn't expose WATCH directly, because one connection is shared by many callers. Its transactions take conditions instead, such as tran.AddCondition(Condition.StringEqual(key, expected)), and Execute returns false when a condition fails (StackExchange.Redis docs).

Lua scripts and Redis Functions

A Lua script runs on the server as one uninterruptible step. It can read, branch and write, so the transfer needs no retry loop. You saw this on Day 10 for locks and rate limits. Since Redis 7.0, Functions are the maintained way to do it: named, loaded once, stored with the data and replicated, instead of sending script text with every call.

Lua
#!lua name=points
local function transfer(keys, args)
  local amount = tonumber(args[1])
  local balance = tonumber(redis.call('GET', keys[1]) or '0')
  if balance < amount then
    return 0
  end
  redis.call('DECRBY', keys[1], amount)
  redis.call('INCRBY', keys[2], amount)
  return 1
end
redis.register_function('transfer', transfer)
Redis
cat points.lua | redis-cli -x FUNCTION LOAD REPLACE      # "points"
SET user:42:points 100
FCALL transfer 2 user:42:points user:99:points 30        # (integer) 1
FCALL transfer 2 user:42:points user:99:points 500       # (integer) 0, not enough points
MGET user:42:points user:99:points                       # "70" "30"
Pipelining batches commands into one round trip with no atomicity. MULTI and EXEC run a batch atomically but cannot branch on a value read mid-transaction. A Lua script runs the whole read-modify-write, with branches, atomically on the server.
Pipelining batches commands into one round trip with no atomicity. MULTI and EXEC run a batch atomically but cannot branch on a value read mid-transaction. A Lua script runs the whole read-modify-write, with branches, atomically on the server.

Why a slow script blocks Redis

The property that makes scripts atomic, that nothing else runs meanwhile, also makes a slow script block every client (Day 1). Keep scripts short and bounded: no loops over large collections. If a script runs past busy-reply-threshold (5 seconds by default), Redis doesn't stop it, since that would break atomicity. Other clients get BUSY errors. You can SCRIPT KILL or FUNCTION KILL it only if it hasn't written anything yet; after a write, the only way out is SHUTDOWN NOSAVE, which loses every change since the last save (Redis docs). Pass every key the script touches in KEYS, not built inside the script, so Redis Cluster can route it; all of them must live in the same slot (Day 8).

Check yourself

The other queued commands still ran. Redis transactions don't roll back on run-time errors, so validate inputs or use a script that checks before writing.

Pipeline them in batches of a few hundred to a few thousand. The time is mostly network round trips, not Redis work.

When the decision logic must run in the application, such as calling another service, and contention on the key is low, so retries are rare.

Scripts run atomically on the single command thread, so nothing else runs until the script finishes. A script that takes two seconds blocks every client for two seconds. Keep scripts short and bounded, and move bulk work into batches outside a script.

Pipelining sends many commands without waiting for each reply, which saves round trips, but other clients' commands can run in between. MULTI/EXEC queues commands and runs them together without interleaving, but doesn't roll back on errors. You can pipeline a transaction to get both benefits.