Redis Pub/Sub vs Redis Streams
On this page
Redis Pub/Sub vs Streams: which one?
Redis has two messaging tools that look similar and behave very differently. The question that separates them: if a consumer is down when a message is sent, does it matter?
| Pub/Sub | Streams | |
|---|---|---|
| Stored? | No, delivered and forgotten | Yes, until you trim the stream |
| A subscriber was offline | The message is lost for it | It reads what it missed when it comes back |
| Delivery | At most once | At least once, with acknowledgments |
| Several workers sharing the load | No, every subscriber gets every message | Yes, consumer groups split messages |
| Replay old messages | No | Yes, by ID range |
Redis Pub/Sub: fire-and-forget broadcast
SUBSCRIBE to a channel, PUBLISH to it, and every connected subscriber receives the message immediately. Nothing is stored. A subscriber that's restarting misses it, permanently.
That makes it ideal when a missed message doesn't matter, or fixes itself:
# on every app server
SUBSCRIBE cache-invalidate
# from whichever server changed the product
PUBLISH cache-invalidate "product:8842" # (integer) 12, the number of receiversEach app server drops its in-memory copy of product 8842. A server that missed it serves stale data until its local TTL expires, which is acceptable. Other good fits: "user is typing" indicators, live dashboards, routing a WebSocket event to whichever server holds that user's connection.
Two limits to know. A subscriber that reads too slowly gets disconnected: by default Redis drops a Pub/Sub client whose pending output passes 32 MB, or stays above 8 MB for 60 seconds (client-output-buffer-limit pubsub), and the messages in that buffer are lost. And in Redis Cluster, a regular PUBLISH is forwarded to every node. For high volume, sharded Pub/Sub (SPUBLISH and SSUBSCRIBE, Redis 7.0 and later) keeps each channel's messages within one shard (Redis docs).
Redis Streams and consumer groups
A stream, added in Redis 5.0, is an append-only log. Each entry gets an ID made from a millisecond timestamp and a sequence number. Entries stay until you trim them.
For work that must not be lost, such as processing orders, combine a stream with a consumer group. Each entry goes to one consumer in the group, and stays pending until that consumer acknowledges it:
XADD orders '*' order_id 90231 status pending # "1791608802165-0"
XGROUP CREATE orders order-workers 0
XREADGROUP GROUP order-workers worker-1 COUNT 1 STREAMS orders '>'
# ... worker-1 processes the order ...
XACK orders order-workers 1791608802165-0If worker-1 crashes before XACK, the entry stays in the group's pending list. Another worker can take over entries that have been pending too long:
XPENDING orders order-workers # 1 pending, owned by worker-1
XAUTOCLAIM orders order-workers worker-2 60000 0-0 COUNT 10 # claim entries idle over 60 sAt-least-once delivery means a message can be processed twice: once by the worker that crashed after doing the work but before acknowledging, and again by the one that claimed it. Make processing idempotent, for example by recording the order ID as processed and skipping repeats.
XPENDING also reports how many times each entry has been delivered. An entry that keeps failing is a poison message. After a few attempts, copy it to a separate dead-letter stream, acknowledge it and alert someone, so workers don't retry it forever.
Trimming a stream with MAXLEN
A stream keeps every entry, including acknowledged ones. Without a limit, it grows until Redis runs out of memory. Cap it when you add:
XADD orders MAXLEN '~' 100000 '*' order_id 90232 status pendingThe ~ lets Redis trim in efficient chunks, so the stream holds roughly 100,000 entries rather than exactly. Pick the cap from how far back you might need to replay, and remember that durability also depends on your persistence settings (Day 5).
Redis Streams vs Kafka
Streams cover a lot of queue and event work for teams already running Redis. A dedicated log like Kafka becomes the better choice when you need to keep days of history on disk, very high throughput across many partitions, or a large ecosystem of connectors. Data in a Redis stream lives in memory.
Check yourself
Pub/Sub. The event matters only for a few seconds, and a missed one has no lasting effect, so storing it adds cost for nothing.
It remains pending in the consumer group because it was never acknowledged. Another worker claims it with XAUTOCLAIM or XCLAIM and processes it, which is why processing must be idempotent.
Acknowledging a message with XACK only removes it from the consumer group's pending list. The entry itself stays in the stream until you trim it. Cap the stream with MAXLEN or MINID on XADD, using ~ for efficient approximate trimming, or run XTRIM periodically.
No. Within a consumer group, each message is delivered to one consumer, which is how work is shared out. If two different services each need every message, give each service its own consumer group.
Pub/Sub is fire and forget: Redis delivers a message to whoever is subscribed at that moment and keeps no copy. A subscriber that was reconnecting simply never sees it. Use Streams when messages must survive disconnections or need acknowledgment.