Catalog My learning Blog
Apache Kafka Tutorial: Event Streaming to Production · Phase 3 · Lesson 16 · 3 min read

Stateless vs Stateful Stream Processing

On this page
  1. Stateless vs stateful operations
  2. State stores: RocksDB and changelog topics
  3. Faster recovery: standby replicas and persistent volumes
  4. Repartition topics: grouping by a new key
  5. Interactive queries on state stores
  6. Check yourself

Stateless vs stateful operations

"Drop test orders" needs nothing but the current record. "Count orders per customer" needs to remember previous counts. "Join payments to orders" needs to remember orders until their payment arrives. The first is stateless, the others stateful, and stateful processing brings storage, recovery and scaling questions with it.

StatelessStateful
filter, map, mapValues, flatMap, branch/split, peekcount, aggregate, reduce, joins, windowing
No memory between recordsKeeps state per key
Trivial to restartState must survive crashes and move with partitions

State stores: RocksDB and changelog topics

Each stateful operation has a state store, by default a RocksDB database on the instance's local disk, one per task. Reads and writes are local and fast, with no network round trip to an external database.

Local disk can be lost with the instance, so every update to the store is also written to a changelog topic in Kafka, compacted (Day 10) so it keeps the latest value per key. If an instance dies and its task moves elsewhere, the new owner rebuilds the store by replaying the changelog.

Each task keeps state in a local RocksDB store and writes every change to a compacted changelog topic, which another instance replays to restore the state after a failure.
Each task keeps state in a local RocksDB store and writes every change to a compacted changelog topic, which another instance replays to restore the state after a failure.

Faster recovery: standby replicas and persistent volumes

Restoring a large store from its changelog can take minutes or longer, and the task doesn't process during restoration. Two settings help:

  • num.standby.replicas=1: another instance keeps a warm copy of each store by continuously reading the changelog. On failover, it takes over almost immediately.
  • Persistent volumes for the state directory (state.dir): a restarted instance on the same disk only replays changes since its last checkpoint. In Kubernetes, run Streams apps as StatefulSets with persistent volumes for this reason.

Since Kafka 3.x, tasks also go through a warm-up phase when the assignment changes, so a task only moves once its new home has caught up.

Repartition topics: grouping by a new key

State is partitioned like the input: the task for partition 3 holds the counts for keys in partition 3. If you group by something other than the current key, the records have to move:

JavaScript
orders.selectKey((orderId, order) -> order.customerId())   // change the key
      .groupByKey()
      .count();

Streams writes the re-keyed records to an internal repartition topic, so all records for one customer end up in the same partition and the same task. That costs an extra write and read through Kafka. Avoid unnecessary re-keying: key the input topic by what you aggregate on (Day 14), or use groupBy deliberately knowing it repartitions.

Interactive queries on state stores

State stores can be read directly with interactive queries: an instance can look up "count for customer 42" in its local store, and with metadata about which instance owns which key, you can build an API that routes each lookup to the right instance. Simple read models don't always need a separate database.

Check yourself

The task moves to another instance, which rebuilds the local store by replaying the store's compacted changelog topic, or takes over a standby replica's copy.

Standby replicas keep a continuously updated copy of each state store on another instance. If the active instance fails, the task moves to the instance with the standby, which only has to catch up on recent changes instead of replaying the whole changelog, so failover takes seconds instead of minutes.

State in Kafka Streams is partitioned by key, so every record for a given key must reach the same task. When you group by a different key, records are written to an internal repartition topic keyed by the new key, so they're redistributed to the right partitions and tasks.

Stateless operations, like filter or mapValues, look at one record at a time. Stateful ones, like count, aggregate, joins and windows, remember data across records in state stores, which must be backed up and restored.

They let your application read a state store directly, for example to serve the current count for a customer from an HTTP endpoint, without writing the results to another database first.