Replication Lag and Read-Your-Writes Consistency
On this page
What replication lag is
Followers apply the leader's changes a little later: milliseconds normally, seconds or minutes under heavy load, network trouble or a large transaction. That's fine for most reads. But users notice three anomalies, and each has a specific fix.
Read-your-writes: my change disappeared
A user saves a new email address, the page reloads, and the old address appears. The write went to the leader; the read came from a follower that hadn't caught up.
Read-your-writes fixes it. Options:
- Read recently modified data from the leader: for example, for 60 seconds after a user's last write, route that user's reads (or reads of what they changed) to the leader.
- Track a replication position: the write returns a position or version (a log sequence number, a timestamp); later reads go to a replica that has reached at least that position, or wait until one has. MongoDB's causally consistent sessions do this (MongoDB course, Day 21).
- Show the update from the client side: the app already knows the new value and displays it, regardless of what the next read returns.
Here's the position check in PostgreSQL, tested on PostgreSQL 16 with a primary and one streaming replica whose replay was paused:
-- on the primary, right after the user's write commits
SELECT pg_current_wal_lsn(); -- 0/3000230: keep it in the user's session
-- on the replica, before serving that user's next read
SELECT pg_last_wal_replay_lsn() >= '0/3000230'::pg_lsn AS caught_up;caught_up = f replay paused at 0/30001C0; the replica still showed the old value
caught_up = t after replay resumed; the read returned the new valueA WAL position (log sequence number) only grows, so "has this replica reached my write?" is a plain comparison. If the answer is no, wait a few milliseconds and check again, try another replica, or read from the primary. Taking the position after the commit, rather than inside the transaction, matters: the commit record is what the replica has to replay.
Monotonic reads: time goes backwards
A user refreshes twice. The first read hits a fresh replica and shows a new comment; the second hits a lagging replica and the comment is gone.
Monotonic reads fix it: each user always reads from the same replica (choose by a hash of the user id), so they may see stale data but never older than before. If that replica fails, switch to one at least as current.
Consistent prefix reads: the answer before the question
Partitioned data on different replicas, lagging differently. A reader sees Bob's reply "Yes, Tuesday works" before Alice's question "Can we meet Tuesday?", because the reply's partition caught up first.
Consistent prefix reads fix it: causally related writes are seen in order. Writing related items to the same partition (keyed by conversation id), or using causal consistency (Day 8), keeps them together.
Measuring replication lag
You can't manage what you don't watch:
- PostgreSQL:
replay_laginpg_stat_replication; MySQL:Seconds_Behind_Source(or better, heartbeat tables); MongoDB:rs.printSecondaryReplicationInfo(); Kafka consumers: consumer lag. - Alert on lag, and take replicas that fall far behind out of the read pool automatically.
- Remember lag spikes during bulk writes, schema changes and backups.
Choosing a guarantee per screen
| Screen | Guarantee needed | Approach |
|---|---|---|
| "Your settings were saved" | Read-your-writes | Leader read or position token |
| Comment thread | Consistent prefix | Same partition per thread |
| Infinite-scroll feed | Monotonic reads | Pin user to a replica |
| Public product page | None | Any replica |
Check yourself
Read that user's recently changed data from the leader, or carry the write's replication position and read only from a replica that has reached it.
Provide monotonic reads: route each user's reads consistently to the same replica, for example by hashing the user id, so they never jump from a fresh replica to a more lagged one and see newer data disappear.
The question and reply were stored in different partitions with different lag; keeping related data in one partition or using causal consistency fixes it.
A WAL position is a single increasing log offset, so "the replica has replayed past my write" is exact; timestamps from two machines' clocks can't be compared reliably (Day 3).
Compare the replica's replayed log position with the primary's current position, or write a heartbeat timestamp row on the primary and measure how old it is on each replica.