Redis Sentinel: High Availability Tutorial
On this page
What Redis Sentinel does
Yesterday's failover was manual: someone notices the primary is down, promotes a replica and repoints everything. Sentinel automates exactly that. By the end of this lesson you'll know how to run it, what it does second by second, and what your application has to do to follow a failover.
Step 1: run three Sentinels with a quorum
A Sentinel is a Redis process started in sentinel mode. Each one gets a tiny config naming the primary to watch:
# sentinel.conf
port 26379
sentinel monitor app 10.0.0.10 6379 2 # name, primary address, quorum
sentinel down-after-milliseconds app 3000 # no reply for 3 s means "down" to this Sentinel
sentinel failover-timeout app 10000redis-server sentinel.conf --sentinel
redis-cli -p 26379 SENTINEL ckquorum app
# OK 3 usable Sentinels. Quorum and failover authorization can be reachedRun at least three, on different hosts from each other and ideally not all next to the primary. Sentinels discover the replicas and each other automatically from the primary.
Step 2: what happens during a Sentinel failover
I ran this locally with three Sentinels, one primary and one replica, then froze the primary. The log of one Sentinel, trimmed:
01:02:55.225 +sdown master app 127.0.0.1 6390
01:02:55.297 +odown master app 127.0.0.1 6390 #quorum 3/2
01:02:56.372 +switch-master app 127.0.0.1 6390 127.0.0.1 6391Reading it line by line:
- Subjectively down (
+sdown). This Sentinel got no valid reply for 3 seconds. One Sentinel's view could be its own network problem, so nothing happens yet. - Objectively down (
+odown). Enough Sentinels agree, here 3 of a required quorum of 2. - Leader election. The Sentinels elect one of themselves to run the failover. This step needs a majority of all Sentinels, not just the quorum, which is why one Sentinel alone can never fail over.
- Promotion (
+switch-master). The leader picks the best replica, mainly the one with the most replicated data, promotes it, and repoints the other replicas. When the old primary comes back, it's turned into a replica.
From the primary freezing to a new primary took about 4 seconds here. In production, plan for several seconds of failed writes. Sentinel shortens the outage; it doesn't remove it.
Step 3: connect clients through Sentinel
After a failover the primary has a new address, so clients can't hardcode it. They ask Sentinel instead:
redis-cli -p 26379 SENTINEL get-master-addr-by-name app
# 1) "127.0.0.1"
# 2) "6391"Mainstream client libraries do this for you. In Python with redis-py:
from redis.sentinel import Sentinel
sentinel = Sentinel([("10.0.0.21", 26379), ("10.0.0.22", 26379), ("10.0.0.23", 26379)],
socket_timeout=0.5)
primary = sentinel.master_for("app", socket_timeout=0.5) # follows failovers
replica = sentinel.slave_for("app", socket_timeout=0.5) # for reads
primary.set("greeting", "hello")In .NET, StackExchange.Redis does the same when the connection string names the service: ConnectionMultiplexer.Connect("10.0.0.21:26379,10.0.0.22:26379,10.0.0.23:26379,serviceName=app") asks Sentinel for the current primary and follows it when it changes (StackExchange.Redis docs).
Make sure your code retries writes that fail during the failover window, with a short backoff.
Common Sentinel problems and fixes
| Symptom | Likely cause | Fix |
|---|---|---|
| Failover never happens | Too few Sentinels reachable for a majority | Run 3 or 5 Sentinels on separate hosts |
| Failovers on every network blip | down-after-milliseconds too low | Raise it above normal latency spikes, often 5 to 30 seconds |
| App keeps writing to the old primary | Client hardcodes the address | Use a Sentinel-aware client connection |
| Writes lost after failover | Async replication gap (Day 6) | min-replicas-to-write, WAIT for critical writes |
On a managed service you don't run Sentinel yourself. The provider detects the failure, promotes a replica and repoints a DNS name or endpoint, so your client connects to that endpoint and only needs to reconnect and retry.
Sentinel gives you high availability for one primary. It doesn't add capacity: all writes still go to one node. When one node can't hold the data or the write load, you need Redis Cluster, tomorrow's lesson.
Check yourself
No. Electing a failover leader needs a majority of all Sentinels, and one of two isn't a majority. Run at least three.
No, that's expected. Sentinel has to notice the primary is gone, which depends on down-after-milliseconds, agree on it, elect a leader and promote a replica, which together usually take a few seconds. Make sure clients reconnect automatically and retry writes with backoff during that window.
Its view could be caused by its own network problem. A quorum must agree, and a majority must elect a leader, before anything changes.
They ask Sentinel, not a fixed address. Sentinel-aware clients connect to the Sentinels, ask for the current primary by its service name, and reconnect when Sentinel announces a switch. Hard-coding the primary's IP address means the app keeps talking to the dead node after a failover.
On three independent failure domains, such as separate hosts or availability zones, ideally not all on the same machines as the Redis nodes. If two Sentinels share a host that fails, the remaining one can't form a majority, and no failover happens.