Majority Quorums Explained: Why 2f + 1 Nodes
On this page
Learning Objective
Compute how many failures a cluster of a given size tolerates, explain why the intersection property of majority quorums is what actually prevents divergence, and justify why consensus clusters are sized at 3, 5, or 7 nodes rather than at larger even numbers.
Concept Explanation
A quorum is simply the minimum number of nodes that must participate for an operation to count. The majority quorum used by every mainstream consensus algorithm is the floor of N divided by 2, plus one: 2 of 3, 3 of 5, 4 of 7. The arithmetic is trivial and the property it produces is the entire foundation of fault-tolerant agreement, so it is worth stating explicitly rather than leaving as an exercise. Any two majority quorums of the same cluster must share at least one node, because two disjoint sets each containing more than half the nodes would together contain more than all the nodes, which is impossible. That one sentence is why majority quorums work.
The intersection property is doing more work than it first appears. Because every decision requires a majority, and every two majorities overlap in at least one node, any new decision is guaranteed to consult at least one node that participated in the previous decision. That overlapping node carries the memory of what was already decided, which is exactly how a newly elected leader learns about entries committed by a leader it never spoke to. Raft's leader-completeness guarantee is built directly on this: a candidate must collect votes from a majority, each voter refuses to vote for a candidate whose log is less up to date than its own, and therefore any candidate that wins must have a log containing every committed entry, because at least one of its voters was in the majority that committed the most recent one.
Sizing follows from a formula worth memorising: to tolerate f crash failures you need N equal to 2f plus 1 nodes. Three nodes tolerate one failure, five tolerate two, seven tolerate three. Note what this does and does not promise. It promises that the cluster continues to commit while f or fewer nodes are down. It does not promise anything about data durability beyond that, and it emphatically does not mean that a 3-node cluster can survive two failures in a degraded mode - at two failures out of three, the cluster is not slow, it is simply unable to commit anything at all, because the single surviving node cannot form a majority no matter how healthy it is.
This is where the counter-intuitive result about even-numbered clusters comes in, and it trips up a lot of otherwise careful engineers. A 4-node cluster requires a quorum of 3, so it tolerates exactly one failure, which is the same fault tolerance as a 3-node cluster. You have added a machine, added its failure probability to the pool, added a node that must acknowledge every write and therefore widened the latency tail, and gained nothing whatsoever in fault tolerance. A 6-node cluster tolerates two failures, identical to a 5-node cluster. This is why every operational guide for etcd, ZooKeeper, and Consul recommends odd cluster sizes, and why adding a node to a healthy 3-node cluster to "make it more reliable" makes it measurably less reliable rather than more.
There is a genuine cost to larger clusters that pushes in the opposite direction from fault tolerance. Because every commit requires a majority to acknowledge a durable write, commit latency is governed by the slowest member of the fastest majority - the median-ish node, not the fastest one. Going from 3 nodes to 5 means waiting for 3 acknowledgements instead of 2, from a larger and more heterogeneous population, and the tail latency of that wait is what your application experiences. This is the reason production etcd and ZooKeeper clusters are almost always 3 or 5 nodes and essentially never 9: beyond 5, you are buying fault tolerance you will rarely use with write latency you pay for on every single request. Read-heavy workloads can add capacity without this cost through non-voting members, which etcd calls learners and ZooKeeper calls observers: they replicate the log and serve reads but take no part in quorum, so they add read throughput without widening the write path.
| Cluster Size (N) | Majority Quorum | Failures Tolerated (f) | Verdict |
|---|---|---|---|
| 1 | 1 | 0 | Not fault tolerant at all; fine only for local development |
| 2 | 2 | 0 | Worse than one node: either failure halts the cluster |
| 3 | 2 | 1 | The standard minimum for production, lowest write latency |
| 4 | 3 | 1 | Strictly worse than 3: same tolerance, more nodes, higher latency |
| 5 | 3 | 2 | The usual choice when surviving two simultaneous failures matters |
| 6 | 4 | 2 | Strictly worse than 5, for the same reason 4 is worse than 3 |
| 7 | 4 | 3 | Rare; the write-latency cost usually outweighs the extra tolerance |
Worked Example: Quorum Arithmetic Across a Rolling Restart
The most common way teams accidentally lose quorum is not a catastrophic outage but an ordinary maintenance window where the arithmetic was never checked. Walking the numbers through a rolling restart makes the failure obvious before it happens rather than during it.
# A 5-node etcd cluster: A B C D E. Majority quorum = 3.
# --- Scenario 1: correct rolling restart ---
nodes_up = 5, quorum = 3
restart A -> nodes_up = 4 >= 3 OK, cluster commits throughout
A rejoins -> nodes_up = 5
restart B -> nodes_up = 4 >= 3 OK
# ... one at a time, waiting for full health between each. Never at risk.
# --- Scenario 2: the classic mistake ---
# Node C has already been down for two days (nobody noticed: the cluster
# was still healthy at 4/5, so no alert fired on write availability).
nodes_up = 4, quorum = 3
restart A and B TOGETHER to save a maintenance window
-> nodes_up = 2 < 3 QUORUM LOST
# The cluster stops accepting writes entirely. In Kubernetes this means
# the API server cannot persist anything: no deployments, no scaling,
# no scheduling decisions recorded. Running pods keep running; the
# control plane is frozen.
# --- The rule that prevents it ---
def safe_to_take_down(total, currently_down, want_to_stop):
quorum = total // 2 + 1
remaining = total - currently_down - want_to_stop
return remaining >= quorum
safe_to_take_down(5, 0, 1) # True - normal rolling restart
safe_to_take_down(5, 1, 1) # True - one already down, take one more
safe_to_take_down(5, 1, 2) # False - this is scenario 2
safe_to_take_down(3, 0, 2) # False - 3-node clusters tolerate exactly one
# --- Why 4 nodes is worse than 3 ---
# N=3: quorum 2, survives 1 failure. Failure probability of a single node p.
# N=4: quorum 3, survives 1 failure - but there are now 4 nodes that can
# each be the one that fails, so the probability of reaching the
# intolerable state is HIGHER, not lower.Real-World Examples
- The official etcd operations documentation recommends odd-sized clusters of 3, 5, or 7 members and states plainly that adding a member to an even-sized cluster increases the number of nodes required for quorum without increasing failure tolerance, which is why the Kubernetes production guidance for stacked control planes is three or five etcd members and never four. The same document notes that cluster sizes beyond seven are discouraged because the write-latency cost of a wider quorum outweighs the marginal fault tolerance.
- ZooKeeper offers observer nodes and etcd offers learner members precisely to decouple read scaling from quorum size: both replicate the full committed state and serve reads, but neither votes, so a deployment can serve reads from twenty nodes while keeping the write quorum at three. ZooKeeper introduced observers specifically after operators discovered that scaling out voting ensembles to serve more reads was degrading write throughput across the entire ensemble.
Review Questions & Answers
A 4-node cluster needs a quorum of 3, so losing 2 nodes leaves it unable to commit, meaning it tolerates exactly one failure - the same as a 3-node cluster, which needs a quorum of 2. It is worse in practice for two reasons: there are four independent machines that could fail instead of three, so the probability of reaching an intolerable state is higher, and every write must now be acknowledged by three nodes instead of two, widening commit latency and its tail. You pay more and receive less.
Two sets that each contain more than half the nodes cannot be disjoint, because their combined size would exceed the total number of nodes. Therefore any two majority quorums share at least one node. Since committing a value requires a majority, and any subsequent decision also requires a majority, the second decision necessarily consults at least one node that participated in the first and that therefore knows what was already decided. That shared node is the mechanism by which knowledge of a committed value cannot be lost, and it is why a newly elected leader that never spoke to its predecessor can still be guaranteed to hold every committed entry.
A single node cannot form a majority of three, so it cannot commit. Allowing it to would destroy the intersection property that makes the algorithm safe: if the other two nodes are not actually dead but merely partitioned away, they together form a genuine majority and could elect a leader and commit their own conflicting writes. Both sides would then be accepting writes, which is exactly the split brain quorums exist to prevent. The surviving node has no way to distinguish "the other two crashed" from "the other two are fine and I am the one who is isolated", which is FLP again, and refusing to commit is the only safe response to that ambiguity.
Because the commit waits for the slowest node within the fastest majority, not for the single fastest node. With 3 nodes the leader waits for 1 follower to durably acknowledge; with 5 it waits for 2; with 7 it waits for 3. Each additional required acknowledgement draws from a larger and typically more heterogeneous population of machines, so the expected wait and especially the tail latency both increase. That cost is paid on every single write, whereas the extra fault tolerance is used only during rare multi-node failures, which is why 3 and 5 dominate real deployments.
Restarting two nodes while one is already down leaves two healthy members against a quorum of three, so the cluster loses quorum and stops committing entirely until a member returns. The alerting failure is that the system only alerted on total unavailability rather than on loss of redundancy: at 4 of 5 the cluster was still committing normally, so nothing fired, even though its remaining fault tolerance had silently dropped from two failures to one. Consensus clusters need an alert on any member being unhealthy at all, not merely on the cluster being down, precisely because the degraded state is invisible from the outside until the moment it becomes catastrophic.