Catalog My learning Blog
Raft and Paxos Explained: Consensus, ZooKeeper and etcd · Phase 2 · Lesson 7 · 11 min read

Raft Leader Election Explained

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: A Full Election Round Including a Rejected Candidate
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Describe Raft's three server states and the term mechanism that orders them, walk through a complete election including the up-to-date log restriction on voting, and explain why randomised election timeouts are the mechanism that resolves split votes.

Concept Explanation

Raft was published by Diego Ongaro and John Ousterhout at USENIX ATC 2014 under the title "In Search of an Understandable Consensus Algorithm", and understandability was an explicit design goal rather than a happy side effect: the authors ran a user study comparing student comprehension of Raft against Paxos and used the results as a design signal. The central design decision is a strong leader. Where Paxos allows any proposer to drive a round at any time, Raft insists that all client writes go through a single leader and that log entries flow in exactly one direction, from leader to followers, never the reverse. That restriction makes the protocol dramatically easier to reason about, at the cost of ruling out some optimisations that leaderless designs permit.

Every server is in one of three states. A follower is passive: it responds to RPCs from leaders and candidates and does nothing on its own initiative. A candidate is a server attempting to become leader. A leader handles all client requests and replicates entries. Time is divided into terms, which act as a logical clock: each term is identified by a monotonically increasing integer, begins with an election, and has at most one leader, though it may have none if the election fails. Every RPC carries the sender's term, and two unconditional rules govern term handling. If a server receives any message with a term greater than its own, it immediately adopts that term and reverts to follower, without argument. If a server receives any message with a term lower than its own, it rejects the message and replies with its own term so the stale sender learns it is behind. Those two rules alone eliminate an entire category of confusion that Paxos implementations have to reason about individually.

An election begins when a follower's election timeout expires without hearing from a leader. It increments its current term, transitions to candidate, votes for itself, and sends RequestVote RPCs to every other server in parallel. A server grants its vote only if three conditions hold: the candidate's term is at least its own, it has not already voted for someone else in this term, and - the condition that carries all the safety weight - the candidate's log is at least as up to date as its own. Up to date is defined by comparing the last entry: the log with the higher last term wins, and if the terms are equal, the longer log wins. This voting restriction is what guarantees the leader completeness property. Because a candidate needs a majority of votes, and any majority intersects the majority that committed the most recent entry, at least one voter holds that entry and will refuse to vote for a candidate missing it. A candidate lacking any committed entry therefore mathematically cannot win.

An election has exactly three outcomes. The candidate collects a majority and becomes leader, immediately sending heartbeats to suppress further elections. Another server establishes itself as leader, which the candidate recognises by receiving an AppendEntries carrying a term at least as large as its own, at which point it steps down to follower. Or nobody wins, because votes split across multiple candidates and none reaches a majority. The third case is why randomisation matters. If every server used the same fixed election timeout, servers that time out together would keep splitting the vote in the same pattern indefinitely, which is precisely the symmetry Ben-Or's randomised consensus was designed to break. Raft instead draws each election timeout randomly from an interval - 150 to 300 milliseconds in the paper - so one server almost always times out meaningfully earlier than the rest, wins before others even start, and split votes resolve within a round or two.

Three pieces of state must be written to stable storage before a server responds to any RPC, and getting this wrong produces bugs that only appear under crash-restart testing. currentTerm must be durable, or a restarted server could vote twice in the same term and allow two leaders. votedFor must be durable for the same reason. The log entries themselves must be durable, or a server could acknowledge an entry, crash, restart without it, and break the majority arithmetic that made the entry committed. Everything else, including commitIndex and lastApplied, can be reconstructed. This is also why fsync latency on the leader's disk is one of the most common causes of Raft election churn in production: a leader that cannot persist fast enough cannot send heartbeats fast enough.

StateResponds ToInitiatesExits When
FollowerAppendEntries and RequestVote from othersNothingElection timeout expires with no leader contact
CandidateRPCs, stepping down on any higher or equal-term leaderRequestVote to all peers, in parallelWins a majority, sees a valid leader, or times out and retries
LeaderAll client requestsPeriodic AppendEntries heartbeats and log replicationDiscovers any server with a higher term, then steps down immediately

Worked Example: A Full Election Round Including a Rejected Candidate

The up-to-date restriction on voting is the part of the election most often skimmed, so this trace includes a candidate that is perfectly healthy, perfectly reachable, and still cannot win because its log is behind.

Code
# 5-node cluster: A B C D E. Majority = 3. Leader A crashes in term 4.
# Log state at the moment A dies (index: term):
#   A: [1:1, 2:2, 3:3, 4:4]   (crashed)
#   B: [1:1, 2:2, 3:3, 4:4]   lastLogIndex=4 lastLogTerm=4
#   C: [1:1, 2:2, 3:3, 4:4]   lastLogIndex=4 lastLogTerm=4
#   D: [1:1, 2:2, 3:3]        lastLogIndex=3 lastLogTerm=3   (lagging)
#   E: [1:1, 2:2, 3:3]        lastLogIndex=3 lastLogTerm=3   (lagging)

# --- D's randomised timeout fires FIRST (it drew 165ms) ---
D: currentTerm 4 -> 5, state=candidate, votedFor=D
D -> B,C,E : RequestVote(term=5, candidateId=D, lastLogIndex=3, lastLogTerm=3)

B: term 5 > 4, so B adopts term 5 and becomes follower.
   Up-to-date check: candidate lastLogTerm=3 < B's lastLogTerm=4
   -> VOTE DENIED. B has a committed entry D is missing.
C: same reasoning -> VOTE DENIED.
E: lastLogTerm=3 equals E's 3, lastLogIndex=3 equals E's 3
   -> vote GRANTED (E has not voted this term).

D has 2 votes (itself + E) of 5. Not a majority. D cannot win.
# This is leader completeness doing its job: D must not become leader
# because index 4 may already be committed, and a leader missing a
# committed entry could overwrite it.

# --- B's timeout fires next (it drew 240ms), now in term 6 ---
B: currentTerm 5 -> 6, state=candidate, votedFor=B
B -> C,D,E : RequestVote(term=6, candidateId=B, lastLogIndex=4, lastLogTerm=4)

C: lastLogTerm 4 >= 4, lastLogIndex 4 >= 4 -> GRANTED
D: term 6 > 5, steps down; B's log is more up to date -> GRANTED
E: same -> GRANTED

B has 4 votes of 5 -> LEADER for term 6.
B immediately sends empty AppendEntries heartbeats to suppress any
further election, and begins bringing D and E up to date.

# --- A returns from the dead ---
A -> C : AppendEntries(term=4, ...)
C -> A : {term: 6, success: false}
A sees 6 > 4 and reverts to follower unconditionally. It does not
argue, retry, or attempt to reclaim leadership.
A Raft election across a five-node cluster: a lagging candidate is denied votes by the up-to-date log restriction and fails to reach a majority, while a candidate holding every committed entry wins the next term and immediately begins sending heartbeats.
A Raft election across a five-node cluster: a lagging candidate is denied votes by the up-to-date log restriction and fails to reach a majority, while a candidate holding every committed entry wins the next term and immediately begins sending heartbeats.

Real-World Examples

  • etcd, the datastore behind every Kubernetes cluster, implements Raft directly and ships with a 100 millisecond heartbeat interval against a 1000 millisecond election timeout by default, with documentation explicitly warning that the heartbeat should be set near the round-trip time between members and that the election timeout should be at least ten times that. Deployments that stretch an etcd cluster across regions without adjusting these values are the most frequently reported cause of spurious leader elections in Kubernetes control planes.
  • HashiCorp Consul uses Raft for its server ring and documents the same tension operationally, shipping tuning presets for datacentre-local versus wide-area deployments precisely because the default election timeout that works over a local network produces continuous leadership churn over higher-latency links. Consul's guidance to keep server counts at three or five and to co-locate them within a single low-latency network is a direct consequence of Raft's liveness depending on heartbeats arriving well inside the election timeout.
Key TakeawaysThe single rule that makes Raft elections safe is the up-to-date log restriction on granting a vote: a voter refuses any candidate whose last log entry is behind its own. Because a candidate needs a majority and any majority intersects the majority that committed the latest entry, a candidate missing a committed entry cannot mathematically win. Randomised election timeouts are not a safety mechanism at all - they exist purely to break the symmetry that would otherwise cause split votes to repeat indefinitely.

Review Questions & Answers

The rule guarantees leader completeness: any elected leader holds every committed entry. A committed entry is stored on a majority, and a candidate needs votes from a majority, and two majorities always intersect, so at least one voter holds that entry and refuses a candidate lacking it. Without the rule, a lagging server could be elected and, because entries flow only from leader to follower in Raft, it would force followers to truncate the committed entries it was missing, destroying data that clients had already been told was durable. The restriction converts leader completeness from something the protocol must repair afterwards into something guaranteed at election time.

The candidate wins a majority and becomes leader; another server establishes itself as leader and the candidate steps down on receiving an AppendEntries with a term at least as large as its own; or nobody wins because the vote splits across multiple candidates and none reaches a majority. The third outcome motivates randomisation: with identical fixed timeouts, the same servers would time out simultaneously every round and split the vote in the same pattern indefinitely. Drawing each timeout from a random interval makes one server very likely to start meaningfully earlier and win before others begin, so split votes resolve after a round or two rather than repeating.

Because losing them across a restart allows two leaders in one term. A server that votes for candidate X in term 9, crashes before persisting that fact, restarts believing it is still in term 8 with no vote cast, and then votes for candidate Y in term 9 has voted twice in the same term. With enough such servers, X and Y could each assemble a majority and both become leader for term 9, which breaks agreement directly. The log entries themselves must be durable for the analogous reason: acknowledging an entry and then losing it on restart invalidates the majority arithmetic that made the entry committed.

The cluster will churn through elections. A Raft leader must persist entries before it can make progress and must send heartbeats within the election timeout; if durable writes take longer than the timeout, followers stop hearing from it and start elections. The newly elected leader will typically be on the same class of hardware and hit the same problem, so leadership moves around the cluster while very little commits. Nothing unsafe happens - this is a pure liveness failure - and the root cause is storage latency rather than the network or the algorithm. The fix is faster durable storage or a widened election timeout so the partial-synchrony assumption holds again.

It gains a dramatic simplification of the correctness argument: there is exactly one authoritative log, conflicts are always resolved by making the follower match the leader, and there is never a need to merge divergent logs or reason about entries arriving from multiple sources. That single constraint is most of why Raft is comprehensible enough to implement from the paper. What it gives up is flexibility available to leaderless or multi-leader designs, such as accepting writes at whichever replica is nearest a client, committing non-conflicting operations concurrently through different nodes, or the leaderless commit paths that variants like Egalitarian Paxos exploit. Raft trades peak flexibility for implementability, which for most infrastructure is the correct trade.