Catalog My learning Blog
Elasticsearch Tutorial: How Full-Text Search Works · Phase 1 · Lesson 2 · 3 min read

Elasticsearch Architecture: Nodes, Clusters, Shards

On this page
  1. Nodes, clusters and shards
  2. Primary shards and replica shards
  3. Why a single-node cluster turns yellow
  4. Elasticsearch node roles
  5. How a search runs: query phase and fetch phase
  6. Check yourself

Nodes, clusters and shards

A single Elasticsearch process is a node. Several nodes with the same cluster name form a cluster. An index is split into shards that are spread across those nodes, so an index can grow beyond one machine and searches run on many machines in parallel.

Picture an encyclopedia too big for one shelf. You split it into volumes (shards) and spread them across several shelves (nodes), so several people can look things up at once. Then you keep a photocopy of each volume on a different shelf (replicas), so losing one shelf loses no volume.

Primary shards and replica shards

  • A primary shard holds a slice of the index's documents. Each shard is a complete Lucene index.
  • A replica shard is a copy of a primary on a different node. It serves searches and takes over if the primary's node fails.
  • The number of primaries is fixed when the index is created (changing it means splitting, shrinking or reindexing, Day 19). Replicas can be changed at any time.
HTTP
PUT /orders
{ "settings": { "number_of_shards": 3, "number_of_replicas": 1 } }

Three primaries and three replicas: six shards in total, needing at least two nodes.

An index split into primary shards spread across nodes, each with a replica on a different node, so any one node can fail without losing data.
An index split into primary shards spread across nodes, each with a replica on a different node, so any one node can fail without losing data.

Why a single-node cluster turns yellow

Created on the course's one-node lab cluster:

Code
{'status': 'yellow', 'active_primary_shards': 3, 'active_shards': 3, 'unassigned_shards': 3}

index  shard prirep state      node
orders 0     p      STARTED    node-1
orders 0     r      UNASSIGNED
orders 1     p      STARTED    node-1
orders 1     r      UNASSIGNED
...

The cluster refuses to put a replica on the same node as its primary, since a copy on the same machine protects against nothing. Asking why with the allocation explain API:

Code
node-1 same_shard - a copy of this shard is already allocated to this node

Cluster health summarizes this:

StatusMeaning
GreenAll primaries and replicas assigned
YellowAll primaries assigned, some replicas not: data is searchable but not fully protected
RedAt least one primary unassigned: some data is unavailable

Setting number_of_replicas to 0 turned the lab cluster green. Do that only in development; indices get 1 replica by default, which is why a single-node cluster is often yellow.

Elasticsearch node roles

In larger clusters, nodes specialize:

RoleJob
Master-eligibleManage cluster state: indices, mappings, shard placement. Run three of them so a majority survives one failure
Data (data_hot, data_warm, data_cold, data_frozen)Hold shards and run searches and indexing; tiers for different hardware (Day 24)
IngestRun ingest pipelines that transform documents before indexing
Coordinating onlyRoute requests and merge results from shards
ML, transformMachine learning jobs and data transforms

Every node can coordinate: the node that receives a search sends it to one copy of each shard, gathers the top results, and returns them.

How a search runs: query phase and fetch phase

  1. Query phase: the coordinating node sends the query to one copy (primary or replica) of every shard; each returns its top document IDs and scores.
  2. Fetch phase: the coordinator merges them, then fetches the full documents for the final page.

More shards mean more parallelism but more overhead per search; shard sizing is a balance (Day 22).

Check yourself

Replicas can't be placed on the node holding their primary, so they stay unassigned; data is available but not redundant.

Not in place; you split, shrink or reindex into a new index. Replica counts can be changed at any time.

A majority of master-eligible nodes must agree on every cluster state change. With three, the cluster keeps a majority after losing one; with two, losing either leaves no majority, so the cluster can't elect a master.

Six copies (3 primaries plus 3 replicas), and at least two nodes, since a replica can't share a node with its own primary.

In the query phase, each shard finds and scores its top matches and returns ids and scores. In the fetch phase, the coordinating node retrieves the full documents for the overall top results.