Availability vs Reliability vs Fault Tolerance
On this page
- Why three 99.9 percent parts don't make a 99.9 percent system
- Availability vs reliability vs fault tolerance
- The nines: how much downtime each allows
- How to calculate system availability
- Why redundancy delivers less than the math promises
- MTBF and MTTR: recover faster, not just fail less
- How to find single points of failure
- Check yourself
Why three 99.9 percent parts don't make a 99.9 percent system
Take a load balancer, an app server and a database, each available 99.9 percent of the time. Most people guess the whole system is also 99.9 percent. It's lower, and the gap grows with every component you add in a chain. Knowing the arithmetic changes how you design.
Availability vs reliability vs fault tolerance
- Availability: the share of time the system works and answers requests.
- Reliability: whether it does the right thing, correctly and consistently. A system can be available but unreliable: it answers instantly, with wrong data.
- Fault tolerance: the ability to keep working, perhaps in a reduced way, when parts fail. It comes from redundancy: having more than one of each critical part, so no single point of failure (SPOF) can take everything down.
The nines: how much downtime each allows
| Availability | Downtime per year | Per month |
|---|---|---|
| 99% | about 3.65 days | about 7.3 hours |
| 99.9% | about 8.8 hours | about 44 minutes |
| 99.99% | about 53 minutes | about 4.4 minutes |
| 99.999% | about 5 minutes | about 26 seconds |
Each extra nine cuts the budget by ten. Four nines leaves no time for a human to wake up, log in and fix things by hand; it requires automatic failover. That's why each nine costs much more than the last.
How to calculate system availability
In series, when every component must work for a request to succeed, multiply their availabilities:
load balancer × app × database = 0.999 × 0.999 × 0.999 ≈ 0.997 (99.7%)That's about 26 hours of downtime a year instead of 8.8. Every hard dependency you add to a request's path lowers availability.
In parallel, when any one of several redundant copies is enough, the system fails only if all fail at once:
two database copies at 99.9% each: 1 − (0.001 × 0.001) = 0.999999 (99.9999%)Recompute the chain with a 99.95 percent load balancer, one app tier at 99.9 percent and the redundant database:
0.9995 × 0.999 × 0.999999 ≈ 0.9985 (99.85%)Now the app tier is the weakest link, so make it redundant too: run several instances behind the load balancer.
Why redundancy delivers less than the math promises
The parallel formula makes two assumptions that real systems break:
- Failures are independent. Two copies in the same rack, on the same power feed, or running the same buggy release fail together. That's why cloud providers offer separate availability zones, and why you roll out changes gradually.
- Failover is instant and always works. In practice a replica takes 30 seconds to a few minutes to be detected as down and promoted, and failover itself sometimes fails because nobody tested it (Day 76). Every failover spends downtime budget.
Cloud SLAs show the effect. AWS commits to 99.5 percent monthly uptime for a single EC2 instance, but 99.99 percent for instances spread across two or more availability zones in a region (EC2 SLA). The gap between those two numbers is what redundancy buys.
MTBF and MTTR: recover faster, not just fail less
Availability depends on two things: how often you fail and how long each failure lasts.
availability = MTBF / (MTBF + MTTR)
MTBF = mean time between failures, MTTR = mean time to recoveryA service that fails once a month (about 720 hours) and takes 1 hour to recover is about 99.86 percent available. Cut recovery to 5 minutes and it's about 99.99 percent, with the same number of failures. Automatic failover, fast rollbacks and good alerts all shrink MTTR, and they're often cheaper than preventing every failure.
How to find single points of failure
Walk a request's path and ask of each component: "If this one disappears, does everything stop?" In a typical three-tier app on one VM, the answer is yes for the VM, the database and often the load balancer. Fix them one layer at a time: multiple app instances, a database replica with failover, a managed load balancer that's redundant itself.
Don't forget the dependencies outside your diagram: DNS, the identity provider, a payment API. On February 28, 2017, a mistyped command during debugging took Amazon S3 in us-east-1 offline for hours, and many services that depended on it went down with it (AWS summary).
In practice, measure availability as the share of requests that succeed, not minutes the server was up. Day 78 turns this into SLOs.
Check yourself
For components in series, availabilities multiply: 0.999^5 ≈ 0.995, or about 99.5 percent, which is roughly 44 hours of downtime a year. That's before counting the load balancer, database and network. Every synchronous dependency you add to a request path lowers its availability.
The parallel availability formula assumes failover is instant and that failures are independent. In reality, detecting the failure and promoting a replica takes time, often minutes, and both nodes may share a cause like a bad deploy or a region outage. Each failover spends part of your downtime budget.
The budget is about 4 minutes a month. Detecting a failure and having a person fix it takes longer than that, so failover must be automatic.
Mean time to recovery dropped, through automatic failover, faster rollbacks or better alerting. Shorter outages raise availability as much as fewer outages.
Availability is the share of time the system is up and answering. Reliability is how often it does the right thing without failure over a period. A system can be highly available but unreliable if it stays up while returning wrong or failed results for some requests.