Requirements, trade-offs and estimates
Good designs start from numbers, not boxes. Turn vague requirements into measurable targets for latency and availability, estimate load with back-of-the-envelope maths, and see why every extra “nine” of availability has a price.
In this lecture you will
- Separate functional requirements from non-functional ones and make the latter measurable
- Express availability in nines and translate it into allowed downtime
- Calculate the availability of components in series and in parallel
- Estimate traffic, storage and bandwidth with back-of-the-envelope calculations
Ask two engineers to “design a URL shortener” and you may get two completely different — and both reasonable — answers. The difference lies in the requirements. A shortener for an internal team and one for a social network with a billion users share almost no design decisions. System design therefore starts with questions and numbers, not with boxes and arrows.
Functional and non-functional requirements
Functional requirements say what the system does: users can create a short link; visiting it redirects to the original URL. Non-functional requirements say how well it must do it — and they drive the architecture:
- Latency
- How fast individual requests complete — usually stated as a percentile, such as “95% of redirects in under 50 ms”.
- Throughput
- How much work the system handles — requests per second, messages per minute.
- Availability
- The fraction of time the system works, often expressed in “nines”.
- Durability
- The probability that stored data is never lost.
- Consistency
- Whether every reader sees the latest write immediately, or may briefly see older data.
- Cost
- Infrastructure spend and the engineering effort to build and run it.
Make them measurable. “Fast” and “highly available” can’t guide a decision; “p99 below 200 ms at 5,000 requests per second, 99.95% monthly availability” can.
Nines, downtime and redundancy
Availability sounds abstract until you convert it into allowed downtime. Explore below: pick a target, then build a request path and see what it achieves.
Two lessons stand out. First, components in series multiply: a chain is always less available than its weakest link. Second, redundancy in parallel is how systems exceed the reliability of any single machine — two app servers, where either can serve, fail together far less often than one fails alone. Each extra nine usually means more redundancy, automation and testing, and costs noticeably more.
Back-of-the-envelope estimates
Quick arithmetic rules options in or out before anyone spends a week on a design. A few numbers are worth memorizing:
| Quantity | Rough value |
|---|---|
| Seconds in a day | ~86,400 (≈ 10⁵ for quick maths) |
| Requests per second from 1 million requests a day | ~12 |
| Memory access vs. SSD read vs. cross-continent round trip | ~100 ns vs. ~100 µs vs. ~100–200 ms |
| One commodity server | thousands of simple HTTP requests per second |
A worked example: an app with 10 million daily active users, each making 20 requests a day, receives 200 million requests a day — about 2,300 per second on average. Traffic is rarely flat, so plan for a peak of perhaps 3–5 times that. If each request writes 1 KB, that is 200 GB of new data a day, about 73 TB a year before replication. Those numbers immediately tell you that one database server won’t hold a few years of data — the design must plan for partitioning (lecture 4).
Estimates like these take a minute once you have the habit. Try your own product’s numbers — the defaults below are the worked example:
Decimal units (1 KB = 1,000 bytes). Server capacity varies enormously with the work per request — measure yours. The hints are rules of thumb, not limits.
Every choice is a trade-off
There is no best architecture, only architectures that fit requirements. Caching buys latency at the cost of freshness. Replication buys availability at the cost of consistency or complexity. Microservices buy team independence at the cost of operational overhead. Good designs state their trade-offs explicitly — which makes them easy to review and to revisit when requirements change.
The same logic shapes how systems grow. Scroll through one architecture as its load grows. Each step fixes the bottleneck of the step before — and each is the subject of a lecture in this module:
- 1One server
Application and database share one machine. It is cheap, simple and perfectly fine for an early product — but any failure, deploy or traffic spike takes everything down.
- 2Separate the database
Moving the database to its own server lets each tier get the resources it needs: CPU for the application, memory and fast disks for the database.
- 3A load balancer and a fleet
With stateless application servers behind a load balancer, capacity grows by adding machines, and health checks route around any server that fails.
- 4A cache
An in-memory cache such as Redis answers repeated reads in about a millisecond, so most requests never reach the database.
- 5Replicas, then shards
Read replicas spread read traffic and stand by for failover. When writes or data outgrow a single leader, sharding splits the data across several databases.
- 6A queue for slow work
Emails, reports and third-party calls go onto a queue. Requests answer quickly; workers process the jobs reliably, and can be scaled on their own.
- 7A CDN at the edge
Images, scripts and cacheable pages are served from locations near users. Every step here added cost and complexity — which is why each is added only when the numbers demand it.
In the rest of this module we apply the core scaling techniques one at a time, starting with the simplest and most common: putting a load balancer in front of several servers.
Key takeaways
- Every design decision trades something — latency, consistency, cost, complexity — so requirements must say what matters most.
- Availability targets translate into downtime budgets; each extra nine is roughly ten times harder.
- Components in series multiply their availabilities; redundancy in parallel is how you climb above any single component.
- Rough estimates — requests per second, storage per year, peak load — rule designs in or out before anyone draws a box.
Check your understanding
3 questions · answers explainedQ1A service promises 99.9% availability. Roughly how much downtime does that allow per year?
Q2A request passes through a load balancer (99.99%), an app server (99.9%) and a database (99.95%). What is the approximate end-to-end availability?
Q3An app has 10 million daily active users, each making about 20 requests a day. Roughly what is the average request rate?