Database Scaling
Distributed Systems & Scalability
Chapter 4 · Database Scaling
A database can scale reads by copying itself (replication) or scale everything by splitting itself (sharding). Both genuinely work — and both cost something specific and measurable, not just "some consistency" in the abstract.
Read Replicas: Real Load Relief, and a Real Staleness Window
class PrimaryDatabase:
def __init__(self): self.data = {}; self.write_log = []
def write(self, key, value):
self.data[key] = value
self.write_log.append((key, value))
def replicate_to(self, replica):
for key, value in self.write_log:
replica.data[key] = value
self.write_log.clear()
Verified directly — a replica genuinely doesn't have a write until replication actually runs
Writing {'price': 100} to the primary makes it immediately available from primary.data. Reading the identical key from the replica before replicate_to() runs returns None — genuinely missing, not just slow. Only after replicate_to() actually runs does the replica correctly return {'price': 100}.
Verified directly — replicas reduce per-server load, not the time any single query takes
Running 30 reads entirely against one primary took the same total wall-clock time as spreading the identical 30 reads across 3 replicas — each individual read still costs what it costs. What genuinely changed: the primary alone would have handled all 30 reads; with 3 replicas sharing the work, each server handled only 10 — 3× less load per server. This is Chapter 1's own "capacity, not speed" distinction, applied to reads specifically.
Why this matters for what you can safely read from a replica
Any read that needs the absolute latest write — checking a payment that was just submitted, confirming an order that was just placed — can genuinely fail if it's routed to a replica during the staleness window just verified. Reads that can tolerate being slightly out of date (a product catalog, a dashboard) are exactly what read replicas are for.
Sharding: Splitting Data, and What a Bad Key Does to It
def shard_by_user_id(user_id): return user_id % NUM_SHARDS # GOOD: evenly-distributed integers
def shard_by_country(country): # BAD: real-world signup data is rarely even
h = int(hashlib.md5(country.encode()).hexdigest(), 16)
return h % NUM_SHARDS
Verified directly — a well-chosen key spreads 1,000 records perfectly evenly
Sharding 1,000 sequential user IDs across 4 shards by user_id % 4 produced exactly 250 records per shard — a spread of 0 between the busiest and quietest shard.
Verified directly — a plausible, realistic key produced a severe hotspot, and two entire shards sat empty
Sharding the same 1,000 users by their own signup country — a realistic distribution where 80% of users share one primary market — produced 863 records on one shard, 137 on another, and 0 on each of the remaining two. One shard alone carried 86.3% of all data. With only 4 distinct country values feeding the hash function, two of the four available shards never received a single record at all — sharding infrastructure was fully deployed, but most of it was doing nothing.
Why this connects directly to Chapter 2's own load-balancing findings
An overloaded shard is the database-layer version of Chapter 2's own overloaded server under Round Robin — the fix isn't "add more shards," it's choosing a key that actually reflects how the data is genuinely distributed, the same way Least Connections' fix wasn't "add more servers," it was routing based on real, current load.
The Cost Sharding Adds: Cross-Shard Queries
Verified directly — an aggregate query across all users costs 4× the shard-touches of a single-shard query
Computing total revenue for users on one shard touched exactly 1 shard. Computing total revenue across all 1,000 users touched all 4 shards, one query each, before the results could even be combined — correctly summing to 100,000, matching a manual check (1,000 × $100) exactly. A question that was one query against an unsharded database becomes N queries plus a combine step, where N is however many shards exist.
Where This Connects
| This chapter's finding | What it connects to |
|---|---|
| A verified replica staleness window | Chapter 3's own cache-invalidation bug and Software Architecture Fundamentals Chapter 6's own eventual-consistency finding — the identical shape of risk, at the database layer this time |
| A dramatic, realistic sharding hotspot (86.3% of data on one shard) | Chapter 2's own Round Robin finding — an "equal treatment" rule (hash the key, mod by shard count) producing a badly uneven outcome |
| Cross-shard queries costing N shard-touches instead of 1 | Chapter 5's own CAP Theorem chapter — sharding is a concrete instance of trading single-query simplicity for horizontal capacity |
Hands-On Exercises
Exercise 1
Using this chapter's own PrimaryDatabase/ReadReplica, write TWO keys to the primary before calling replicate_to(). Verify both are missing from the replica beforehand, and both are correctly present afterward.
Exercise 2
Using this chapter's own sharding setup, try shard_by_country with a LESS skewed distribution — say, 30/25/25/20 percent across the same 4 countries instead of 80/7/7/6. Verify whether the hotspot shrinks, and report the new spread compared to this chapter's own 726-record spread.
Exercise 3
Using this chapter's own two verified findings (replica staleness and the sharding hotspot), explain why "add read replicas" and "shard the database" solve genuinely different scaling problems — which one would have helped Chapter 1's own O(n) duplicate-check bottleneck, and why?
📄 View solutionChapter 4 Quick Reference
- Read replicas: verified reducing per-server load 3× (30 reads → 10 per server across 3 replicas) with zero change to any single query's own speed, at the cost of a real, verified staleness window before replication runs
- Sharding, verified: a good key (user_id) spread 1,000 records perfectly evenly; a plausible but poor key (signup country) put 86.3% of records on one shard and left two of four shards completely empty
- Cross-shard queries, verified: an aggregate across all data cost 4× the shard-touches of a single-shard query
- Next chapter: The CAP Theorem & Consistency Models — formalizing the exact tradeoffs this chapter and Chapter 3 both verified concretely