The Normal Distribution & the Central Limit Theorem

Probability & Statistics Fundamentals

Chapter 8 · The Normal Distribution & the Central Limit Theorem

Chapters 6 and 7 covered discrete distributions — counting successes, counting events. The normal distribution is continuous — a measurement like response time or height can take any value along a range, not just whole numbers. It's the single most common shape in all of statistics, for a reason this chapter's second half explains directly.

The Bell Curve, Defined by Two Numbers

A normal distribution is completely determined by just two parameters: its mean μ (where the peak sits) and its standard deviation σ (how wide the spread is). Its shape is always the same symmetric bell — only the center and width change.

The empirical rule (68-95-99.7)

For any normal distribution: about 68% of values fall within 1 standard deviation of the mean, about 95% within 2 standard deviations, and about 99.7% within 3.

Z-Scores — Standardizing Any Normal Distribution

A z-score converts any value from any normal distribution into "how many standard deviations from the mean" — a universal, distribution-independent scale:

Z-score formula

z = (x − μ) / σ

Once a value is expressed as a z-score, its probability can be looked up against the standard normal distribution (μ=0, σ=1) — the same lookup works for literally any normal distribution, regardless of its own original mean and standard deviation.

Worked Example: Server Response Times

Response times are normally distributed with μ = 200ms, σ = 30ms. What fraction of requests take longer than 260ms?

Step	Calculation	Result
z-score for 260ms	(260 − 200) / 30	z = 2
Empirical rule estimate	95% within ±2σ → 5% outside, split evenly	≈ 2.5% above 260ms
Precise value	1 − standard normal CDF(2)	≈ 2.275%

The empirical rule's quick estimate (2.5%) and the precise calculation (2.275%) are close, as expected — the empirical rule is a fast approximation, useful for a sanity check even when reaching for the precise number.

Getting the precise number without calculus

The exact normal distribution probability technically requires integrating its density curve — genuine calculus, deliberately out of scope for this course per Chapter 1. In practice, nobody derives that integral by hand: the standard tool is either a printed z-table, or (in code) the error function, erf, which Python's standard library already implements — no calculus knowledge required to use it correctly.

The Central Limit Theorem

Here's the genuinely remarkable part: it doesn't matter what the underlying distribution looks like. Take repeated samples of size n from any distribution — normal, Poisson, wildly skewed, anything with a finite mean and variance — and compute each sample's own average. As n grows, the distribution of those averages approaches a normal distribution, centered on the true population mean.

The Central Limit Theorem (CLT)

For samples of size n drawn from a distribution with mean μ and variance σ², the sample mean's own distribution approaches Normal(μ, σ²/n) as n grows — regardless of the original distribution's shape.

That σ²/n term is the key: the sample mean's own variance shrinks as n grows, meaning larger samples produce averages that cluster more and more tightly around the true mean.

Illustrating with a single die roll (X, uniform, not remotely normal): Var(X) = 35/12 ≈ 2.917.

Sample size n	Var(sample mean) = σ²/n	SD(sample mean)
10	0.2917	≈ 0.540
100	0.02917	≈ 0.171

Why this matters — and where it's headed next

A single die roll is as far from a bell curve as a distribution gets — flat, uniform, no peak at all. Yet the average of many rolls behaves almost normally, and gets more tightly clustered around 3.5 the more rolls are averaged. This exact σ/√n quantity — the standard deviation of a sample mean — has its own name, standard error, and it's the direct foundation of confidence intervals: Probability & Statistics Fundamentals' own sibling course, Statistical Inference & Applied Statistics, builds its entire second chapter on precisely this idea.

The Normal Distribution & CLT in Code

import math

def standard_normal_cdf(z):
    return 0.5 * (1 + math.erf(z / math.sqrt(2)))

def z_score(x, mu, sigma):
    return (x - mu) / sigma

z = z_score(260, mu=200, sigma=30)
print(1 - standard_normal_cdf(z))  # 0.02275 — matches the worked example

# CLT: standard error shrinks as sample size grows
def standard_error(sigma, n):
    return sigma / math.sqrt(n)

die_sd = math.sqrt(35 / 12)
print(standard_error(die_sd, 10))   # 0.540...
print(standard_error(die_sd, 100))  # 0.171...

Hands-On Exercises

Exercise 1

Request latency is normally distributed with μ = 150ms, σ = 20ms. Compute the z-score for 110ms, use the empirical rule to estimate the probability of a request taking less than 110ms, then compute the precise probability using the standard normal CDF. Compare the two estimates.

📄 View solution

Exercise 2

A metric has a population standard deviation of σ = 6. Using this chapter's own standard error formula, compute the standard deviation of the sample mean for sample sizes n = 9, n = 36, and n = 144. Describe the pattern you notice in how much n has to grow to halve the standard error.

📄 View solution

Exercise 3

Chapter 5's own weekly-incident-cost example had a highly skewed distribution (mean $1,500, standard deviation ≈ $7,057) — nothing close to a normal bell curve. Using this chapter's own Central Limit Theorem, explain why the average cost computed across many independent weeks would behave far more predictably (more tightly clustered, more normally shaped) than any single week's own cost, even though individual weeks are wildly unpredictable.

📄 View solution

Chapter 8 Quick Reference

Normal distribution: a continuous, symmetric bell curve fully defined by μ (mean) and σ (standard deviation)
Empirical rule: ≈68% within 1σ, ≈95% within 2σ, ≈99.7% within 3σ
Z-score: z = (x − μ) / σ — converts any normal value onto the universal standard normal scale
Precise probabilities come from the standard normal CDF (via math.erf in Python) — no calculus knowledge required to use it
Central Limit Theorem: the sample mean's distribution approaches Normal(μ, σ²/n) as n grows, regardless of the original distribution's shape
Standard error (σ/√n) is the sample mean's own standard deviation — shrinks as sample size grows, and is the direct foundation of confidence intervals in this subject's own next course
Next chapter: Descriptive statistics — mean, median, variance & standard deviation