Descriptive Statistics — Mean, Median, Variance & Standard Deviation

Probability & Statistics Fundamentals

Chapter 9 · Descriptive Statistics: Mean, Median, Variance & Standard Deviation

Every chapter so far started with a known model and predicted outcomes — the probability direction from Chapter 1's own opening table. This chapter finally works the other way: given real, already-collected data, how do you summarize it fairly? This is genuinely different math from Chapter 5's variance, not just a repeat of it — Chapter 5 computed statistics from a known theoretical distribution; this chapter computes them directly from actual numbers you've measured.

Mean & Median

The mean is the familiar average: sum every value, divide by the count. The median is the middle value once the data is sorted — the average of the two middle values if there's an even count.

Mean and median formulas

mean = (Σ x) / n median = the middle value of the sorted data (average of the two middle values if n is even)

Worked Example: When One Slow Request Skews Everything

Seven real request response times (ms): 120, 115, 130, 125, 118, 122, 890 — six ordinary requests, and one genuinely slow one.

Dataset	Mean	Median
All 7 values (with the 890ms outlier)	231.43ms	122ms
Just the 6 ordinary values	121.67ms	121ms

The mean moved by over 100ms; the median barely moved at all

A single outlier dragged the mean from 121.67ms up to 231.43ms — nearly double — while the median stayed almost exactly where it was (121ms → 122ms). This is exactly the dashboard trap named back in Chapter 1: an "average response time" panel can be badly distorted by a handful of extreme values, making the service look far worse (or, in other contexts, far better) than what most real requests actually experienced. The median is far more resistant to outliers, since it only cares about the middle position, not the extreme values' actual size.

Sample Variance & Standard Deviation

Chapter 5 defined variance for a known random variable: Var(X) = E[X²] − (E[X])². For real, already-collected sample data, the formula looks similar but has one crucial, easy-to-miss difference:

Sample variance formula

s² = Σ(x − x̄)² / (n − 1) s = √s² (sample standard deviation)

Why n − 1, not n — Bessel's correction

Dividing by n would use the sample's own mean (x̄, estimated from the same limited data) as if it were the true population mean — and a sample's mean is, by construction, the value that minimizes the sum of squared deviations for that specific sample, which means dividing by n systematically underestimates the true population variance. Dividing by n − 1 instead corrects for that bias. In practice, unless you genuinely have every single data point that will ever exist (the entire population, not a sample of it), n − 1 is almost always the correct choice.

Applying this to the same two datasets from above:

Dataset	Sample variance (s²)	Sample std dev (s)
All 7 values (with outlier)	84,357.29	≈ 290.44ms
Just the 6 ordinary values	28.27	≈ 5.32ms

Variance and standard deviation are hit even harder by the outlier than the mean was — the single 890ms value inflates the standard deviation more than fiftyfold, from ≈5.32ms to ≈290ms. Squaring deviations, as the formula does, punishes large outliers disproportionately.

Descriptive Statistics in Code

import statistics as stats

data = [120, 115, 130, 125, 118, 122, 890]

print(stats.mean(data))      # 231.4285... 
print(stats.median(data))    # 122
print(stats.variance(data))  # 84357.29... — Python's stdlib already uses n-1
print(stats.stdev(data))     # 290.44...

# Implementing sample variance from scratch, to see the n-1 explicitly
def sample_variance(data):
    mean = sum(data) / len(data)
    return sum((x - mean) ** 2 for x in data) / (len(data) - 1)

print(sample_variance(data))  # matches stats.variance(data)

Hands-On Exercises

Exercise 1

CPU usage samples from six consecutive checks (%): 45, 48, 50, 47, 46, 95. Compute the mean and the median. Which one better represents a "typical" reading from this dataset, and why, per this chapter's own outlier-resistance finding?

📄 View solution

Exercise 2

Given the sample 10, 12, 14, compute the mean, then the sample variance using this chapter's own n − 1 formula, showing each squared deviation separately, and finally the sample standard deviation.

📄 View solution

Exercise 3

Explain, in your own words, why a data engineer analyzing a sample of 10,000 user sessions out of millions that occurred should almost always use the n − 1 sample variance formula rather than dividing by n — and describe the one specific circumstance (per this chapter's own Bessel's-correction explanation) where dividing by plain n would actually be the mathematically correct choice instead.

📄 View solution

Chapter 9 Quick Reference

Mean: sum divided by count; median: the middle value of sorted data — far more resistant to outliers than the mean
A single extreme outlier can drag the mean dramatically while barely moving the median — the exact mechanism behind the "average hides a real spike" dashboard trap named in Chapter 1
Sample variance: s² = Σ(x−x̄)² / (n−1) — divides by n−1 (Bessel's correction), not n, whenever the data is a sample rather than the entire population
Squaring deviations means variance and standard deviation are even more sensitive to outliers than the mean is
This chapter's statistics are computed from real, already-collected data — distinct from Chapter 5's variance of a known theoretical random variable
Next chapter: Capstone — probability & statistics in practice