Every system you build eventually runs into something it can't fully control — a request that might fail, a user who might click, a server that might go down at an inconvenient hour. Probability and statistics are the two branches of math built specifically for reasoning carefully about exactly that kind of uncertainty, rather than guessing or hoping it averages out.

Probability vs. Statistics — Two Directions of Reasoning

These two names get used almost interchangeably in casual conversation, but they answer genuinely opposite questions.

Both matter, and in practice they're used together constantly — you build a probability model of how something should behave, then use statistics to check whether real, observed data actually matches it. This particular course focuses on the probability side and the basic descriptive tools for summarizing data; the reasoning-from-data-back-to-conclusions side (confidence intervals, hypothesis testing, A/B testing, regression) is deliberately this subject's own separate next course.

Five Concrete Connections to Code Already On This Site

Every topic in this course maps onto something already relevant to real engineering work, whether or not the underlying math was ever named:

What This Course Won't Cover

A substantial, genuinely related set of topics is deliberately left for this subject's own next course, Statistical Inference & Applied Statistics, rather than folded in here:

• Sampling & confidence intervals — how to reason honestly about uncertainty in an estimate drawn from a sample rather than an entire population
• Hypothesis testing & A/B testing — the formal machinery for deciding whether an observed difference is real or just noise
• Correlation, regression & Bayesian updating — inferring relationships between variables from real data, and formally updating a belief as new evidence arrives

Where This Course Is Headed

Hands-On Exercises

Every probability question starts by pinning down two things precisely: every outcome that could possibly happen, and the specific outcomes you actually care about. Get those two definitions right, and most basic probability calculations turn into simple counting — genuinely just set operations wearing a different name.

Sample Space & Events

The sample space (usually written S) is the set of every possible outcome of an experiment. An event is any subset of that sample space — the specific outcomes that count as a "success" for whatever question is being asked.

Computing Probability for Equally Likely Outcomes

When every outcome in the sample space is equally likely (a fair die, a fair coin), probability is a straightforward ratio:

For event A above: P(A) = |{2,4,6}| / |{1,...,6}| = 3/6 = 0.5. For event B: P(B) = |{5,6}| / 6 = 2/6 = 1/3.

The Probability Axioms

Every valid probability assignment must satisfy three basic rules, regardless of how the probabilities were derived:

The Complement Rule

The complement of an event, written Aᶜ ("not A"), is every outcome not in A. Since A and Aᶜ together always cover the whole sample space exactly once:

The probability of not rolling a 6: P(not 6) = 1 − 1/6 = 5/6. Often faster than counting the complement's outcomes directly, especially when "not A" covers far more cases than "A" does.

The Union (Addition) Rule

For "A or B," the naive instinct is to add P(A) + P(B) — but that double-counts any outcome that belongs to both events. The correct rule subtracts the overlap back out:

Continuing the dice example: A ∩ B = {6} (even and greater than 4), so P(A ∩ B) = 1/6.

Mutually Exclusive Events — When the Simple Sum Is Correct

If two events can never both happen at once (A ∩ B = ∅), they're called mutually exclusive — and the subtraction term simply vanishes, since P(∅) = 0:

Rolling a 1 (C = {1}) and rolling a 6 (D = {6}) can never both happen on the same roll: P(C ∪ D) = 1/6 + 1/6 = 1/3 — directly correct here, with no overlap to subtract.

Sample Spaces & Events in Code

Hands-On Exercises

Chapter 2's rules all assumed no extra information. Real questions are rarely that clean — "what's the chance this request fails, given that the last three also failed?" This chapter is about updating a probability once new information narrows down what's actually possible.

Conditional Probability — Narrowing the Sample Space

Conditional probability, written P(A|B) ("the probability of A, given B"), asks: once we know B has happened, what fraction of that narrowed-down world does A still cover?

Reusing Chapter 2's dice example: S = {1,...,6}, A = "even" = {2,4,6}, B = "greater than 4" = {5,6}, A ∩ B = {6}.

Independence

Two events are independent if knowing one happened tells you nothing new about the other: P(A|B) = P(A). Rearranging the conditional probability formula gives an equivalent, more practical test:

For contrast, reusing Chapter 2's own exercise events E = "odd" = {1,3,5} and F = "less than 4" = {1,2,3}: P(E) × P(F) = 0.5 × 0.5 = 0.25, but P(E ∩ F) = P({1,3}) = 1/3 ≈ 0.333. Since 0.25 ≠ 0.333, E and F are dependent — knowing a roll is odd genuinely does change the probability it's also less than 4.

The Multiplication Rule for Independent Events, in Practice

When events genuinely are independent, the multiplication rule becomes a fast, direct tool — and it generalizes cleanly to more than two events, which is exactly what reliability calculations need.

This is the exact combination of two rules from this course so far: the multiplication rule for independent events, and Chapter 2's own complement rule, chained together to answer a genuinely practical reliability question.

The General Multiplication Rule — A Bayes' Theorem Forward Reference

Rearranging the conditional probability formula, without assuming independence, gives the fully general version:

That last equality — two different ways of writing the exact same joint probability — is the entire foundation Chapter 4's Bayes' Theorem is built from. It's what makes it possible to solve for P(B|A) when only P(A|B) is actually known, which turns out to be an extremely common real situation.

Conditional Probability & Independence in Code

Hands-On Exercises

Chapter 3 ended with a promise: a way to solve for P(B|A) when only P(A|B) is actually known. That tool is Bayes' Theorem — and it delivers one of the most consistently surprising, genuinely useful results in this entire course.

Deriving Bayes' Theorem

Chapter 3's general multiplication rule gave two equal expressions for the same joint probability:

Dropping the middle term and dividing both sides by P(A) isolates P(B|A) directly:

Written in the notation it's usually taught with — H for hypothesis, E for evidence:

Each term has a name worth knowing: P(H) is the prior (what you believed before seeing the evidence), P(E|H) is the likelihood, P(E) is the evidence's overall probability, and P(H|E) is the posterior — the updated belief after accounting for the evidence.

The Law of Total Probability — Filling in the Denominator

P(E) is rarely given directly — it needs to be built from the two ways evidence can occur: when the hypothesis is true, and when it's false.

The Classic Example: A Diagnostic Test

A test for a rare disease is genuinely quite accurate:

Question: given a positive result, what's the actual probability of having the disease?

A Second Worked Example: A Spam Filter

Reusing this course's own forward reference from Chapter 1 — a spam filter reasoning about a single word:

Seeing "free" in an email raises the belief it's spam from a 40% prior all the way to an 80% posterior — a real, quantified update, exactly the mechanism named in Chapter 1's own connections table.

Why This Matters for Real Alerting Systems

Bayes' Theorem in Code

Hands-On Exercises

Every chapter so far has asked about the probability of a specific event. This chapter asks a slightly different question: given a whole range of numeric outcomes, each with its own probability, what's a single number that fairly summarizes the whole situation? That number is the expected value, and it's the mathematical foundation of essentially every risk, cost, and budgeting calculation built on top of uncertainty.

Random Variables & Probability Distributions

A random variable (conventionally written as a capital letter, like X) assigns a number to every outcome in a sample space. Its probability distribution (or probability mass function, for a discrete variable) lists every possible value it can take, together with the probability of each.

Expected Value

The expected value E[X] is the probability-weighted average of every possible outcome:

For the die: E[X] = (1+2+3+4+5+6)/6 = 21/6 = 3.5.

Worked Example: Expected Weekly Incident Cost

A service's weekly incident cost, X, has this distribution:

(Probabilities sum to exactly 1.0, as any valid distribution must — Chapter 2's own certainty axiom.)

E[X] = 0 + 100 + 400 + 1,000 = $1,500 — the expected weekly incident cost, useful directly for budgeting a reserve fund or setting an insurance premium, exactly the reasoning behind Technical Support's own backup1/incident1 material approached from a numeric angle.

Variance & Standard Deviation of a Random Variable

Expected value alone hides something important: how spread out the possible outcomes actually are. Variance measures that spread directly, for a known distribution — a genuinely different calculation from Chapter 9's own variance, which is computed from real, already-collected sample data rather than a theoretical distribution.

For the incident-cost example: E[X²] = 0²(0.70) + 500²(0.20) + 5,000²(0.08) + 50,000²(0.02) = 0 + 50,000 + 2,000,000 + 50,000,000 = 52,050,000.

Random Variables & Expected Value in Code

Hands-On Exercises

Chapter 5 covered random variables in general — any list of outcomes and probabilities was fair game. This chapter is about the single most common specific shape that list takes in real engineering work: counting how many times something succeeds out of a fixed number of independent attempts, each with the same fixed probability.

When the Binomial Distribution Applies

The binomial distribution requires four conditions, all of which are worth checking explicitly before using it:

The Binomial Formula

X = the number of successes in n trials. The probability of getting exactly k successes is:

C(n, k) is "n choose k" — the number of different ways to pick which k of the n trials were the successes (the same combinations formula from Discrete Mathematics Fundamentals Chapter 9, for anyone who's taken that course; here it's just n! / (k! (n−k)!)). The rest of the formula is one specific sequence's own probability — pᵏ for the k successes, (1−p)ⁿ⁻ᵏ for the remaining failures — multiplied by how many different sequences produce that same count.

Worked Example: Flipping a Coin

Flip a fair coin n = 4 times. X = number of heads. What's P(X = 2)?

Mean & Variance — A Shortcut Built on Chapter 5

Computing E[X] and Var(X) the long way, using Chapter 5's own formulas across every possible value of k, works — but the binomial distribution has direct shortcuts that skip all of it:

For the coin-flip example: E[X] = 4 × 0.5 = 2 heads on average, Var(X) = 4 × 0.5 × 0.5 = 1.

A Practical Example: A/B Test Conversions

A new checkout flow has a 20% conversion rate (p = 0.2). Out of the next n = 10 users, what's the probability exactly 3 convert?

E[X] = 10 × 0.2 = 2 expected conversions, Var(X) = 10 × 0.2 × 0.8 = 1.6, SD(X) ≈ 1.26. This exact framing — a fixed conversion rate applied across a fixed number of users — is precisely the model behind Probability & Statistics Fundamentals' own sibling course, Statistical Inference & Applied Statistics, and its A/B testing material.

Revisiting Chapter 1's Defect-Rate Example

Chapter 1's own opening exercise asked: given a known 2% defect rate, how many defects are expected in a batch of 500? That's exactly E[X] = np = 500 × 0.02 = 10 — the binomial mean, finally formalized. The full distribution can answer sharper questions too:

The Binomial Distribution in Code

Hands-On Exercises

Chapter 6's binomial distribution needed a fixed, countable number of trials — flip the coin exactly 4 times, check exactly 500 units. Plenty of real questions don't have a natural "number of trials" at all: how many incidents hit a server in a week? How many requests arrive at an endpoint in a second? There's no fixed n to point to — just a rate. The Poisson distribution is built specifically for exactly that shape of question.

What the Poisson Distribution Models

The Poisson distribution counts how many times a rare, independent event happens within a fixed interval (of time, space, or anything else continuous), given only the average rate at which it happens — a single parameter, conventionally called λ (lambda).

e here is Euler's number (≈2.71828), the same constant used throughout continuous math; k! is a factorial, exactly Discrete Mathematics Fundamentals Chapter 9's own territory for anyone who's taken that course. λ is both the distribution's only parameter and, conveniently, its mean.

Mean, Variance — and a Distinctive Property

Worked Example: Weekly Server Incidents

A service experiences an average of λ = 3 incidents per week. What's the probability of exactly 5 incidents in a given week?

For contrast, the probability of a completely quiet, incident-free week: P(X=0) = (3⁰ × e⁻³) / 0! = e⁻³ ≈ 0.0498 — about a 5% chance, despite the average being 3 incidents. This is directly Technical Support's perfdiag1/incident1 territory, now with a precise number attached instead of just "incidents happen sometimes."

A Second Example: Request Arrival Rate

An API endpoint receives an average of λ = 2 requests per second. What's the probability of exactly 4 requests arriving in a given second?

This is the exact reasoning behind capacity planning: knowing the full distribution of "requests per second," not just the average, is what lets a team provision for the realistic worst case rather than just the mean.

The Poisson Distribution as a Limit of the Binomial

The Poisson distribution isn't an unrelated new idea — it's what the binomial distribution turns into when n gets very large and p gets very small, while their product np stays fixed at λ. This makes intuitive sense: "requests per second" has effectively infinite tiny sub-instants where a request could arrive (huge n), each individually very unlikely (tiny p), but the overall rate λ = np stays meaningful.

The two agree to four decimal places — confirming the Poisson distribution as a genuinely accurate, and far simpler, stand-in whenever n is huge and p is tiny.

The Poisson Distribution in Code

Hands-On Exercises

Chapters 6 and 7 covered discrete distributions — counting successes, counting events. The normal distribution is continuous — a measurement like response time or height can take any value along a range, not just whole numbers. It's the single most common shape in all of statistics, for a reason this chapter's second half explains directly.

The Bell Curve, Defined by Two Numbers

A normal distribution is completely determined by just two parameters: its mean μ (where the peak sits) and its standard deviation σ (how wide the spread is). Its shape is always the same symmetric bell — only the center and width change.

Z-Scores — Standardizing Any Normal Distribution

A z-score converts any value from any normal distribution into "how many standard deviations from the mean" — a universal, distribution-independent scale:

Once a value is expressed as a z-score, its probability can be looked up against the standard normal distribution (μ=0, σ=1) — the same lookup works for literally any normal distribution, regardless of its own original mean and standard deviation.

Worked Example: Server Response Times

Response times are normally distributed with μ = 200ms, σ = 30ms. What fraction of requests take longer than 260ms?

The empirical rule's quick estimate (2.5%) and the precise calculation (2.275%) are close, as expected — the empirical rule is a fast approximation, useful for a sanity check even when reaching for the precise number.

The Central Limit Theorem

Here's the genuinely remarkable part: it doesn't matter what the underlying distribution looks like. Take repeated samples of size n from any distribution — normal, Poisson, wildly skewed, anything with a finite mean and variance — and compute each sample's own average. As n grows, the distribution of those averages approaches a normal distribution, centered on the true population mean.

That σ²/n term is the key: the sample mean's own variance shrinks as n grows, meaning larger samples produce averages that cluster more and more tightly around the true mean.

Illustrating with a single die roll (X, uniform, not remotely normal): Var(X) = 35/12 ≈ 2.917.

The Normal Distribution & CLT in Code

Hands-On Exercises

Every chapter so far started with a known model and predicted outcomes — the probability direction from Chapter 1's own opening table. This chapter finally works the other way: given real, already-collected data, how do you summarize it fairly? This is genuinely different math from Chapter 5's variance, not just a repeat of it — Chapter 5 computed statistics from a known theoretical distribution; this chapter computes them directly from actual numbers you've measured.

Mean & Median

The mean is the familiar average: sum every value, divide by the count. The median is the middle value once the data is sorted — the average of the two middle values if there's an even count.

Worked Example: When One Slow Request Skews Everything

Seven real request response times (ms): 120, 115, 130, 125, 118, 122, 890 — six ordinary requests, and one genuinely slow one.

Sample Variance & Standard Deviation

Chapter 5 defined variance for a known random variable: Var(X) = E[X²] − (E[X])². For real, already-collected sample data, the formula looks similar but has one crucial, easy-to-miss difference:

Applying this to the same two datasets from above:

Variance and standard deviation are hit even harder by the outlier than the mean was — the single 890ms value inflates the standard deviation more than fiftyfold, from ≈5.32ms to ≈290ms. Squaring deviations, as the formula does, punishes large outliers disproportionately.

Descriptive Statistics in Code

Hands-On Exercises

One continuous worked project, touching every chapter of this course in the order a real engineer would actually reach for each idea: monitoring the canary rollout of a new checkout feature, from the first hour's error logs through the final response-time analysis.

A Full Worked Rollout — Monitoring a New Checkout Feature

This is, in essence, exactly what a real feature-rollout review looks like — every step traceable to a specific chapter of this course, none of it abstract math floating free of the actual monitoring dashboard.

What This Course Doesn't Cover

In the interest of an honest accounting: sampling and confidence intervals, hypothesis testing and A/B testing, and correlation, regression, and Bayesian updating were all named in Chapter 1 as deliberately out of scope, reserved for this subject's own next course, Statistical Inference & Applied Statistics. This course built the probability vocabulary and distributions every one of those techniques is built on top of, not a substitute for them.

This Course's Throughline, Restated

Where This Course Connects

This course is the direct foundation under Technical Support's own diagnostic material — perfdiag1's and incident1's handling of incident rates and monitoring dashboards is exactly Chapters 7–9's territory, applied without the underlying math ever being named explicitly there. Within this subject's own next course, Statistical Inference & Applied Statistics builds directly on this course's Chapter 8 (the Central Limit Theorem feeds its own sampling-distribution chapter) and Chapter 4 (Bayes' Theorem feeds its own Bayesian-updating chapter) — nothing here was built in isolation from where this subject is actually headed next.

Hands-On Exercises