Conditional Probability & Independence

Probability & Statistics Fundamentals

Chapter 3 · Conditional Probability & Independence

Chapter 2's rules all assumed no extra information. Real questions are rarely that clean — "what's the chance this request fails, given that the last three also failed?" This chapter is about updating a probability once new information narrows down what's actually possible.

Conditional Probability — Narrowing the Sample Space

Conditional probability, written P(A|B) ("the probability of A, given B"), asks: once we know B has happened, what fraction of that narrowed-down world does A still cover?

Conditional probability formula

P(A|B) = P(A ∩ B) / P(B)

Reusing Chapter 2's dice example: S = {1,...,6}, A = "even" = {2,4,6}, B = "greater than 4" = {5,6}, A ∩ B = {6}.

Quantity	Calculation	Result
P(A\|B)	(1/6) / (1/3)	1/2 — given the roll is 5 or 6, half the time it's also even
P(B\|A)	(1/6) / (1/2)	1/3 — given the roll is even, only 1 in 3 times is it also >4

P(A|B) ≠ P(B|A), in general

These two numbers answer genuinely different questions and there's no reason to expect them to match — here they don't (1/2 vs 1/3). Confusing the two is a classic, real mistake (sometimes called "the prosecutor's fallacy" in legal/forensic contexts): "the probability of this evidence given innocence" is not the same number as "the probability of innocence given this evidence." Chapter 4's Bayes' Theorem exists specifically to convert correctly between the two.

Independence

Two events are independent if knowing one happened tells you nothing new about the other: P(A|B) = P(A). Rearranging the conditional probability formula gives an equivalent, more practical test:

The independence test / multiplication rule

A and B are independent ⟺ P(A ∩ B) = P(A) × P(B)

A genuinely non-obvious result, checked directly

Checking Chapter 2's own A and B: P(A) × P(B) = 0.5 × (1/3) = 1/6, which exactly equals P(A ∩ B) = 1/6. "Even" and "greater than 4" are, on a fair die, actually independent — a fact that isn't obvious just by looking at the two events, and only confirmed by checking the numbers directly. Never assume independence from intuition alone; always check.

For contrast, reusing Chapter 2's own exercise events E = "odd" = {1,3,5} and F = "less than 4" = {1,2,3}: P(E) × P(F) = 0.5 × 0.5 = 0.25, but P(E ∩ F) = P({1,3}) = 1/3 ≈ 0.333. Since 0.25 ≠ 0.333, E and F are dependent — knowing a roll is odd genuinely does change the probability it's also less than 4.

The Multiplication Rule for Independent Events, in Practice

When events genuinely are independent, the multiplication rule becomes a fast, direct tool — and it generalizes cleanly to more than two events, which is exactly what reliability calculations need.

Worked example: three independent retry attempts

An operation is retried up to 3 times. Each attempt independently succeeds 90% of the time (a 10% failure rate). What's the probability all three attempts fail?

Quantity	Calculation	Result
P(all 3 fail)	0.1 × 0.1 × 0.1	0.001 (0.1%)
P(at least one succeeds)	1 − 0.001 (complement rule, Ch.2)	0.999 (99.9%)

This is the exact combination of two rules from this course so far: the multiplication rule for independent events, and Chapter 2's own complement rule, chained together to answer a genuinely practical reliability question.

The General Multiplication Rule — A Bayes' Theorem Forward Reference

Rearranging the conditional probability formula, without assuming independence, gives the fully general version:

General multiplication rule

P(A ∩ B) = P(A|B) × P(B) = P(B|A) × P(A)

That last equality — two different ways of writing the exact same joint probability — is the entire foundation Chapter 4's Bayes' Theorem is built from. It's what makes it possible to solve for P(B|A) when only P(A|B) is actually known, which turns out to be an extremely common real situation.

Conditional Probability & Independence in Code

S = {1, 2, 3, 4, 5, 6}
A = {2, 4, 6}
B = {5, 6}

def prob(event, sample_space):
    return len(event) / len(sample_space)

P_A = prob(A, S)
P_B = prob(B, S)
P_A_given_B = prob(A & B, S) / P_B

print(P_A_given_B)               # 0.5
print(P_A_given_B == P_A)       # True — independent, per this chapter's own test

# Retry reliability: independent multiplication rule
p_fail = 0.1
p_all_fail = p_fail ** 3
print(1 - p_all_fail)  # 0.999 — probability at least one of 3 attempts succeeds

Hands-On Exercises

Exercise 1

Rolling a single die: event G = "rolling a number ≤ 3" ({1,2,3}), event H = "rolling an even number" ({2,4,6}). Compute P(G|H) and P(H|G), and determine whether G and H are independent by applying this chapter's own multiplication-rule test.

📄 View solution

Exercise 2

An operation is retried up to 4 times, each attempt independently succeeding 80% of the time. Compute the probability all 4 attempts fail, and the probability at least one succeeds.

📄 View solution

Exercise 3

In a product analytics dataset: P(user is on mobile) = 0.6, P(user completes checkout) = 0.1, P(user is on mobile AND completes checkout) = 0.03. Compute P(checkout | mobile), compare it to the overall P(checkout) to say whether mobile users are more or less likely than average to complete checkout, and determine whether "on mobile" and "completes checkout" are independent events.

📄 View solution

Chapter 3 Quick Reference

Conditional probability: P(A|B) = P(A ∩ B) / P(B) — the probability of A, restricted to the world where B already happened
P(A|B) ≠ P(B|A) in general — confusing the two is a classic real mistake
Independence: P(A|B) = P(A), equivalently P(A ∩ B) = P(A) × P(B) — always check the numbers, never assume from intuition
The independent-events multiplication rule generalizes to any number of events — e.g., P(all n fail) = p^n for n independent attempts each with failure probability p
General multiplication rule: P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A) — the direct foundation of Chapter 4's Bayes' Theorem
Next chapter: Bayes' Theorem