Exercise 3: An Intrusion-Detection System and Alert Fatigue — Possible Solution ==================================================================== GIVEN ------------------------------ P(alert | malicious) = 0.99 P(alert | not malicious) = 0.01 P(malicious) = 1/10,000 = 0.0001 STEP 1: THE LAW OF TOTAL PROBABILITY — P(alert) ------------------------------ P(alert) = P(alert|malicious) x P(malicious) + P(alert|not malicious) x P(not malicious) = (0.99 x 0.0001) + (0.01 x 0.9999) = 0.000099 + 0.009999 = 0.010098 STEP 2: APPLYING BAYES' THEOREM ------------------------------ P(malicious|alert) = [P(alert|malicious) x P(malicious)] / P(alert) = 0.000099 / 0.010098 ~= 0.0098 (about 0.98%, roughly 1 in 102) WHAT THIS MEANS FOR THE SYSTEM'S DESIGN ------------------------------ Despite sounding excellent on paper (99% true-positive rate, only 1% false-positive rate), fewer than 1 in 100 alerts this system generates actually correspond to a real malicious connection - the other ~99% are false alarms. This is this chapter's own base-rate-neglect effect taken to an extreme, because malicious connections are so rare (1 in 10,000) relative to the enormous volume of ordinary traffic that even a small 1% false-positive rate produces vastly more false alerts than true ones in absolute terms. This system is NOT "good enough" to alert a human on every single trigger as-is - doing so would produce overwhelming alert fatigue, exactly the operational problem this chapter names directly. A practical fix requires either a dramatically lower false-positive rate (extremely hard to achieve), combining the alert with additional independent signals before escalating to a human (effectively repeating this chapter's own Bayesian-updating process with more evidence to sharpen the posterior further), or automating a first-pass triage step rather than treating every alert as equally worth immediate human attention. WHY THIS WORKS AS AN ANSWER ------------------------------ The posterior probability is computed using this chapter's own two- step Bayes' Theorem method, and the practical conclusion is drawn directly from this chapter's own base-rate-neglect / alert-fatigue finding rather than treating the 99%/1% headline accuracy figures as sufficient on their own to judge the system's real-world usefulness.