What Observability Actually Means — The Three Pillars

cloud1-8 introduced the vocabulary — metrics, logs, traces, automatic vs. configured monitoring, alert fatigue. cloud2-4 built real incident-response discipline on top of that vocabulary — the first-five-minutes triage habit, cross-service correlation IDs, "find the earliest alert, not the loudest." Both stayed conceptual and tool-agnostic, on purpose. This course goes deep specifically on the real tooling underneath — Prometheus, Grafana, Loki, OpenTelemetry, Jaeger — the systems that actually produce the metrics, logs, and traces those two chapters assumed were already there.

Monitoring vs. Observability — A Real Distinction

Monitoring watches for failure modes you already anticipated — a dashboard for CPU usage, an alert for disk space below 10%, a check for "is the process running." It answers questions decided in advance. Observability is different: the ability to ask a new question about a system's internal state, using only its external outputs, without shipping new code to answer it. cloud2-4's own "first five minutes" already implied this — during a genuinely novel incident, you usually don't know what to look for yet. Monitoring gives you the dashboards someone thought to build in advance; observability gives you the raw material to explore a question nobody anticipated.

The Three Pillars

Three distinct categories of telemetry, each answering a different kind of question:

• Metrics — numeric measurements over time (request rate, CPU%, P99 latency). Cheap to store, efficient to aggregate, ideal for trends and alerting — but they summarize; a metric can't tell you what happened to one specific failed request.
• Logs — discrete, timestamped event records, as rich and arbitrary as the code that emits them. Detailed, but expensive to store and query at real scale, and hard to correlate across services without deliberate effort — exactly the correlation-ID discipline cloud2-4 already named.
• Traces — the path one specific request takes across every service it touches, with timing for each hop. The pillar built specifically to answer "where, in this whole chain, did the time actually go?"

A Concrete Walkthrough — One Request, Three Views

A checkout request is slow. Each pillar answers a different piece of the same incident:

• Metrics say: P99 checkout latency has climbed from 200ms to 4s over the last ten minutes — a trend, aggregated across every request, telling you something is wrong and roughly when it started.
• Logs say: the payment service logged three timeout errors and a retry in the last minute — detail, but only for the services that happened to log something, and only if you already know which service to look at.
• Traces say: for this specific slow request, 3.8 of the 4 seconds were spent waiting on a single downstream call to the inventory service — the one view that actually pinpoints where in the chain the time went, rather than just confirming that some slowdown exists.

No single pillar answers the whole question alone. Metrics tell you something is wrong; logs tell you what a given service saw; traces tell you where, across the whole request's path, the problem actually lives.

Hands-On Exercises