Metrics & the Prometheus Data Model

obs1-1 named metrics as the pillar that tells you something is wrong, cheaply, in aggregate. This chapter goes underneath that claim: exactly how Prometheus — the tool this course builds around for the metrics pillar — actually represents a metric. Everything in Ch.3 (scraping) and Ch.4 (PromQL) builds directly on the data model established here.

A Metric Is a Time Series

A Prometheus metric isn't one number — it's a named stream of (timestamp, value) pairs collected over time, identified by a metric name plus a set of labels, key-value pairs attached to that specific series.

http_requests_total is the metric name; the three labels together identify exactly which time series this particular value belongs to. This isn't one running total for the whole application — it's one time series among potentially many under the same metric name.

Labels — The Dimension That Makes Aggregation Possible

Every unique combination of label values creates a genuinely separate time series:

Three distinct time series, one metric name. This is exactly what makes Chapter 4's PromQL genuinely useful — labels are the dimension you filter and aggregate along, letting a single instrumented metric answer "what's my GET rate," "what's my 500 rate," and "what's my total request rate across every method and status combined" from the same underlying data.

The Four Metric Types

Prometheus defines exactly four kinds of metric, each with a different shape of value and a different intended use:

• Counter — only ever increases (or resets to zero on restart). Right for "total requests served," "total errors," anything that's a running cumulative count. A raw counter value is rarely useful by itself; Chapter 4's rate() turns it into a meaningful per-second figure.
• Gauge — a value that can go up or down freely. Right for "current memory usage," "active connections," "queue depth right now."
• Histogram — sorts observations (like request durations) into configurable buckets, alongside a running sum and count. Quantiles are computed later, at query time, from the bucket counts.
• Summary — similar intent to a histogram, but quantiles are calculated client-side, inside the instrumented application itself, before the metric is ever exposed.

Histogram vs. Summary — A Real Operational Tradeoff

These two look similar but behave very differently once there's more than one instance of a service running. A histogram's raw bucket counts from many different pods can simply be summed together — Chapter 4's histogram_quantile() then computes one fleet-wide p99 from the combined buckets. A summary's quantile, by contrast, is already computed inside one specific instance before it's ever exposed — there is no meaningful way to average two different instances' own p99 values together and get a real fleet-wide p99. For anything that might ever need aggregating across replicas — which, in practice, is nearly everything in a real production system — histograms are the safer default.

Hands-On Exercises