journald & Reading Logs

systemd1-1's own warn-box named journald as part of systemd's broader scope beyond process supervision. This chapter is that promised depth — and it's also the single-host counterpart to obs1-7's own centralized logging material.

What journald Actually Is

systemd's own built-in logging daemon. It automatically captures stdout/stderr from every unit systemd manages, plus kernel messages, plus anything using the standard syslog() call — with no separate logging setup required for a systemd-managed service. Storage is structured and indexed, not plain text like a traditional /var/log/*.log file — queryable directly by unit, priority, or time range, without reaching for grep/awk gymnastics.

journalctl — The Basic Query Tool

-u myapp ties directly to systemd1-3's own worked example. -f is genuinely comparable to tail -f on a traditional log file. -b -1 is a real, practical tool for diagnosing a crash or unexpected reboot after the fact — the previous boot's own logs, isolated from everything since.

Persistent vs. Volatile Logging

By default on many systems, the journal lives in /run/log/journal/ — RAM-backed, volatile, wiped completely on reboot. This has a real, practical consequence: journalctl -b -1 only works at all if persistent storage is actually configured; without it, the previous boot's logs are simply gone.

Creating /var/log/journal/ with correct ownership is the actual, real-world trigger that switches journald into persistent mode automatically — logs now survive reboots. This matters directly for the same reason ws1's own security material cared about logging at all: a system with only volatile logs loses everything the moment it's rebooted, a real, practical gap for any kind of incident investigation after the fact.

Priority Filtering

The classic 8-level syslog priority scale — emerg, alert, crit, err, warning, notice, info, debug, ordered from most to least severe. -p err shows entries at error level or more severe — the scale is ordered, so this filters "at least this severe," not "exactly this level." A real, practical way to find genuinely serious issues in a noisy, chatty log without reading every routine info-level line.

The Single-Host Counterpart to obs1-7's Own Centralized Logging

obs1-7 covered structured logging and centralized aggregation (Loki) across many hosts and services, at real scale. journald solves a genuinely related but narrower problem — one host's own logs, structured and queryable, with no separate aggregation system required at all. The same underlying idea — structured, queryable logs beat unstructured, grep-able text files — applies at both scales, just with different tooling for different scopes. Real production setups often use both: journald locally on each host, with an agent (the same DaemonSet-based approach obs1-10 described) shipping journal entries onward to centralized storage. journald isn't replaced by Loki — it's frequently the local source a Loki-equivalent agent reads from in the first place.

Hands-On Exercises