Working an Incident: Keeping a Live Timeline

Incident Response & Ticketing Workflows

Chapter 5 · Working an Incident: Keeping a Live Timeline

Chapter 1 promised this would be the single highest-payoff habit in the whole course. Here's why, and exactly how to actually do it: a timeline is a running, timestamped log of what was checked and what was found, written down as it happens — not reconstructed afterward from memory.

Why "Live" Specifically Matters

Human memory doesn't reconstruct events in true chronological order after the fact — it fills gaps with plausible-sounding assumptions shaped by how the story eventually made sense, not by what was actually observed at each moment. A timeline captured live avoids this failure mode entirely, because it records the real sequence and the real findings at the moment they occurred, before anyone knows how the incident will end.

What Belongs in an Entry

Timestamp, what was checked or done, what was found — and who, if more than one person is involved. Terse and factual; a timeline is an evidence log, not prose.

02 UTC — Ticket filed: checkout API intermittently returning 500s since ~13:45 UTC.
05 UTC — [J. Lee] Checked CPU/memory on checkout-api-3: both normal. Not a resource issue.
11 UTC — [J. Lee] Checked network path to payment gateway: clean, no packet loss.
14 UTC — [J. Lee] Theory: 13:40 deploy introduced the bug. Checking version skew across instances.
19 UTC — [J. Lee] All 8 instances report the same version — ruling out version skew.
23 UTC — [J. Lee] Checked connection pool metrics: active=20/20, waiting=9. Pool exhaustion confirmed.
27 UTC — [J. Lee] pg_stat_activity shows one query stuck 8+ min on order_items table scan.
31 UTC — [J. Lee] Confirmed missing index on order_items.customer_id, added in a post-13:40 schema change.
34 UTC — Index added; pool recovering; error rate back to baseline by 14:38 UTC.

A Timeline Is Not the Same as a Narrative

Notice the entries at 14:14 and 14:19 above: a genuinely reasonable theory (version skew) was checked and ruled out before the real cause was found. That's not a mistake to edit out afterward — it's exactly what the timeline is supposed to preserve.

A messy timeline is normal, not embarrassing

An incident that looks clean and linear in hindsight almost never was, in the moment. Wrong theories checked and ruled out are a completely normal part of real diagnosis — they belong in the record exactly as they happened, not smoothed away to make the investigation look more efficient than it actually was. This "mess" is precisely what Chapter 9's post-incident review needs: it's what actually made the incident hard to diagnose, not a footnote to be tidied up.

Where to Keep It: Low Friction Beats "Correct" Tooling

The best timeline tool is whichever one is fast enough that people actually use it while the incident is happening — a shared document, a dedicated incident chat thread with timestamped messages, or a purpose-built incident-management tool are all genuinely fine. The worst timeline tool is a more "proper" one nobody actually updates in the moment because it's too much friction to reach for mid-incident.

A Live Coordination Tool, Not Just a Historical Record

During a multi-person incident, a shared live timeline does real-time work too: it prevents two people independently checking the exact same thing because neither knew the other already had, and it lets someone joining partway through catch up by reading the timeline instead of interrupting people who are actively working to ask "what's been checked so far?"

Don't wait until you're sure something is significant

A real, common trap: holding off on writing something down until you're confident it'll matter. This quietly loses most of the timeline's own value, since you genuinely can't reliably predict in the moment which detail will turn out to be important later — the exact same lesson log1 teaches about not knowing in advance which log line matters. Write it down as you check it, ruled-out theories included, before you know how the story ends.

Hands-On Exercises

Exercise 1

Explain why this chapter says human memory is a genuinely poor substitute for a timeline captured live, even for someone with an excellent memory.

📄 View solution

Exercise 2

Using this chapter's own worked timeline, explain why the version-skew theory entries at 14:14 and 14:19 should stay in the record rather than being removed once the real cause was found.

📄 View solution

Exercise 3

Explain the two separate benefits a shared live timeline provides during a multi-person incident, beyond serving as a historical record afterward.

📄 View solution

Chapter 5 Quick Reference

A timeline = timestamp + what was checked + what was found, captured live, not reconstructed afterward
Memory reconstructs events non-chronologically after the fact — a live timeline avoids that specific failure mode
Keep ruled-out theories in the record — a messy timeline is normal, not something to clean up
Pick whichever tool is low-friction enough to actually use during the incident — that beats a "more correct" tool nobody updates
During a multi-person incident, a shared timeline also prevents duplicated effort and lets latecomers catch up without interrupting
Write it down as you check it — you can't predict in the moment which detail will matter later
Next chapter: Mitigate vs. Fix: Buying Time Safely Without Losing the Evidence