Sustained vs. Transient Load: When a Spike Isn't a Problem

System Monitoring & Performance Diagnosis

Chapter 9 · Sustained vs. Transient Load: When a Spike Isn't a Problem

Every earlier chapter has been about confirming a genuine problem. This one is the deliberate flip side — recognizing elevated resource usage that's actually normal, bursty, expected behavior, and telling it apart from a real, sustained trend that genuinely needs action. Chapter 1's own baseline theme (an 80% CPU batch server being perfectly normal) gets its fullest treatment here.

Transient Spikes: Normal, Bursty Behavior

Plenty of legitimate causes produce a real, visible spike that isn't a problem at all: application startup (populating caches, JIT warmup), a scheduled batch job, a genuine burst of real traffic (a marketing email going out, a product launch), and — in managed-memory languages — garbage collection pauses, a normal, expected part of how the runtime reclaims memory, not inherently a bug.

"Normal" garbage collection has its own limit
A brief GC pause is routine. A GC pause that's unusually long, or happening far more frequently than normal, can itself be a genuine symptom — often of memory pressure serious enough that the runtime is working hard to reclaim space. The line between "normal GC" and "a real problem showing up as GC activity" is exactly the same kind of baseline comparison this whole course keeps returning to.

Sustained Trends: The Kind That Actually Needs Action

A genuine sustained trend looks different from a transient spike in one specific way: it doesn't fully return to where it started. Real, sustained growth usually means one of two things — organic growth (more real users, more real data, a genuine capacity question) or a resource leak (a bug causing gradual, unbounded consumption that's never actually released, even during quiet periods when load is low).

The "Floor" Technique: Comparing Quiet Periods, Not Peaks

The single most useful technique in this chapter: instead of watching the peaks, watch the low points — the quiet periods, night after night, week after week. A transient spike, no matter how dramatic, returns fully to the same floor each time. A real leak or genuine growth trend leaves a slightly higher floor after every cycle, even if the peaks themselves look similar day to day.

The floor is the tell, not the peak
Two servers can show visually similar-looking daily graphs — a climb during the day, a drop overnight — and still be in completely different states. One returns to exactly the same overnight floor every single night: healthy, bursty, normal. The other's overnight floor creeps very slightly higher each night: a real, sustained problem hiding underneath what looks, at a glance, like the same repeating pattern.

A Genuine Judgment Call: When to Escalate a Trend

Recognizing a real, sustained trend is necessary, but it isn't the same as knowing what to do about it. Whether a confirmed trend needs urgent action, a scheduled fix, or genuine capacity planning depends on context this course can't fully supply on its own — how close the trend is to a hard limit, how fast it's actually climbing, and whether it represents real, wanted growth or a bug that needs fixing. Recognizing the trend is this chapter's job; deciding the response is a separate judgment call, often above what a single support engineer decides alone.

Working Example: The Nightly Restart That Was Never a Fix

A dashboard shows memory climbing to a concerning 85% by the end of every day — but a nightly scheduled restart drops it back to a comfortable-looking 40% each time, and the pattern has repeated, unremarked-on, for weeks. This is worth pausing on directly: Chapter 1 warned that a restart destroys the evidence needed to diagnose a problem. Here's the longer-term version of that same warning — a nightly restart didn't just destroy evidence once, it's been quietly masking a real problem for so long that nobody remembers it's a problem at all.

Applying the floor technique — checking available memory at the exact same point each day, right after the restart, over several weeks — tells a very different story than the daily graph alone:

WeekAvailable memory, right after restart
3 weeks ago9.2 GB
2 weeks ago8.1 GB
Last week6.9 GB
This week5.8 GB

Even the "reset" state is degrading, week over week — a real, slow leak, one that the nightly restart schedule has been tolerating rather than fixing. Left alone, the same trend that's been climbing for weeks eventually reaches the point where a nightly restart is no longer enough to buy back the space, and the actual leak — never diagnosed, only postponed — finally causes a real incident.

Hands-On Exercises

Exercise 1

Explain the difference between a transient spike and a sustained trend, using this chapter's own definition of what distinguishes the two.

📄 View solution
Exercise 2

Explain the "floor" technique this chapter describes, and why comparing quiet-period low points reveals a real leak that looking only at daily peaks might miss.

📄 View solution
Exercise 3

Explain how this chapter's worked example connects back to Chapter 1's own warning about restarts destroying evidence, and why weeks of nightly restarts made the underlying problem harder to notice, not easier.

📄 View solution

Chapter 9 Quick Reference

  • Transient spikes (startup, batch jobs, real traffic bursts, GC pauses) are normal and resolve fully on their own
  • A genuine sustained trend doesn't fully return to where it started — organic growth or a real leak
  • The "floor" technique: compare quiet-period low points over time, not peaks — a healthy system returns to the same floor every time; a real problem leaves a slightly higher floor each cycle
  • Recognizing a trend is this course's job; deciding the response (capacity planning vs. a bug fix) is a separate, context-dependent judgment call
  • A recurring "fix" (like a nightly restart) can mask a real problem for weeks — echoing Chapter 1's own restart-destroys-evidence warning, at a longer timescale
  • Next chapter: Capstone: Triaging Three Real Performance Tickets