Orchestration Beyond Compose
🧩 Orchestration Beyond Compose
What Compose Is Actually Good At (Recap)
Everything Chapters 3–7 covered — override files, healthchecks, custom networks, named volumes, security hardening, CI integration — makes Compose a genuinely excellent tool for its actual scope: starting, networking, and managing a defined set of containers on one machine. That scope covers a huge share of real deployments.
Where Compose Stops Scaling
A docker-compose.yml describes services that run on one host — whichever machine docker compose up is run on. Compose has no built-in concept of "multiple hosts" at all:
- No cross-machine scheduling — Compose can't decide to run a service on a different machine than the one it's invoked on.
- No real self-healing — Compose's restart policies can restart a crashed container on the same host, but if the entire host goes down, nothing in Compose itself notices, let alone reschedules that workload elsewhere.
- No fleet-wide rolling deployment — deploying a new version across many machines isn't something Compose was built to coordinate.
What Orchestration Actually Adds
Orchestration treats a cluster of many machines as one pool of resources, with a scheduler deciding which machine runs which container — and automatically rescheduling workloads if a machine dies. This is self-healing at the cluster level, not just "restart this one container": an entire machine disappearing gets its workload redistributed elsewhere, automatically. This is a structurally different capability Compose was never designed to have — not a missing feature waiting to be added.
Docker Swarm
Swarm is Docker's own native orchestrator, with a genuinely similar mental model to Compose — a docker-compose.yml can often be deployed nearly as-is via docker stack deploy. Lower learning curve coming from Compose, though it's seen less industry-wide adoption than Kubernetes in recent years.
Kubernetes (Conceptual Orientation Only)
Kubernetes is the dominant orchestrator industry-wide, with a genuinely different mental model and a much steeper learning curve than Swarm or Compose — Pods (not quite the same thing as a single container), Deployments, Services, and a full declarative API rather than Compose's simpler file format. This chapter deliberately does not go deep on Kubernetes — just naming these core concepts so they're recognizable as the natural next step once orchestration needs genuinely outgrow this course's scope.
Compose vs. Swarm/Kubernetes
Compose
Single host, a defined set of containers, no cluster-level scheduling or self-healing.
Swarm / Kubernetes
A cluster of many machines treated as one pool, with automatic scheduling and self-healing across the whole fleet.
| Compose | Swarm | Kubernetes | |
|---|---|---|---|
| Scope | Single host | Multi-host cluster | Multi-host cluster |
| Learning curve from Compose | — | Low | Steep |
| Industry adoption | Extremely common (dev, small deployments) | Declining | Dominant at scale |
Compose Is Still Enough When
The deployment genuinely fits on one well-resourced machine — most side projects, single-server production deployments, and everything Chapters 1–7 of this course already cover.
Real Orchestration Is Needed When
True high availability across multiple machines is a genuine requirement, or the workload has outgrown what a single host can realistically serve.
💻 Coding Challenges
Challenge 1: Explain the Structural Limit
Explain, in your own words, why Compose's inability to reschedule work after a host fails is a structural limitation, not a missing feature that a future Compose update could simply add.
Goal: Practice articulating the actual scope Compose was designed for, versus what orchestration adds.
Challenge 2: Compose, Swarm, or Kubernetes?
A solo developer runs a small side project on one VPS. A company runs a large production system across dozens of machines with strict uptime requirements. Recommend an approach for each, using this chapter's comparison.
Goal: Practice matching orchestration complexity to actual, not assumed, scaling needs.
Challenge 3: Connect to a Prior Site Gotcha
This chapter's closing gotcha warns against adopting Kubernetes prematurely. Name a similar "don't do this before you need it" gotcha from another course on this site, and explain the shared underlying principle.
Goal: Practice recognizing a recurring judgment-call pattern across genuinely different technologies.
"Kubernetes for a single side project" is a well-known industry joke for a reason — adopting cluster orchestration before there's a genuine multi-host scaling or availability problem to solve adds real, substantial operational complexity (a cluster to run, a much steeper learning curve, an entirely different mental model) for a problem that doesn't yet exist. This is the exact same judgment call mongodb2-6's closing exercise made about sharding a MongoDB collection that "comfortably fits on a single 3-node replica set with room to spare" — don't adopt distributed-systems complexity ahead of a real, demonstrated need. The underlying principle recurs across genuinely different technologies for the same reason every time: match infrastructure complexity to an actual, current requirement — not to where the project might theoretically be someday.
🎯 What's Next
The final chapter is the Capstone: Productionizing a Multi-Container App — combining multi-stage builds, Compose, security hardening, and CI integration into one small real application.