API Gateways & Service Discovery

Distributed Systems & Scalability

Chapter 8 · API Gateways & Service Discovery

Software Architecture Fundamentals Chapter 4 measured a real, non-zero cost for every network call. This chapter shows the two standard fixes for a microservices system that's split into enough pieces for that cost to actually hurt: an API Gateway that reduces how many round trips a client pays for, and a service registry that lets services find each other without hardcoding addresses that will eventually go stale.

API Gateway: Aggregating Three Calls Into One Round Trip

class ApiGateway:
    def __init__(self, order_service, inventory_service, review_service):
        self.order_service = order_service
        self.inventory_service = inventory_service
        self.review_service = review_service
    def get_order_page(self, order_id, product_id):
        order = self.order_service.get_order_summary(order_id)
        stock = self.inventory_service.get_stock_info(product_id)
        reviews = self.review_service.get_reviews(product_id)
        return {'order': order, 'stock': stock, 'reviews': reviews}
    

Verified directly — the client is genuinely faster, not just making fewer calls in the abstract

A client on a slow external connection (simulated at 50ms per round trip) calling three services directly — order, inventory, reviews — pays that 50ms cost three separate times: 166.9ms total. The identical client, calling one ApiGateway endpoint instead, pays the 50ms external cost once, with the gateway making the same three calls internally over a fast internal network (5ms each): 66.0ms total — 2.53× faster. Both approaches return identical data.

Why this isn't just "fewer function calls"

The speedup here comes specifically from moving the multi-call fan-out from a slow, external network hop to a fast, internal one — the gateway still makes three calls, it just makes them from somewhere the network cost is small. A gateway that fanned out over the same slow network the client was on wouldn't help at all; this is the same "where does the cost actually live" question Software Architecture Fundamentals Chapter 4 first measured.

Service Discovery: Finding a Service That Doesn't Stay in One Place

class ServiceRegistry:
    def __init__(self): self.services = {}
    def register(self, name, address): self.services[name] = address
    def discover(self, name): return self.services.get(name)
    

Verified directly — a hardcoded address silently breaks after a real redeploy; a discovery-based lookup doesn't

InventoryService registers at 10.0.0.5:8080. A hardcoded client captures that address once, at startup. The service then genuinely redeploys — a real scaling or restart event — and re-registers at a new address, 10.0.0.9:8080; only that new address is actually listening now. The hardcoded client's own call, using its stale captured address, correctly returns False — it would silently fail against a service that no longer exists there. A client using service discovery — looking up the current address fresh, at the moment of the call — correctly finds 10.0.0.9:8080 and returns True.

Why this connects directly to Chapter 1's own horizontal scaling

Chapter 1 verified genuine scaling means adding or replacing running instances — exactly the kind of event that changes a service's own network address. A hardcoded address isn't just fragile in theory; it's specifically incompatible with the scaling behavior this course has spent seven chapters establishing as necessary. Service discovery is what makes horizontal scaling and service discovery compatible with each other at all.

Where This Connects

This chapter's finding	What it connects to
A verified 2.53× client-side speedup from aggregating three calls into one	Software Architecture Fundamentals Chapter 4's own ~11,661× network-call overhead finding — this chapter's gateway is a direct, concrete application of minimizing exactly that cost
A hardcoded address silently breaking after a real redeploy	Chapter 2's own health-checking finding — both are examples of a system correctly routing around a change, versus one that doesn't notice at all
Service discovery as the missing piece behind horizontal scaling	Chapter 1's own scaling findings — this chapter closes the gap between "you can add more instances" and "the rest of the system can actually find them"

Hands-On Exercises

Exercise 1

Add a fourth service, ShippingService, to this chapter's own ApiGateway (with the same simulated 5ms internal latency), and measure the new total time with and without the gateway. Verify the gap between the two approaches widens as more services are aggregated.

📄 View solution

Exercise 2

Using this chapter's own ServiceRegistry, simulate a second redeploy (a third address) happening after the first. Verify a discovery-based client still correctly finds the newest address, and verify the hardcoded client (still holding its original, very first captured address) remains broken exactly as before.

📄 View solution

Exercise 3

Using this chapter's own two verified findings, explain why an API gateway that fans out to three services using hardcoded addresses would eventually break in the exact way this chapter's own hardcoded client did — and why a real gateway needs service discovery internally, not just aggregation.

📄 View solution

Chapter 8 Quick Reference

API Gateway: aggregates multiple backend calls into one client-facing round trip — verified: 2.53× faster from the client's own perspective (166.9ms → 66.0ms), identical results
Service registry: lets services be found by name instead of hardcoded address — verified: a hardcoded client silently broke after a real redeploy; a discovery-based client correctly kept working
The connection to Chapter 1: service discovery is what makes horizontal scaling's own instance churn survivable for the rest of the system
Next chapter: Fault Tolerance & Resilience Patterns — circuit breakers, retries with backoff, and graceful degradation