Exercise 1: Why Define-by-Run Mattered Specifically for Research — Possible Solution ==================================================================== WHAT DEFINE-THEN-RUN MADE DIFFICULT, PER THIS CHAPTER ------------------------------ Per this chapter, TensorFlow's original define-then-run approach was "efficient for deployment and optimization, but genuinely awkward for debugging — you couldn't simply inspect an intermediate value mid-computation the way you'd step through ordinary Python." Because the entire computation graph had to be fully built and fixed BEFORE any data was run through it, a researcher couldn't pause partway through and directly examine what a specific layer's output actually looked like for a specific input — the graph existed as one complete, static structure, separate from the step-by-step Python execution a developer would normally use to inspect values. WHAT DEFINE-BY-RUN CHANGED, PER THIS CHAPTER ------------------------------ Per this chapter, PyTorch's own eager execution meant "the computation graph is built dynamically as code actually executes, line by line, exactly like ordinary Python." Because the graph is constructed as a direct, real-time consequence of running actual Python statements one at a time, a researcher can use completely ordinary Python debugging techniques — printing an intermediate value, setting a breakpoint, inspecting a variable — at any point in the computation, exactly as they would with any other Python program. WHY THIS SPECIFICALLY MATTERS FOR RESEARCH AND EXPERIMENTATION ------------------------------ Research work is defined by frequent, exploratory changes — trying a new layer configuration, checking whether an intermediate activation looks reasonable, quickly testing a hypothesis about why a model is behaving unexpectedly. Each of these tasks benefits enormously from being able to directly inspect what's actually happening at any given point in the computation, in real time, using familiar tools. Define- then-run's own separation between "build the graph" and "run the graph" adds real friction to exactly this kind of rapid, iterative investigation — a researcher would need to work through the graph's own separate execution model rather than simply examining values the way they naturally would while writing and testing any other piece of Python code. WHY THIS DIFFERENCE WAS LESS COSTLY FOR PRODUCTION USE ------------------------------ Per this chapter, the same define-then-run approach that hindered research was "efficient for deployment and optimization" — a production system typically runs a model with a fixed, already- finalized architecture repeatedly at scale, where the ability to inspect intermediate values interactively matters far less than raw execution efficiency. This is exactly why the trade-off cut in opposite directions for the two different use cases, and why PyTorch's own advantage specifically showed up in research settings rather than uniformly everywhere. WHY THIS WORKS AS AN ANSWER ------------------------------ It explains precisely what capability define-then-run's static graph lacked (interactive, mid-computation inspection using ordinary Python tools), explains why define-by-run restores that capability by building the graph through real Python execution itself, and connects this specifically to why rapid, exploratory research work benefits from it far more than fixed, already-finalized production deployment does.