Exercise 1: Why Generation Can Start From Pure Noise and Still Produce a Coherent Image — Possible Solution ==================================================================== WHAT TRAINING ACTUALLY TEACHES THE NETWORK ------------------------------ Per this chapter, training works by "taking a real image, adding noise in increasing amounts across many steps, [and training] a network to predict and remove the noise added at each step." Crucially, this training happens across the FULL RANGE of noise levels — from images that are only slightly noised all the way up to images that are essentially pure noise. The network isn't just learning "how to clean up a mostly-clear photo" — it's learning, for every possible noise level from mild to total, what a slightly-cleaner version of that noise level would look like, conditioned on the text embedding. WHY THIS MEANS "ALMOST PURE NOISE" IS JUST ANOTHER TRAINING CASE ------------------------------ Because the forward process trains the network across that entire range, "pure random noise" isn't actually an unfamiliar starting point for the network — it's functionally identical to the final, most- noised step the network already saw thousands of times during training. The network doesn't need to recognize "is this a real photo that's been noised, or synthetic random noise" — per this chapter, it was never taught to make that distinction in the first place; it was only ever taught "given noise at this level (and this text embedding), predict what a slightly less noisy version looks like." WHY REPEATED SMALL STEPS ADD UP TO A COHERENT IMAGE ------------------------------ Per this chapter's own generation description: "run the trained network's denoising step repeatedly, watch a coherent image emerge from static." Each individual step only needs to make one small, locally plausible improvement — nudge the noise slightly toward something more image-like, steered by the text embedding (covered in the chapter's next section). Because that same small improvement is applied hundreds of times in sequence, and each step's output becomes the next step's input, small local improvements compound into a fully formed, globally coherent image by the final step — without the network ever needing to "know," at any single step, what the final image will look like. WHY THIS WORKS AS AN ANSWER ------------------------------ It explains, using the chapter's own description of the forward training process across the full noise range, why starting from pure noise isn't actually a special or unfamiliar case for the network, and explains how repeated small denoising steps compound into a coherent result without requiring any single step to "understand" the whole image.