Exercise 1: The Parallel Between RNN Parameter Sharing (Time) and CNN Parameter Sharing (Space) — Possible Solution ==================================================================== WHAT nn1-7's OWN CNN PARAMETER SHARING DOES ------------------------------ Per nn1-7, a convolutional kernel "slides across the image, computing a small, local weighted sum at each position. Crucially, the same set of weights is reused at every position." The identical small kernel is applied over and over, once at each spatial location in the image, rather than each position having its own independent, separately- learned set of weights. WHAT THIS CHAPTER'S OWN RNN PARAMETER SHARING DOES ------------------------------ Per this chapter, "the exact same weights are reused at every time step" — the identical RNN cell (with its own single, fixed set of weights) is applied over and over, once at each step of the sequence, rather than each time step having its own independent, separately- learned set of weights. WHY THIS IS THE SAME UNDERLYING IDEA, JUST A DIFFERENT AXIS ------------------------------ Per this chapter's own tip-box, "the exact same weights are reused at every time step — a direct structural cousin of nn1-7's own convolutional kernel, which reused the same weights at every spatial position... the sharing happens across time instead of across space, but it's the identical underlying idea: one small, learnable transformation, applied repeatedly." In both cases, the core design choice is identical: rather than learning a separate transformation for every individual "slot" (a spatial position for a CNN, a time step for an RNN), one single, small transformation is learned once and then applied repeatedly across every slot. The only thing that differs is what kind of "slot" the sharing happens across — a 2D spatial position for the CNN, a point in a temporal sequence for the RNN. WHY THIS SHARED PRINCIPLE PRODUCES ANALOGOUS BENEFITS IN BOTH CASES ------------------------------ Per nn1-7's own reasoning, the CNN's own spatial parameter sharing reduces the total parameter count and provides translation invariance (a pattern is detected the same way regardless of where it appears). The RNN's own temporal parameter sharing produces an analogous pair of benefits: a much smaller parameter count than would be needed if every time step required its own independent weights, and a form of "temporal invariance" — the same kind of pattern (say, "this word tends to follow that one") is recognized consistently no matter which point in the sequence it happens to occur at, rather than needing to be relearned separately for each possible position in the sequence. WHY THIS PARALLEL IS PRECISE, NOT JUST A LOOSE ANALOGY ------------------------------ Both mechanisms follow the exact same mathematical pattern: one fixed set of weights, applied identically and repeatedly across a dimension the data varies along — space for images, time for sequences. This precise structural correspondence, not merely a superficial similarity in wording, is why the chapter calls the RNN a "direct structural cousin" of the CNN's own kernel, rather than a vaguely related idea. WHY THIS WORKS AS AN ANSWER ------------------------------ It restates both mechanisms precisely using each chapter's own description, identifies the shared underlying principle (one small transformation, reused across a repeating structural dimension), and explains why the only genuine difference between the two is which dimension (space vs. time) that sharing happens across.