Exercise 2: Why Purely Linear Layers Collapse, and Why That Matters Here — Possible Solution
====================================================================

WHAT "LINEAR" MEANS FOR A LAYER'S OWN COMPUTATION
------------------------------
Per nn1-1, a single neuron computes a weighted sum plus a bias, then
applies an activation function. If that activation function is itself
purely linear (for instance, simply passing the weighted sum through
unchanged, or multiplying it by a fixed constant), the entire neuron's
own output is just a linear combination of its inputs — no bends,
curves, or thresholds in its own response, just straight-line scaling
and shifting.

WHY STACKING TWO LINEAR LAYERS PRODUCES ONLY ONE EFFECTIVE LAYER
------------------------------
Per this chapter, "stack two purely linear layers with no nonlinearity
between them, and the math collapses: a linear function of a linear
function is still just one linear function." Algebraically, if the
first layer computes some linear combination of the inputs, and the
second layer computes a linear combination of THAT result, the overall
composed computation is still expressible as a single linear
combination of the ORIGINAL inputs — multiplying and adding linear
things together never produces anything but another linear thing. No
matter how many purely linear layers are stacked, the entire network
is mathematically equivalent to some single linear layer with
different, combined weights.

WHY THIS DIRECTLY THREATENS THIS CHAPTER'S OWN XOR SOLUTION
------------------------------
Per this chapter, "this trick only works because the hidden layer's own
activation function... is nonlinear." This chapter's own worked
solution relied specifically on the hidden layer computing something
(the step function, approximating OR and NAND) that a single linear
layer could never compute — a step function has a genuine threshold, a
sharp bend in its response, which is precisely the kind of nonlinear
behavior that lets the hidden layer create a NEW geometric arrangement
of the four points (Exercise 1's own worked transformation) rather than
merely re-scaling or shifting the original, unsolvable diagonal
arrangement.

WHY A LINEAR VERSION OF THIS SOLUTION WOULD NOT WORK AT ALL
------------------------------
If the hidden layer's own activation were purely linear instead of the
step function used here, per the collapse argument above, the entire
two-layer network (hidden layer plus output layer) would be
mathematically equivalent to a single linear layer — precisely
nn1-1's own single neuron, precisely the model class nn1-2 already
proved cannot solve XOR. The hidden layer's own nonlinearity isn't an
incidental detail of this chapter's specific worked example — it's the
one property making the whole transformation trick possible at all.
Without it, this chapter's own two-layer network would be no more
capable than the single perceptron nn1-2 already showed fails on XOR.

WHY THIS WORKS AS AN ANSWER
------------------------------
It explains algebraically why composing purely linear layers always
collapses into an equivalent single linear layer, and connects this
directly to why this chapter's own XOR solution specifically depends on
its hidden layer's nonlinear step function — without which the entire
two-layer network would reduce back to exactly the XOR-incapable single
neuron nn1-2 already ruled out.