Exercise 2: Gradients and One Gradient Descent Step at (2,1) — Possible Solution ==================================================================== GIVEN ------------------------------ Data: (1,3), (2,5), (3,7), (4,8), (5,11) m=2, b=1 (per Exercise 1, only (4,8) contributes any error) STEP 1: COMPUTE dL/dm ------------------------------ dL/dm = (2/n) * sum((prediction-y)*x) over all points For each point, (prediction-y)*x: (1): (3-3)*1 = 0 (2): (5-5)*2 = 0 (3): (7-7)*3 = 0 (4): (9-8)*4 = 1*4 = 4 (5): (11-11)*5 = 0 Sum = 0+0+0+4+0 = 4 dL/dm = (2/5)*4 = 8/5 = 1.6 STEP 2: COMPUTE dL/db ------------------------------ dL/db = (2/n) * sum(prediction-y) over all points For each point, (prediction-y): (1): 0, (2): 0, (3): 0, (4): 1, (5): 0 Sum = 1 dL/db = (2/5)*1 = 0.4 STEP 3: APPLY ONE GRADIENT DESCENT STEP, alpha=0.01 ------------------------------ m_new = m - alpha*dL/dm = 2 - 0.01*1.6 = 2 - 0.016 = 1.984 b_new = b - alpha*dL/db = 1 - 0.01*0.4 = 1 - 0.004 = 0.996 RESULT ------------------------------ dL/dm = 1.6, dL/db = 0.4 After one step: m=1.984, b=0.996 Both gradients point in the direction that would REDUCE m and b slightly - which makes sense, since the one error the model has (overshooting the point (4,8) by predicting 9 instead of 8) would be reduced by a slightly smaller line, pulling both m and b down a little. WHY THIS WORKS AS AN ANSWER ------------------------------ Both gradient formulas are applied term by term across every data point (reusing the same per-point error values already established in Exercise 1, rather than recomputing them from scratch), and the resulting direction of the update is checked for sensibility against which specific data point was actually contributing error.