Eigenvalues & Eigenvectors
Linear Algebra Fundamentals
Chapter 9 · Eigenvalues & Eigenvectors
Most vectors, transformed by a matrix, get both moved to a new direction and scaled. This chapter is about the special exceptions: directions a given transformation only scales, never rotates off their own line. Those special directions — and how much they get scaled by — turn out to be some of the most useful numbers in all of linear algebra.
The Definition
A v = λ v — for a nonzero vector v (the eigenvector) and a scalar λ (the eigenvalue), applying A to v produces exactly v scaled by λ — nothing else.
The simplest possible example reuses Chapter 5's own reflection matrix, Rx = [[1, 0], [0, -1]] (reflection across the x-axis) — since it's diagonal, its eigenvalues are just the diagonal entries themselves:
| Eigenvector | Eigenvalue | Geometric meaning |
|---|---|---|
| [1, 0] | 1 | Points on the x-axis are completely unchanged by this reflection |
| [0, 1] | -1 | Points on the y-axis get flipped to the opposite side — same line, reversed direction |
Both stay on their own line — one unmoved, one flipped — which is exactly what makes them eigenvectors of this particular transformation.
Finding Eigenvalues — The Characteristic Equation
For a matrix without such an obviously convenient diagonal shape, eigenvalues need to be solved for directly. Starting from A v = λ v, rewrite as A v − λ v = 0, then (A − λI) v = 0. For a nonzero v to satisfy this, (A − λI) can't have an inverse — per Chapter 7, that means its determinant must be exactly zero:
det(A − λI) = 0
For a 2×2 matrix A = [[a, b], [c, d]], this expands to a quadratic in λ:
Worked Example
Let A = [[2, 1], [1, 2]]. Trace = 4, determinant = 2(2) − 1(1) = 3. The characteristic equation is:
| Step | Working |
|---|---|
| Characteristic equation | λ² − 4λ + 3 = 0 |
| Factor | (λ − 1)(λ − 3) = 0 |
| Eigenvalues | λ = 1, λ = 3 — sum = 4 ✓ trace, product = 3 ✓ det |
To find each eigenvector, substitute the eigenvalue back into (A − λI) v = 0 and solve — this is exactly a homogeneous version of Chapter 6's system-solving, and it's guaranteed to have a whole line of solutions (not just v = 0), since (A − λI) is singular by construction.
| λ | A − λI | Equation from row 1 | Eigenvector direction |
|---|---|---|---|
| 1 | [[1, 1], [1, 1]] | v₁ + v₂ = 0 → v₂ = −v₁ | [1, −1] |
| 3 | [[-1, 1], [1, -1]] | −v₁ + v₂ = 0 → v₂ = v₁ | [1, 1] |
Checking both: A[1,-1] = [2−1, 1−2] = [1,-1] = 1×[1,-1] ✓, and A[1,1] = [2+1, 1+2] = [3,3] = 3×[1,1] ✓.
A to v = [1, 0] (not one of the eigenvectors above) gives [2, 1] — pointing in a genuinely different direction (rotated about 26.6° away from the x-axis), not just a scaled copy of [1, 0]. Most vectors behave this way; only the two special directions found above don't.
Repeated Application & the Dominant Eigenvalue
Applying A over and over to almost any starting vector reveals something striking: the result increasingly lines up with the eigenvector belonging to the largest-magnitude eigenvalue. Starting from v₀ = [1, 0] (not an eigenvector) and repeatedly applying A = [[2,1],[1,2]]:
| n | Aⁿv₀ | Normalized direction |
|---|---|---|
| 1 | [2, 1] | [0.894, 0.447] |
| 2 | [5, 4] | [0.781, 0.625] |
| 3 | [14, 13] | [0.733, 0.680] |
| 5 | [122, 121] | [0.710, 0.704] |
The direction is visibly converging toward [0.707, 0.707] — the normalized form of the λ = 3 eigenvector, [1, 1], which is the larger of the two eigenvalues. And the magnitude roughly triples at each step for large n, matching that same dominant eigenvalue.
Real Relevance
| Application | How eigenvalues/eigenvectors are used |
|---|---|
| Principal Component Analysis (dimensionality reduction) | The eigenvectors of a dataset's covariance matrix are the directions of maximum spread ("variance"); the eigenvalues rank how much variance each direction captures. Keeping only the top few eigenvectors is exactly how a 50-column dataset gets reduced to its 2 or 3 most informative directions — Chapter 1's own forward reference, finally paid off. |
| Google's original PageRank algorithm | Models the web as a giant transition matrix (probability of clicking from one page to another). The steady-state importance ranking is the eigenvector belonging to eigenvalue 1 — precisely the "repeated application converges to the dominant eigenvector" behavior demonstrated above. |
| Stability analysis | For a system that gets repeatedly transformed (a simulation step, a feedback loop), whether it settles down or blows up depends entirely on the largest eigenvalue's magnitude: < 1 shrinks toward zero over time (stable), > 1 grows without bound (unstable), exactly as this chapter's own repeated-application table demonstrated with λ = 3. |
Eigenvalues & Eigenvectors in Code
Hands-On Exercises
Find both eigenvalues of C = [[4, 2], [1, 3]] using the characteristic equation, then find the corresponding eigenvector for each. Verify both eigenpairs by computing C v and confirming it equals λ v.
A colleague claims λ = 4, v = [2, 1] is a genuine eigenpair of D = [[5, -2], [1, 2]]. Check this directly by computing D v and comparing it to 4v. Then independently verify using the characteristic equation (trace and determinant) that 4 really is one of D's eigenvalues.
A repeated transformation has two eigenvalues: λ₁ = 0.5 (eigenvector direction u₁) and λ₂ = 1.5 (eigenvector direction u₂). If this transformation is applied many times in a row to a generic starting vector that isn't aligned with either eigenvector, explain — using this chapter's own repeated-application finding — which eigenvector's direction the result will end up dominated by, and whether the overall magnitude will grow, shrink, or stay bounded.
📄 View solutionChapter 9 Quick Reference
- Eigenvector/eigenvalue:
A v = λ v— a direction the transformation only scales, never rotates off its own line - Characteristic equation:
det(A − λI) = 0, derived from Chapter 7's singularity requirement for a nonzero solution to exist - For 2×2 matrices:
λ² − trace(A)λ + det(A) = 0— sum of eigenvalues = trace, product of eigenvalues = determinant - Once λ is known, solve
(A − λI)v = 0(Chapter 6-style) to find the matching eigenvector direction - Repeated application of a matrix drives almost any vector toward the eigenvector of the largest-magnitude eigenvalue — the "dominant" eigenvector
- Real applications: PCA (dimensionality reduction), PageRank (steady-state ranking via the eigenvalue-1 eigenvector), stability analysis (dominant eigenvalue magnitude above/below 1)
- Next chapter: Capstone — applying every chapter's material to a single worked access-control system