Week 9: Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors Defined#

Week 2 treated a square matrix $A$ as a linear transformation: every input vector $x$ is sent to a new vector $Ax$. In general $Ax$ points somewhere new — length and direction both change. There is a special class of directions where the transformation is as simple as possible: $A$ only scales the vector, without rotating it off its line.

A nonzero vector $x$ is an eigenvector of $A$ with eigenvalue $\lambda$ when

The left side applies the matrix; the right side multiplies by a scalar. Equality means $Ax$ lies on the same line through the origin as $x$. The number $\lambda$ is the stretch factor along that line: $\lambda > 1$ stretches, $0 < \lambda < 1$ shrinks, $\lambda < 0$ flips the direction and scales, and $\lambda = 0$ sends $x$ to the zero vector (so $x$ is in the null space $N(A)$).

The zero vector is excluded by definition: $A0 = \lambda 0$ holds for every $\lambda$, so it would not pin down a meaningful direction or eigenvalue.

Geometric picture. Think of $A$ as a map of the plane. Most arrows $x$ get sent to arrows $Ax$ that are not parallel to $x$. Eigenvectors are the rare arrows that stay parallel: the map stretches or flips them along their own axis. Those invariant axes are the skeleton of the transformation — later weeks (diagonalization, symmetric matrices, SVD) rebuild $A$ from them.

The eigenvectors of $A=\begin{pmatrix}2&1\\1&2\end{pmatrix}$ define two invariant lines. A generic vector $x$ (green) does not stay on its own line — its image $Ax$ points in a different direction — but vectors along the eigen-lines are only scaled.

Quick check. Take

Then $Ax = (2, 0)^\top = 2x$, so $x$ is an eigenvector with $\lambda = 2$. The vector $(0,1)^\top$ is an eigenvector with $\lambda = 3$. Every axis-aligned stretch has the standard basis as eigenvectors; the interesting case is when $A$ is not diagonal and those special directions are tilted.

The Characteristic Equation#

Rewrite $Ax = \lambda x$ by moving every term to one side:

So $x$ is an eigenvector with eigenvalue $\lambda$ exactly when $x$ is a nonzero vector in the null space of $A - \lambda I$:

Week 4: a square matrix $B$ has a nonzero null-space vector if and only if $\det(B) = 0$ (equivalently, $B$ is singular). Apply that test with $B = A - \lambda I$:

This is the characteristic equation of $A$. Its roots are the eigenvalues. Expanding the determinant produces a polynomial of degree $n$ in $\lambda$ — the characteristic polynomial $p_A(\lambda) = \det(A - \lambda I)$ (equivalently $\det(\lambda I - A)$, which differs only by a sign) — so an $n \times n$ matrix has $n$ eigenvalues counted with algebraic multiplicity (some may be complex even when $A$ is real).

Why determinants reappear. The same singularity test that told you whether $Ax = b$ had a unique solution now tells you which stretch factors $\lambda$ are compatible with $A$. No new machinery: only “when is $A - \lambda I$ singular?”

Trace and determinant shortcuts ($2 \times 2$). For a $2 \times 2$ matrix the characteristic polynomial is always

You can verify this by expanding $\det\begin{pmatrix} a-\lambda & b \\ c & d-\lambda \end{pmatrix} = (a-\lambda)(d-\lambda) - bc$. Vieta's formulas then read: the two eigenvalues sum to the trace and multiply to $\det(A)$. These identities are not accidents of the $2 \times 2$ case — they hold for every $n \times n$ matrix, counted with algebraic multiplicity — but the explicit quadratic is special to $2 \times 2$.

Finding Eigenvectors#

Once you have a candidate $\lambda$, finding eigenvectors is a null-space computation from Week 6: row-reduce $A - \lambda I$ and solve $(A - \lambda I)x = 0$. Every nonzero solution is an eigenvector for that $\lambda$. Scaling an eigenvector still yields an eigenvector for the same $\lambda$ — you usually report a convenient representative (integer components, or unit length).

Worked example end to end. Let

Step 1 — eigenvalues. Trace $\operatorname{tr}(A) = 4 + 3 = 7$ and $\det(A) = 12 - 2 = 10$, so

Factor: $(\lambda - 5)(\lambda - 2) = 0$. The eigenvalues are $\lambda_1 = 5$ and $\lambda_2 = 2$.

Step 2 — eigenvector for $\lambda = 5$. Form

Row 2 is $-2$ times row 1, so the rank is $1$. The equation $(-1)x_1 + x_2 = 0$ gives $x_2 = x_1$. Free variable $x_1 = t$ yields

Take $v_1 = (1, 1)^\top$. Check: $A v_1 = (5, 5)^\top = 5 v_1$.

Step 3 — eigenvector for $\lambda = 2$. Form

The single independent equation is $2x_1 + x_2 = 0$, so $x_2 = -2x_1$. Thus

is an eigenvector. Check: $A v_2 = (4 - 2,\ 2 - 6)^\top = (2, -4)^\top = 2 v_2$.

Summary. $A$ stretches the line spanned by $(1,1)^\top$ by $5$ and the line spanned by $(1,-2)^\top$ by $2$. Those two directions are the eigenbasis of this plane map.

Recipe for $n \times n$.

1. Form $p_A(\lambda) = \det(A - \lambda I)$ and find the roots $\lambda$.
2. For each root, solve $(A - \lambda I)x = 0$ by elimination; report a basis of the solution space (nonzero vectors only).
3. Optionally normalize eigenvectors or choose integer multiples for readability.

For larger $n$, hand expansion of $\det(A - \lambda I)$ is painful; production code uses iterative algorithms (QR iteration and variants). The conceptual pipeline — “characteristic roots, then null spaces” — stays the same. The browser lab below uses numpy.linalg.eig for the same $2 \times 2$ matrix and verifies $Av = \lambda v$.

Eigenspaces and Multiplicity#

For a fixed eigenvalue $\lambda$, the set of all solutions of $(A - \lambda I)x = 0$ is the null space $N(A - \lambda I)$. That set always includes the zero vector, and it is a subspace of $\mathbb{R}^n$. It is called the eigenspace of $A$ for $\lambda$:

(The zero vector is allowed here because subspaces must contain zero; “eigenvectors” still means the nonzero members of $E_\lambda$.)

The geometric multiplicity of $\lambda$ is $\dim E_\lambda$ — how many independent eigenvectors you can find for that eigenvalue. The algebraic multiplicity is the multiplicity of $\lambda$ as a root of the characteristic polynomial. These always sit in a fixed order (a standard result — see Strang, §6.1):

When the two multiplicities match for every eigenvalue and you can assemble a full basis of eigenvectors, $A$ is diagonalizable — Week 10's main topic. They need not match.

Repeated root without enough eigenvectors. Consider

Then $\det(J - \lambda I) = (1 - \lambda)^2$, so $\lambda = 1$ has algebraic multiplicity $2$. But

has rank $1$, so $\dim N(J - I) = 1$. Every eigenvector is a multiple of $(1, 0)^\top$; there is no second independent eigenvector. Geometric multiplicity is $1$, less than algebraic multiplicity $2$. Matrices like $J$ are defective: they cannot be diagonalized. Week 10 returns to the exact criterion (a full set of independent eigenvectors) and to the Jordan form that replaces diagonalization when that criterion fails.

Knowledge check#

Browser lab: eigenvalues and eigenvectors#

Compute eigenvalues/eigenvectors with numpy.linalg.eig and verify $Av = \lambda v$.

import numpy as np

A = np.array([[4., 1.],
              [2., 3.]])

eigvals, eigvecs = np.linalg.eig(A)
print("eigenvalues =", eigvals)
print("eigenvectors (columns) =\n", eigvecs)

for i in range(2):
    lhs = A @ eigvecs[:, i]
    rhs = eigvals[i] * eigvecs[:, i]
    print(f"A v{i+1} ≈ λ{i+1} v{i+1}:", np.allclose(lhs, rhs))

print("Notice: eigenvalues are 5 and 2, and Av ≈ λv prints True for each eigenpair.")

Try it: Change A to [[2, 1], [1, 2]] and verify the eigenvectors are orthogonal (their dot product is zero). Swap in [[0, 1], [-1, 0]] — what are the eigenvalues now and why are they complex?

Done when#

You can find the eigenvalues of a 2×2 matrix from $\det(A-\lambda I)=0$, produce an eigenvector for each, and verify $Av = \lambda v$. Check it against the browser lab: for $A=\begin{pmatrix}4&1\\2&3\end{pmatrix}$ the printed eigenvalues are 5. and 2., and both A v ≈ λ v lines print True; then confirm the Try-it matrix gives orthogonal eigenvectors and the rotation has purely imaginary eigenvalues.

Further Reading#

• Strang, G. Introduction to Linear Algebra, 6th ed. (2023). Chapters 6.1–6.2. The classic eigenvalue exposition — clear worked examples and the trace/determinant shortcuts.
• 3Blue1Brown Essence of Linear Algebra, Chapters 13–14. Visual animations of eigenvectors as axes that stay on their span under a transformation.
• Axler, S. Linear Algebra Done Right, 4th ed. (2024). Chapter 5. Develops eigenvalues without determinants first (via invariant subspaces), then ties to the characteristic polynomial.
• Trefethen, L.N. and Bau, D. Numerical Linear Algebra (1997). Lectures 24–28. How production eigenvalue solvers (QR iteration, Arnoldi) work — the bridge from pen-and-paper to numpy.linalg.eig.