Week 9: Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors Defined#

Week 2 treated a square matrix $A$ as a linear transformation: every input vector $x$ is sent to a new vector $Ax$. In general $Ax$ points somewhere new — length and direction both change. There is a special class of directions where the transformation is as simple as possible: $A$ only scales the vector, without rotating it off its line.

A nonzero vector $x$ is an eigenvector of $A$ with eigenvalue $\lambda$ when

The left side applies the matrix; the right side multiplies by a scalar. Equality means $Ax$ lies on the same line through the origin as $x$. The number $\lambda$ is the stretch factor along that line: $\lambda > 1$ stretches, $0 < \lambda < 1$ shrinks, $\lambda < 0$ flips the direction and scales, and $\lambda = 0$ sends $x$ to the zero vector (so $x$ is in the null space $N(A)$).

The zero vector is excluded by definition: $A0 = \lambda 0$ holds for every $\lambda$, so it would not pin down a meaningful direction or eigenvalue.

Geometric picture. Think of $A$ as a map of the plane. Most arrows $x$ get sent to arrows $Ax$ that are not parallel to $x$. Eigenvectors are the rare arrows that stay parallel: the map stretches or flips them along their own axis. Those invariant axes are the skeleton of the transformation — later weeks (diagonalization, symmetric matrices, SVD) rebuild $A$ from them.

Quick check. Take

Then $Ax = (2, 0)^\top = 2x$, so $x$ is an eigenvector with $\lambda = 2$. The vector $(0,1)^\top$ is an eigenvector with $\lambda = 3$. Every axis-aligned stretch has the standard basis as eigenvectors; the interesting case is when $A$ is not diagonal and those special directions are tilted.

The Characteristic Equation#

Rewrite $Ax = \lambda x$ by moving every term to one side:

So $x$ is an eigenvector with eigenvalue $\lambda$ exactly when $x$ is a nonzero vector in the null space of $A - \lambda I$:

Week 4: a square matrix $B$ has a nonzero null-space vector if and only if $\det(B) = 0$ (equivalently, $B$ is singular). Apply that test with $B = A - \lambda I$:

This is the characteristic equation of $A$. Its roots are the eigenvalues. Expanding the determinant produces a monic polynomial of degree $n$ in $\lambda$ — the characteristic polynomial $p_A(\lambda) = \det(A - \lambda I)$ — so an $n \times n$ matrix has $n$ eigenvalues counting algebraic multiplicity (some may be complex even when $A$ is real).

Why determinants reappear. The same singularity test that told you whether $Ax = b$ had a unique solution now tells you which stretch factors $\lambda$ are compatible with $A$. No new machinery: only “when is $A - \lambda I$ singular?”

Trace and determinant shortcuts ($2 \times 2$). For a $2 \times 2$ matrix the characteristic polynomial is always

You can verify this by expanding $\det\begin{pmatrix} a-\lambda & b \\ c & d-\lambda \end{pmatrix} = (a-\lambda)(d-\lambda) - bc$. The Vieta relations follow immediately: the sum of the two eigenvalues equals the trace, and their product equals $\det(A)$. Those identities hold for every $n \times n$ matrix as well (sum of eigenvalues = trace, product = determinant), but the quadratic formula is special to $2 \times 2$.

Finding Eigenvectors#

Once you have a candidate $\lambda$, finding eigenvectors is a null-space computation from Week 6: row-reduce $A - \lambda I$ and solve $(A - \lambda I)x = 0$. Every nonzero solution is an eigenvector for that $\lambda$. Scaling an eigenvector still yields an eigenvector for the same $\lambda$ — you usually report a convenient representative (integer components, or unit length).

Worked example end to end. Let

Step 1 — eigenvalues. Trace $\operatorname{tr}(A) = 4 + 3 = 7$ and $\det(A) = 12 - 2 = 10$, so

Factor: $(\lambda - 5)(\lambda - 2) = 0$. The eigenvalues are $\lambda_1 = 5$ and $\lambda_2 = 2$.

Step 2 — eigenvector for $\lambda = 5$. Form

Row 2 is $-2$ times row 1, so the rank is $1$. The equation $(-1)x_1 + x_2 = 0$ gives $x_2 = x_1$. Free variable $x_1 = t$ yields

Take $v_1 = (1, 1)^\top$. Check: $A v_1 = (5, 5)^\top = 5 v_1$.

Step 3 — eigenvector for $\lambda = 2$. Form

The single independent equation is $2x_1 + x_2 = 0$, so $x_2 = -2x_1$. Thus

is an eigenvector. Check: $A v_2 = (4 - 2,\ 2 - 6)^\top = (2, -4)^\top = 2 v_2$.

Summary. $A$ stretches the line spanned by $(1,1)^\top$ by $5$ and the line spanned by $(1,-2)^\top$ by $2$. Those two directions are the eigenbasis of this plane map.

Recipe for $n \times n$.

1. Form $p_A(\lambda) = \det(A - \lambda I)$ and find the roots $\lambda$.
2. For each root, solve $(A - \lambda I)x = 0$ by elimination; report a basis of the solution space (nonzero vectors only).
3. Optionally normalize eigenvectors or choose integer multiples for readability.

For larger $n$, hand expansion of $\det(A - \lambda I)$ is painful; production code uses iterative algorithms (QR iteration and variants). The conceptual pipeline — “characteristic roots, then null spaces” — stays the same. The browser lab below uses numpy.linalg.eig for the same $2 \times 2$ matrix and verifies $Av = \lambda v$.

Eigenspaces and Multiplicity#

For a fixed eigenvalue $\lambda$, the set of all solutions of $(A - \lambda I)x = 0$ is the null space $N(A - \lambda I)$. That set always includes the zero vector, and it is a subspace of $\mathbb{R}^n$. It is called the eigenspace of $A$ for $\lambda$:

(The zero vector is allowed here because subspaces must contain zero; “eigenvectors” still means the nonzero members of $E_\lambda$.)

The geometric multiplicity of $\lambda$ is $\dim E_\lambda$ — how many independent eigenvectors you can find for that eigenvalue. The algebraic multiplicity is the multiplicity of $\lambda$ as a root of the characteristic polynomial. Always

When the two multiplicities match for every eigenvalue and you can assemble a full basis of eigenvectors, $A$ is diagonalizable — Week 10's main topic. They need not match.

Repeated root without enough eigenvectors. Consider

Then $\det(J - \lambda I) = (1 - \lambda)^2$, so $\lambda = 1$ has algebraic multiplicity $2$. But

has rank $1$, so $\dim N(J - I) = 1$. Every eigenvector is a multiple of $(1, 0)^\top$; there is no second independent eigenvector. Geometric multiplicity is $1$, less than algebraic multiplicity $2$. Matrices like $J$ are defective: they cannot be diagonalized. Week 10 returns to the exact criterion (a full set of independent eigenvectors) and to the Jordan form that replaces diagonalization when that criterion fails.

What you need this week. Compute eigenvalues from $\det(A - \lambda I) = 0$, compute eigenspaces as null spaces, and read geometric multiplicity as the dimension of that null space. The diagonalizability question is reserved for next week; the example above only flags that a repeated root does not automatically give you two independent directions.

Knowledge check#

Browser lab#

Compute eigenvalues/eigenvectors with numpy.linalg.eig and verify $Av = \lambda v$.

import numpy as np

A = np.array([[4., 1.],
              [2., 3.]])

eigvals, eigvecs = np.linalg.eig(A)
print("eigenvalues =", eigvals)
print("eigenvectors (columns) =\n", eigvecs)

for i in range(2):
    lhs = A @ eigvecs[:, i]
    rhs = eigvals[i] * eigvecs[:, i]
    print(f"A v{i+1} ≈ λ{i+1} v{i+1}:", np.allclose(lhs, rhs))