Week 6: The Four Fundamental Subspaces

Column Space and Null Space#

Every $m \times n$ matrix $A$ defines two subspaces of special importance: one in the output space $\mathbb{R}^m$, and one in the input space $\mathbb{R}^n$.

Column space. Write the columns of $A$ as $a_1, a_2, \ldots, a_n \in \mathbb{R}^m$. The column space $C(A)$ is their span:

So $C(A)$ is exactly the set of all possible outputs of the map $x \mapsto Ax$. It is a subspace of $\mathbb{R}^m$ (Week 5: span is always a subspace). Its dimension is the rank of $A$ — the number of pivot columns, or equivalently the size of any basis drawn from the columns.

Null space. The null space (or nullspace) of $A$ is the set of all inputs that map to zero:

It is a subspace of $\mathbb{R}^n$: if $Au = 0$ and $Av = 0$, then $A(u+v) = 0$ and $A(cu) = 0$. Geometrically, $N(A)$ is the set of directions that $A$ “kills.” Algebraically it is the solution set of the homogeneous system $Ax = 0$.

Worked picture. Take the $2 \times 3$ matrix

The second row is twice the first, so every column is a multiple of $(1,2)^\top$; every $Ax$ lies on that same line — a multiple of $\begin{pmatrix} 1 \\ 2 \end{pmatrix}$. Thus

a one-dimensional subspace of $\mathbb{R}^2$, and $\operatorname{rank}(A) = 1$. The null space lives in $\mathbb{R}^3$ and will turn out to be two-dimensional — we compute it explicitly below.

Solvability of $Ax = b$. The equation $Ax = b$ has a solution if and only if $b \in C(A)$. When a solution exists and $N(A)$ is nontrivial, solutions are never unique: if $Ax_0 = b$ and $n \in N(A)$, then $A(x_0 + n) = b$ as well. The general solution is a particular solution plus the null space.

Row Space and Left Null Space#

The same two constructions applied to the transpose $A^\top$ produce the remaining pair.

Row space. The rows of $A$ are the columns of $A^\top$, so the row space is

It is the span of the $m$ row vectors of $A$, viewed as vectors in $\mathbb{R}^n$. A fundamental fact from elimination is that row operations do not change the row space, and that

Column rank equals row rank: the number of independent rows equals the number of independent columns.

Left null space. The left null space is the null space of $A^\top$:

Vectors in $N(A^\top)$ are orthogonal to every column of $A$ (because $y^\top A = 0$ means $y$ is orthogonal to each column). Equivalently, $y^\top (Ax) = 0$ for every $x$, so $y$ is orthogonal to the entire column space. That is the seed of the orthogonal complements we will prove carefully in Week 7.

Where each subspace lives.

| Subspace | Symbol | Ambient space | Dimension | | --- | --- | --- | --- | | Column space | $C(A)$ | $\mathbb{R}^m$ | $r$ | | Null space | $N(A)$ | $\mathbb{R}^n$ | $n - r$ | | Row space | $C(A^\top)$ | $\mathbb{R}^n$ | $r$ | | Left null space | $N(A^\top)$ | $\mathbb{R}^m$ | $m - r$ |

Two live in $\mathbb{R}^n$ (row space and null space) and two live in $\mathbb{R}^m$ (column space and left null space). Their dimensions are completely determined by $m$, $n$, and the single number $r = \operatorname{rank}(A)$.

Picture. Think of two ambient spaces linked by $A$:

• Domain $\mathbb{R}^n$: row space $C(A^\top)$ (dim $r$) and null space $N(A)$ (dim $n-r$) sit side by side and will later turn out to be orthogonal complements.
• Codomain $\mathbb{R}^m$: column space $C(A)$ (dim $r$) and left null space $N(A^\top)$ (dim $m-r$) are the matching pair.

The map $A \colon \mathbb{R}^n \to \mathbb{R}^m$ sends the row space onto the column space and collapses the null space to $0$.

Computing the Null Space#

To find a basis for $N(A)$, solve $Ax = 0$ by elimination and express the pivot variables in terms of the free variables. Each free variable produces one special solution by setting that free variable to $1$ and the others to $0$. Those special solutions form a basis of $N(A)$.

Worked example. Continue with

Row-reduce: subtract $2$ times row 1 from row 2 to get

There is one pivot, in column 1, so $x_1$ is a basic (pivot) variable and $x_2, x_3$ are free. The single nonzero equation is

Special solution for $x_2 = 1$, $x_3 = 0$:

Special solution for $x_2 = 0$, $x_3 = 1$:

Check: $A n_1 = -2\begin{pmatrix}1\\2\end{pmatrix} + 1\begin{pmatrix}2\\4\end{pmatrix} + 0\begin{pmatrix}3\\6\end{pmatrix} = 0$, and similarly $A n_2 = 0$. The set $\{n_1, n_2\}$ is independent (the free-variable identity block in the last two coordinates makes that immediate) and spans every solution, so it is a basis of $N(A)$ and $\dim N(A) = 2$.

In general, after reduction to row echelon form with $r$ pivots and $n - r$ free variables, you get exactly $n - r$ special solutions — a basis of the null space.

The Dimension Formula#

The counts above are not an accident. For any $m \times n$ matrix $A$ of rank $r$,

This is the rank–nullity theorem (also called the dimension formula). Equivalently, $\dim N(A) = n - r$.

Why it holds. Elimination produces $r$ pivot variables and $n - r$ free variables. The free variables parametrize $N(A)$, so $\dim N(A) = n - r$. The $r$ pivot columns form a basis of $C(A)$, so $\operatorname{rank}(A) = r$. Adding these identities gives the formula.

Geometric meaning. The map $x \mapsto Ax$ “uses up” $r$ independent input directions (those that produce the $r$-dimensional column space) and “kills” the remaining $n - r$ directions (the null space). Input dimension splits cleanly into range contribution plus kernel:

For the transpose the same theorem says $\operatorname{rank}(A^\top) + \dim N(A^\top) = m$, and since $\operatorname{rank}(A^\top) = r$ we get $\dim N(A^\top) = m - r$ — completing the four dimension counts in the table above.

Check on the example. For our $2 \times 3$ matrix, $n = 3$ and $r = 1$, so $\dim N(A) = 3 - 1 = 2$, matching the two special solutions. Also $\dim N(A^\top) = 2 - 1 = 1$: the left null space is spanned by $(-2, 1)^\top$, because that vector is orthogonal to every column of $A$.

Kernel, Range, and the Four Subspaces#

Week 2 defined a linear transformation $T \colon \mathbb{R}^n \to \mathbb{R}^m$ by $T(x) = Ax$. The four subspaces are the kernel and range of $T$ and of $T^\top$ under different names.

Range of $T$. By definition,

So the column space is the range (image) of the transformation.

Kernel of $T$. Likewise,

The null space is the kernel. Rank–nullity then reads

the classical form for any linear map between finite-dimensional spaces.

The transpose map. The map $T^\top \colon \mathbb{R}^m \to \mathbb{R}^n$ given by $T^\top(y) = A^\top y$ has range $C(A^\top)$ (row space) and kernel $N(A^\top)$ (left null space). Everything we said about $A$ applies to $A^\top$ with the roles of $m$ and $n$ swapped.

Summary dictionary.

| Transformation language | Matrix language | | --- | --- | | $\operatorname{range}(T)$ for $T(x)=Ax$ | $C(A)$ | | $\ker(T)$ for $T(x)=Ax$ | $N(A)$ | | $\operatorname{range}(T^\top)$ | $C(A^\top)$ | | $\ker(T^\top)$ | $N(A^\top)$ |

Week 7 will add one more structural fact: within $\mathbb{R}^n$, the row space and null space are orthogonal complements of each other, and within $\mathbb{R}^m$ the column space and left null space are orthogonal complements. Week 13’s SVD will build orthonormal bases for all four of these spaces at once.

Knowledge check#

Browser lab#

Compute rank and a null-space basis with numpy only (SVD — no scipy). For an $m \times n$ matrix of rank $r$, the last $n - r$ right singular vectors (rows of $V^\top$, or columns of $V$) form an orthonormal basis of $N(A)$. That basis spans the same null space as the special solutions from elimination above, but the individual vectors need not match those special solutions numerically.

import numpy as np

A = np.array([[1., 2., 3.],
              [2., 4., 6.]])

# Full SVD: A = U @ diag(S) @ Vt  (Vt is V^T)
U, S, Vt = np.linalg.svd(A, full_matrices=True)
rank = np.linalg.matrix_rank(A)
# Right singular vectors for near-zero singular values: last n - rank rows of Vt
ns = Vt[rank:].T  # columns form a basis of N(A)

print("rank(A) =", rank)
print("null space basis (columns):\n", ns)
print("A @ ns ≈ 0?", np.allclose(A @ ns, 0))
print("dim N(A) should be n - r =", A.shape[1] - rank)