Week 5: Vector Spaces, Independence, and Basis

Vector Spaces and Subspaces#

A vector space is a set $V$ of objects (called vectors) together with two operations — addition and scalar multiplication — that stay inside $V$ and obey the usual algebraic rules from Week 1: addition is commutative and associative, there is a zero vector, every vector has an additive inverse, and scalars distribute and associate as expected. In this course the default space is $\mathbb{R}^n$, but the same language applies to matrices, polynomials, or functions once the operations are defined.

The practical test is closure: if $u, v \in V$ and $c \in \mathbb{R}$, then both $u + v$ and $cu$ must still lie in $V$. Combined with the presence of the zero vector, closure under addition and scalar multiplication is what makes a subset of a larger space into a vector space of its own.

A subspace is a vector space living inside a larger one. Formally, a nonempty subset $W \subseteq V$ is a subspace of $V$ when:

1. $0 \in W$,
2. $u, v \in W$ implies $u + v \in W$,
3. $u \in W$ and $c \in \mathbb{R}$ implies $cu \in W$.

(The full list of vector-space axioms then holds automatically because they already hold in $V$.)

Examples in $\mathbb{R}^2$ and $\mathbb{R}^3$.

• Any line through the origin in $\mathbb{R}^2$ is a subspace: if $v$ and $w$ both lie on that line, so do $v + w$ and $cv$. Algebraically it is $\operatorname{span}\{v\}$ for some nonzero $v$.
• Any plane through the origin in $\mathbb{R}^3$ is a subspace: it is $\operatorname{span}\{v_1, v_2\}$ for two non-parallel vectors in the plane.
• The set $\{0\}$ (only the zero vector) is a subspace — the trivial subspace.
• The whole space $\mathbb{R}^n$ is a subspace of itself.

Non-examples.

• A line that does not pass through the origin fails: it does not contain $0$, and the sum of two points on the line need not stay on the line when you treat them as vectors from the origin. (Affine lines are important in geometry, but they are not subspaces.)
• The unit circle $x^2 + y^2 = 1$ in $\mathbb{R}^2$ is not a subspace: the sum of two unit vectors is generally not a unit vector, and $0$ is not on the circle.
• The set of vectors with integer components is closed under addition but not under arbitrary real scalar multiplication (scale $(1,0)$ by $\frac{1}{2}$).

Span is always a subspace. For any vectors $v_1, \ldots, v_k$ in a vector space $V$,

is a subspace of $V$. It is the smallest subspace containing every $v_i$: any subspace that contains the $v_i$ must contain all their linear combinations. Week 1 introduced span as “what these vectors generate”; here we recognize that the generated set is itself a vector space.

Once you know a span is always a subspace, the remaining question is how to describe that subspace economically: which vectors in a generating set are redundant, mere combinations of the others? That is the question of linear independence.

Linear Independence#

A set of vectors $v_1, \ldots, v_k$ is linearly independent if the only linear combination that produces the zero vector is the trivial one:

If there exist coefficients, not all zero, that still sum to zero, the set is linearly dependent. Dependence means redundancy: if $c_1 \neq 0$, the equation $c_1 v_1 + \cdots + c_k v_k = 0$ rearranges to express $v_1$ as a combination of the rest.

Matrix test. Form the matrix $M$ whose columns are $v_1, \ldots, v_k$:

The equation $c_1 v_1 + \cdots + c_k v_k = 0$ is exactly $M c = 0$, where $c = (c_1, \ldots, c_k)^\top$. By Week 3, the nullspace of $M$ is nontrivial precisely when elimination produces fewer than $k$ pivots — that is, when

So:

• $\operatorname{rank}(M) = k$ (full column rank) $\iff$ the columns are independent,
• $\operatorname{rank}(M) < k$ $\iff$ the columns are dependent.

If $k > n$ (more vectors than the ambient dimension of $\mathbb{R}^n$), rank is at most $n$, so the vectors are automatically dependent.

Worked example. Take

in $\mathbb{R}^3$. Form

Elimination: subtract $2$ times row 1 from row 2, and $1$ times row 1 from row 3:

Replace row 3 by $\text{row}_3 - \frac{2}{3}\,\text{row}_2$:

There is a zero row and only two pivots, so $\operatorname{rank}(M) = 2 < 3$. The three vectors are dependent. Explicitly, the free variable $c_3$ gives a nontrivial null vector; you can check that

so $2v_1 - v_2 - v_3 = 0$ with coefficients not all zero. Any two of $\{v_1, v_2\}$ (or $\{v_1, v_3\}$, or $\{v_2, v_3\}$) are independent and span the same plane.

Independence strips away redundancy by itself, but it does not guarantee that the set reaches every vector in the space. Spanning guarantees full coverage but may carry excess. A basis fuses the two ideas: a minimal generating set — independent enough to have no waste, large enough to touch every point.

Basis and Dimension#

A basis of a vector space $V$ is a set of vectors that is both:

1. Linearly independent, and
2. Spanning — every vector in $V$ is a linear combination of the basis vectors.

Standard basis. The standard basis of $\mathbb{R}^n$ is

These $n$ vectors are independent (the matrix they form is $I$, rank $n$) and span $\mathbb{R}^n$ (any $x = (x_1, \ldots, x_n)^\top$ equals $x_1 e_1 + \cdots + x_n e_n$). So $\{e_1, \ldots, e_n\}$ is a basis of $\mathbb{R}^n$.

Same number of vectors. A standard theorem of linear algebra — see Strang, §3.4 — says that every basis of a given space has the same number of vectors. That common number is the dimension of the space, written $\dim(V)$. In particular:

and if $W = \operatorname{span}\{v_1, \ldots, v_k\}$ with the $v_i$ independent, then $\dim(W) = k$. More generally, the dimension of a span equals the rank of the matrix whose columns are the spanning vectors: drop dependent columns until you have a basis, and count what remains.

Finding a basis from a spanning set. Start with vectors that span $V$ (for example, the columns of a matrix). Run elimination and keep only the columns that produce pivots; those pivot columns form a basis of the column space. The number of pivots is both the rank and the dimension of that subspace.

Examples.

• A line through the origin in $\mathbb{R}^3$ has dimension $1$: any single nonzero vector on the line is a basis.
• A plane through the origin has dimension $2$: two independent vectors in the plane form a basis.
• The three dependent vectors $v_1, v_2, v_3$ from the previous section span a $2$-dimensional subspace of $\mathbb{R}^3$; a basis is $\{v_1, v_2\}$.

By the same theorem (Strang, §3.4), a set with more vectors than $\dim(V)$ cannot be independent, and one with fewer cannot span $V$: either would force a basis with the wrong number of vectors. A basis hits the count exactly and satisfies both properties.

Coordinates Relative to a Basis#

Once a basis $B = \{b_1, \ldots, b_n\}$ of $V$ is fixed, every vector $v \in V$ can be written uniquely as

The scalars $(c_1, \ldots, c_n)$ are the coordinates of $v$ relative to $B$, often written $[v]_B$. Uniqueness is exactly linear independence: if two coefficient lists produced the same $v$, their difference would be a nontrivial combination equal to zero. Existence is exactly spanning.

In the standard basis of $\mathbb{R}^n$, the coordinates of a vector are just its usual components. In a different basis, the same geometric vector has different numbers attached to it — the change-of-basis story that returns when you study similarity and diagonalization later in the course.

Worked example. Let

in $\mathbb{R}^2$. These are independent (neither is a scalar multiple of the other) and there are two of them in a $2$-dimensional space, so $B = \{b_1, b_2\}$ is a basis of $\mathbb{R}^2$. For $v = \begin{pmatrix} 4 \\ 2 \end{pmatrix}$, solve $c_1 b_1 + c_2 b_2 = v$:

Adding the two equations: $2c_1 = 6$, so $c_1 = 3$. Subtracting: $2c_2 = 2$, so $c_2 = 1$. Thus

meaning $v = 3b_1 + 1\cdot b_2$ (as in the Week 1 span example). In the standard basis the same vector is simply $(4, 2)^\top$. The vector did not change — only the labels relative to the chosen basis did.

Why bases matter. Coordinates turn abstract linear algebra into numerical linear algebra: matrices store the action of a linear map on a basis, and changing basis changes the matrix without changing the map. Week 6 will use independence and dimension to classify the four fundamental subspaces of a matrix (column space, nullspace, row space, left nullspace) and relate their dimensions by the rank–nullity theorem.

Knowledge check#

Browser lab: rank and independence#

Test three vectors for independence using numpy.linalg.matrix_rank.

import numpy as np

v1 = np.array([1, 2, 1])
v2 = np.array([2, 1, 0])
v3 = np.array([0, 3, 2])

M = np.column_stack([v1, v2, v3])
rank = np.linalg.matrix_rank(M)

print("M =\n", M)
print("rank(M) =", rank)
print("Independent:", rank == M.shape[1])

print("Notice: rank(M) = 2 < 3, so Independent prints False — v3 = 2v1 - v2.")

Done when#

You can test a set of vectors for linear independence by computing the rank of the matrix they form, and give a basis and dimension for a given space. Check it against the browser lab: for $v_1=(1,2,1)$, $v_2=(2,1,0)$, $v_3=(0,3,2)$ the output shows rank(M) = 2 and Independent: False, and you can exhibit the dependence $v_2 + v_3 = 2v_1$.

Further Reading#

• Strang, G. Introduction to Linear Algebra, 5th ed. — Chapters 3.1–3.4 cover vector spaces, independence, basis, and dimension with worked examples that complement this week's treatment.
• 3Blue1Brown — Essence of Linear Algebra YouTube series. Episodes "Linear combinations, span, and basis vectors" and "Linear independence" offer geometric intuition for the abstract ideas in this week.
• Lay, D., Lay, S., and McDonald, J. Linear Algebra and Its Applications, 6th ed. — Chapter 2.8–2.9 on subspaces of $\mathbb{R}^n$, basis, and dimension from an applied perspective.