Week 1: Vectors and Linear Combinations

Vectors as Objects#

A vector is the basic object of linear algebra. In $\mathbb{R}^n$, a vector is an ordered list of $n$ real numbers. We write it as a column:

Each entry $v_i$ is a component (or coordinate). Two vectors are equal only when every corresponding component matches. The space $\mathbb{R}^n$ is the set of all such $n$-tuples; most of this course works in $\mathbb{R}^2$ and $\mathbb{R}^3$ for geometry, then lifts the same rules to higher dimensions used in data and machine learning.

Geometry in the plane. In $\mathbb{R}^2$, the vector $v = \begin{pmatrix} v_1 \\ v_2 \end{pmatrix}$ can be drawn as an arrow from the origin $(0,0)$ to the point $(v_1, v_2)$. The tip of the arrow is the point with those coordinates; the shaft is the directed segment from the origin. For example,

points three units right and two units up. In $\mathbb{R}^3$, the same idea holds in three axes; in $\mathbb{R}^n$ with $n > 3$ we keep the algebraic definition and drop the picture, but addition, scaling, and the dot product still work component-wise exactly as in the plane.

Vectors are not the same as free-floating points: a vector has direction and magnitude relative to a chosen origin and basis. In this course we always fix the standard basis of $\mathbb{R}^n$ unless we say otherwise, so the components of $v$ are exactly its coordinates in that basis.

Vector Addition and Scalar Multiplication#

Two fundamental operations turn the set of vectors into a vector space: addition and scalar multiplication.

Addition is component-wise. If

then

Geometrically, place the tail of $v$ at the tip of $u$; the diagonal of the parallelogram they form is $u + v$. That is the parallelogram law: the same sum is obtained whether you start with $u$ then add $v$, or start with $v$ then add $u$.

Scalar multiplication multiplies every component by a real number $c$:

If $c > 1$, the arrow stretches; if $0 < c < 1$, it shrinks; if $c < 0$, it flips direction and scales by $|c|$. The zero vector $0$ has every component zero; it is the unique vector with $v + 0 = v$ for all $v$.

Worked example. Let $u = \begin{pmatrix} 1 \\ 2 \end{pmatrix}$ and $v = \begin{pmatrix} 3 \\ -1 \end{pmatrix}$. Then

Algebra rules (we state them without proof; they follow from the same rules for real numbers, applied component-wise):

• Commutativity of addition: $u + v = v + u$
• Associativity of addition: $(u + v) + w = u + (v + w)$
• Distributivity: $c(u + v) = cu + cv$ and $(c + d)u = cu + du$
• Compatibility of scalars: $(cd)u = c(du)$
• Identity scalars: $1 \cdot u = u$ and $0 \cdot u = 0$

These rules are the entire algebraic interface of a vector space. Everything else in the course — linear combinations, matrices as maps, bases, and projections — is built from them.

Linear Combinations and Span#

A linear combination of vectors $v_1, v_2, \ldots, v_k$ is any vector of the form

where the coefficients $c_1, \ldots, c_k$ are real scalars. Linear combinations are how we build new vectors from a given set: add them after scaling each one independently.

Worked example. Take

in $\mathbb{R}^2$. Can we reach the target $b = \begin{pmatrix} 4 \\ 2 \end{pmatrix}$? We need scalars $c_1, c_2$ such that

Adding the two component equations: $2c_1 = 6$, so $c_1 = 3$. Subtracting: $2c_2 = 2$, so $c_2 = 1$. Check:

So $b = 3v_1 + v_2$ is a linear combination of $v_1$ and $v_2$.

Span. The span of a set of vectors $\{v_1, \ldots, v_k\}$ is the set of all linear combinations of those vectors:

If the span is all of $\mathbb{R}^n$, the set is said to span $\mathbb{R}^n$. In the example above, $v_1$ and $v_2$ are not parallel, so their span is the whole plane $\mathbb{R}^2$: every target in $\mathbb{R}^2$ is some linear combination of them. If instead both vectors lie on the same line through the origin, their span is only that line.

Span is the geometric answer to “what can these vectors generate?” Week 5 will refine this with linear independence and bases; for now, treat span as the reachable set under addition and scaling.

The Dot Product#

The dot product (also called the inner product on $\mathbb{R}^n$) of two vectors $x, y \in \mathbb{R}^n$ is the scalar

It is symmetric ($x \cdot y = y \cdot x$) and linear in each argument separately.

Length (norm). The length or Euclidean norm of $x$ is defined from the dot product with itself:

This is the ordinary distance from the origin to the tip of $x$. For example, $\|(3,4)\| = \sqrt{9 + 16} = 5$.

Angle between vectors. The geometric meaning of the dot product is the angle $\theta$ between two nonzero vectors:

with $\theta$ taken in $[0, \pi]$. When $\cos\theta > 0$ the angle is acute; when $\cos\theta < 0$ it is obtuse; when $\cos\theta = 0$ the vectors are perpendicular (next section).

Worked example. Let $x = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $y = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$. Then

So

That matches the picture: the unit vector along the $x$-axis and the diagonal of the unit square meet at $45^\circ$.

Orthogonality#

Two vectors $x$ and $y$ are orthogonal (written $x \perp y$) when their dot product is zero:

(for nonzero vectors; the zero vector is orthogonal to every vector by the same algebraic definition). Orthogonality is the coordinate-free way to say “perpendicular”: the angle formula gives $\cos\theta = 0$, so $\theta = 90^\circ$.

Examples. In $\mathbb{R}^2$, $\begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $\begin{pmatrix} 0 \\ 1 \end{pmatrix}$ are orthogonal. So are $\begin{pmatrix} 2 \\ 1 \end{pmatrix}$ and $\begin{pmatrix} -1 \\ 2 \end{pmatrix}$, because $2\cdot(-1) + 1\cdot 2 = 0$. In $\mathbb{R}^3$, the standard basis vectors $e_1, e_2, e_3$ are pairwise orthogonal.

Pythagorean theorem for vectors. When $x \perp y$, lengths add in the familiar squared form:

Why. Expand the left side with the dot product:

If $x \cdot y = 0$, the cross term vanishes and we recover $\|x\|^2 + \|y\|^2$. Conversely, if the Pythagorean identity holds, then $x \cdot y = 0$, so the vectors are orthogonal. This is the clean algebraic form of the classical theorem in the plane, and it holds in every $\mathbb{R}^n$.

Orthogonality will reappear constantly: orthogonal projections (Week 7), orthonormal bases and the QR idea later in the course, and least-squares solutions where residual vectors are orthogonal to the column space.

Knowledge check#

Browser lab#

Compute the dot product, length, and angle between two vectors, then visualize them.

import numpy as np
import matplotlib.pyplot as plt

u = np.array([2, 1])
v = np.array([1, 3])

dot = np.dot(u, v)
length_u = np.linalg.norm(u)
length_v = np.linalg.norm(v)
cos_theta = dot / (length_u * length_v)
angle_deg = np.degrees(np.arccos(cos_theta))

print(f"u . v = {dot}")
print(f"|u| = {length_u:.3f}, |v| = {length_v:.3f}")
print(f"angle between u and v = {angle_deg:.1f} degrees")

fig, ax = plt.subplots(figsize=(5, 5))
ax.quiver(0, 0, u[0], u[1], angles="xy", scale_units="xy", scale=1, color="tab:blue", label="u")
ax.quiver(0, 0, v[0], v[1], angles="xy", scale_units="xy", scale=1, color="tab:orange", label="v")
ax.set_xlim(-1, 4)
ax.set_ylim(-1, 4)
ax.set_aspect("equal")
ax.legend()
ax.set_title("Vectors u and v")
plt.show()