Week 1: Vectors and Linear Combinations

Vectors as Objects#

A vector is the basic object of linear algebra. In $\mathbb{R}^n$, a vector is an ordered list of $n$ real numbers. We write it as a column:

Each entry $v_i$ is a component (or coordinate). Two vectors are equal only when every corresponding component matches. The space $\mathbb{R}^n$ is the set of all such $n$-tuples.

Geometry in the plane. In $\mathbb{R}^2$, the vector $v = \begin{pmatrix} v_1 \\ v_2 \end{pmatrix}$ can be drawn as an arrow from the origin $(0,0)$ to the point $(v_1, v_2)$. The tip of the arrow is the point with those coordinates; the shaft is the directed segment from the origin. For example,

points three units right and two units up. In $\mathbb{R}^3$, the same idea holds across three axes. For $n > 3$ we keep the algebraic definition and drop the picture; addition, scaling, and the dot product still work component-wise, exactly as in the plane. Those are the dimensions where data and machine learning actually live.

Vectors are not the same as free-floating points: a point has a location, while a vector carries a direction and magnitude of its own, independent of where it is drawn. Its components are the coordinates the vector receives in a chosen basis; in this course we always fix the standard basis of $\mathbb{R}^n$ unless we say otherwise.

Vector Addition and Scalar Multiplication#

Two fundamental operations turn the set of vectors into a vector space: addition and scalar multiplication.

Addition is component-wise. If

then

Geometrically, place the tail of $v$ at the tip of $u$, or the other way around; the two arrows form a parallelogram, and $u + v$ is its diagonal. This picture is the parallelogram law.

Scalar multiplication multiplies every component by a real number $c$:

If $c > 1$, the arrow stretches; if $0 < c < 1$, it shrinks; if $c < 0$, it flips direction and scales by $|c|$. The zero vector $0$ has every component zero; it is the unique vector with $v + 0 = v$ for all $v$.

Worked example. Let $u = \begin{pmatrix} 1 \\ 2 \end{pmatrix}$ and $v = \begin{pmatrix} 3 \\ -1 \end{pmatrix}$. Then

Algebra rules (we state them without proof; they follow from the same rules for real numbers, applied component-wise):

• Commutativity of addition: $u + v = v + u$
• Associativity of addition: $(u + v) + w = u + (v + w)$
• Distributivity: $c(u + v) = cu + cv$ and $(c + d)u = cu + du$
• Compatibility of scalars: $(cd)u = c(du)$
• Identity scalars: $1 \cdot u = u$ and $0 \cdot u = 0$

These rules are the entire algebraic interface of a vector space. Everything else in the course — linear combinations, matrices as maps, bases, and projections — is built from them.

Linear Combinations and Span#

A linear combination of vectors $v_1, v_2, \ldots, v_k$ is any vector of the form

where the coefficients $c_1, \ldots, c_k$ are real scalars.

Worked example. Take

in $\mathbb{R}^2$. Can we reach the target $b = \begin{pmatrix} 4 \\ 2 \end{pmatrix}$? We need scalars $c_1, c_2$ such that

Adding the two component equations: $2c_1 = 6$, so $c_1 = 3$. Subtracting: $2c_2 = 2$, so $c_2 = 1$. Check:

So $b = 3v_1 + v_2$ is a linear combination of $v_1$ and $v_2$.

Span. The span of a set of vectors $\{v_1, \ldots, v_k\}$ is the set of all linear combinations of those vectors:

If the span is all of $\mathbb{R}^n$, the set is said to span $\mathbb{R}^n$. In the example above, $v_1$ and $v_2$ are not parallel, so their span is the whole plane $\mathbb{R}^2$: every target in $\mathbb{R}^2$ is some linear combination of them. If instead both vectors lie on the same line through the origin, their span is only that line.

Span is the geometric answer to “what can these vectors generate?” Week 5 refines this into the notions of linear independence and bases.

The span of two non-parallel vectors is the whole plane. The target $b$ is assembled as $3v_1 + v_2$; the dashed edges of the parallelogram are the two component vectors.

The Dot Product#

The dot product (also called the inner product on $\mathbb{R}^n$) of two vectors $x, y \in \mathbb{R}^n$ is the scalar

It is symmetric ($x \cdot y = y \cdot x$) and linear in each argument separately; both facts follow from writing the sums out component-wise.

Length (norm). The length or Euclidean norm of $x$ is defined from the dot product with itself:

This is the ordinary distance from the origin to the tip of $x$. For example, $\|(3,4)\| = \sqrt{9 + 16} = 5$.

Angle between vectors. The geometric content of the dot product is the angle $\theta$ between two nonzero vectors:

with $\theta$ taken in $[0, \pi]$. To see why, draw the triangle with vertices $0$, $x$, and $y$: the side opposite $\theta$ has length $\|x - y\|$, so by the law of cosines (a standard plane-geometry identity),

Expanding the same length with the dot product instead gives

Equating the two expressions, the $\|x\|^2$ and $\|y\|^2$ terms cancel, leaving $x \cdot y = \|x\|\,\|y\|\cos\theta$.

Worked example. Let $x = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $y = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$. Then

So

That matches the picture: the unit vector along the $x$-axis and the diagonal of the unit square meet at $45^\circ$.

Orthogonality#

Two vectors $x$ and $y$ are orthogonal (written $x \perp y$) when their dot product is zero:

(for nonzero vectors; the zero vector is orthogonal to every vector by the same algebraic definition). Orthogonality is the coordinate-free way to say “perpendicular”: from the angle formula, $x \cdot y = 0$ forces $\cos\theta = 0$, i.e. $\theta = 90^\circ$.

Examples. In $\mathbb{R}^2$, $\begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $\begin{pmatrix} 0 \\ 1 \end{pmatrix}$ are orthogonal. So are $\begin{pmatrix} 2 \\ 1 \end{pmatrix}$ and $\begin{pmatrix} -1 \\ 2 \end{pmatrix}$, because $2\cdot(-1) + 1\cdot 2 = 0$. In $\mathbb{R}^3$, the standard basis vectors $e_1, e_2, e_3$ are pairwise orthogonal.

Pythagorean theorem for vectors. When $x \perp y$, lengths add in the familiar squared form:

Why. Expand the left side with the dot product:

If $x \cdot y = 0$, the cross term vanishes and we recover $\|x\|^2 + \|y\|^2$. Conversely, if the Pythagorean identity holds, then $x \cdot y = 0$, so the vectors are orthogonal. This is the clean algebraic form of the classical theorem in the plane, and it holds in every $\mathbb{R}^n$.

Orthogonality will reappear constantly: orthogonal projections (Week 7), orthonormal bases and the QR idea later in the course, and least-squares solutions where residual vectors are orthogonal to the column space.

Knowledge check#

Browser lab: dot product, length, and angle#

Compute the dot product, length, and angle between two vectors, then visualize them.

import numpy as np
import matplotlib.pyplot as plt

u = np.array([2, 1])
v = np.array([1, 3])

dot = np.dot(u, v)
length_u = np.linalg.norm(u)
length_v = np.linalg.norm(v)
cos_theta = dot / (length_u * length_v)
angle_deg = np.degrees(np.arccos(cos_theta))

print(f"u . v = {dot}")
print(f"|u| = {length_u:.3f}, |v| = {length_v:.3f}")
print(f"angle between u and v = {angle_deg:.1f} degrees")

fig, ax = plt.subplots(figsize=(5, 5))
ax.quiver(0, 0, u[0], u[1], angles="xy", scale_units="xy", scale=1, color="tab:blue", label="u")
ax.quiver(0, 0, v[0], v[1], angles="xy", scale_units="xy", scale=1, color="tab:orange", label="v")
ax.set_xlim(-1, 4)
ax.set_ylim(-1, 4)
ax.set_aspect("equal")
ax.legend()
ax.set_title("Vectors u and v")
plt.show()

print("Notice: u·v = 5 gives cos θ ≈ 0.707, so the angle between the arrows is 45°.")

Done when#

You can add and scale vectors, express a target vector as an explicit linear combination of two given vectors (or show it is impossible), and use the dot product to report a length, an angle, and orthogonality — all without notes. Check it against the browser lab: for u = (2, 1) and v = (1, 3) the printed output must read u . v = 5, |u| = 2.236, |v| = 3.162, and angle ... = 45.0 degrees, and each number must match your own hand calculation.

Further Reading#

• Gilbert Strang, Introduction to Linear Algebra (5th ed., Wellesley-Cambridge Press, 2016). Chapters 1–2 cover vectors, linear combinations, and dot products with exceptional geometric clarity.
• David C. Lay, Steven R. Lay & Judi J. McDonald, Linear Algebra and Its Applications (6th ed., Pearson, 2021). Chapter 1 provides a thorough treatment of vectors in $\mathbb{R}^n$ with abundant exercises.
• Stephen Boyd & Lieven Vandenberghe, Introduction to Applied Linear Algebra (Cambridge University Press, 2018). Chapters 1–3 present vectors and linear combinations from an applied data-science perspective. Freely available at vmls-book.stanford.edu.
• 3Blue1Brown, Essence of Linear Algebra (YouTube playlist). The first three videos (“Vectors, what even are they?”, “Linear combinations, span, and basis vectors”, “Linear transformations and matrices”) are outstanding geometric companions to this week’s material.
• Khan Academy, Vectors and Spaces module. Free interactive exercises on vector addition, scalar multiplication, and linear combinations.