Week 1: Probability Spaces and Events

Two lists that share a name#

Sixty students take calculus and forty-five take linear algebra. Twenty take both. Adding 60 and 45 counts those twenty twice.

That correction is the whole of this week. A sample space lists every outcome. An event is the subset you asked about. A probability adds those outcomes once each.

Sample spaces and events#

A sample space $\Omega$ is the set of outcomes an experiment can produce.

• A fair coin has $\Omega = \{\text{H}, \text{T}\}$.
• A six-sided die has $\Omega = \{1, 2, 3, 4, 5, 6\}$.
• Tomorrow's temperature in °C has $\Omega = \mathbb{R}$.
• Two coin tosses have $\Omega = \{\text{HH}, \text{HT}, \text{TH}, \text{TT}\}$.

An event is a subset of $\Omega$. On the die, "even" is $E = \{2, 4, 6\}$ and "at least 5" is $F = \{5, 6\}$.

Events are sets, so the set operations are the event operations.

• Union $A \cup B$ means $A$ or $B$ or both.
• Intersection $A \cap B$ means $A$ and $B$.
• Complement $A^c = \Omega \setminus A$ means not $A$.
• Disjoint events have $A \cap B = \varnothing$. They cannot happen on the same roll.

On the die, $E \cap F = \{6\}$ and $E \cup F = \{2, 4, 5, 6\}$.

Kolmogorov axioms#

A probability $P$ assigns each event a number in $[0, 1]$. Kolmogorov (1933) required three rules.

1. Non-negativity. $P(A) \ge 0$ for every event $A$.
2. Normalisation. $P(\Omega) = 1$. Some outcome occurs.
3. Additivity. If $A_1, \ldots, A_k$ are pairwise disjoint, then

Disjoint events add. The countable form is the same rule on an infinite list.

Four consequences fall out of the three rules.

• $P(\varnothing) = 0$.
• Complement. $P(A^c) = 1 - P(A)$.
• Monotonicity. If $A \subseteq B$, then $P(A) \le P(B)$.
• Inclusion-exclusion.

The subtraction is the picture at the top of the page. In a class of 100, with $P(A) = 60/100$, $P(B) = 45/100$, and $P(A \cap B) = 20/100$,

The complement says $P(\text{neither}) = 1 - 0.85 = 0.15$.

Finite spaces and counting#

When $\Omega$ is finite and every outcome is equally likely,

Roll two fair dice. There are 36 ordered pairs. The pairs that sum to 7 are $(1,6)$, $(2,5)$, $(3,4)$, $(4,3)$, $(5,2)$, and $(6,1)$.

The same counting identity, divided by $|\Omega|$, is inclusion-exclusion. $|A \cup B| = |A| + |B| - |A \cap B|$.

Discrete and continuous#

A discrete space lists outcomes and adds them. A continuous space integrates a density $f(x) \ge 0$. One real number has probability zero.

Week 2 builds the density, the mass function, and the cdf. The three axioms are the same in both settings. Discrete models add. Continuous models integrate.

$P(X = 3.7) = 0$ in a continuous model even when the density $f(3.7)$ is positive. The density is probability per unit of $x$. The probability is the area.

Knowledge check#

Browser lab#

Simulate die rolls and compare empirical frequencies to the axiom-based probabilities of events.

import numpy as np
import matplotlib.pyplot as plt

def roll_die(n):
    return np.random.randint(1, 7, size=n)

n_rolls = 6000
rolls = roll_die(n_rolls)

events = {
    "Even (2,4,6)": (rolls % 2 == 0),
    "≥ 5 (5,6)": (rolls >= 5),
    "Prime (2,3,5)": np.isin(rolls, [2, 3, 5]),
    "Divisible by 3 (3,6)": (rolls % 3 == 0),
}

true_probs = {"Even (2,4,6)": 0.5, "≥ 5 (5,6)": 1/3,
              "Prime (2,3,5)": 0.5, "Divisible by 3 (3,6)": 1/3}

fig, axes = plt.subplots(2, 2, figsize=(8, 7))
for ax, (name, mask) in zip(axes.flat, events.items()):
    empirical = mask.sum() / n_rolls
    ax.bar(["Axiom (true)", "Empirical"], [true_probs[name], empirical],
           color=["tab:blue", "tab:orange"])
    ax.set_ylim(0, 1)
    ax.set_title(name)
    ax.set_ylabel("Probability")

fig.suptitle(f"Empirical vs True Probabilities ({n_rolls} rolls)", fontsize=13)
plt.tight_layout()
plt.show()

print("Empirical frequencies:")
for name, mask in events.items():
    print(f"  {name}: {mask.sum() / n_rolls:.4f}  (true: {true_probs[name]:.4f})")

print("Notice: every empirical frequency tracks its true value (0.5087 vs 0.5, 0.3363 vs 1/3) — the remaining gaps are sampling noise.")

Key takeaways#

• A model is a sample space, events as subsets, and a measure obeying non-negativity, normalisation, and additivity.
• The complement rule is $P(A^c) = 1 - P(A)$. Overlapping events use $P(A \cup B) = P(A) + P(B) - P(A \cap B)$.
• Equally likely outcomes reduce probability to counting, $P(A) = |A| / |\Omega|$.
• Discrete models add point masses. Continuous models integrate a density, and $P(X = c) = 0$ at every single point.

Further reading#

• Joseph K. Blitzstein & Jessica Hwang, Introduction to Probability (2nd ed., CRC Press, 2019): the axiomatic development plus counting, with worked examples.
• George Casella & Roger L. Berger, Statistical Inference (2nd ed., Duxbury, 2002): Chapter 1 sets up probability from set theory.
• Larry Wasserman, All of Statistics (Springer, 2004): a fast, modern tour that starts here and connects directly to statistics and ML.