Random Variables

Definition. A random variable is a measurable function $X : \Omega \to \mathbb{R}$. The cumulative distribution function (CDF) of $X$ is

$F_X(x) = P(X \leq x)$

Proposition 2.1 (Properties of the CDF).

1. $F$ is non-decreasing: if $a \leq b$Then $F(a) \leq F(b)$.
2. $\lim_{x \to -\infty} F(x) = 0$ and $\lim_{x \to +\infty} F(x) = 1$.
3. $F$ is right-continuous: $\lim_{x \to a^+} F(x) = F(a)$.

Proof. (1) If $a \leq b$Then $\{X \leq a\} \subseteq \{X \leq b\}$So $F(a) = P(X \leq a) \leq P(X \leq b) = F(b)$ by Proposition 1.1(3).

(2) As $x \to -\infty$The events $\{X \leq x\}$ decrease to $\emptyset$So by continuity from above of probability measures, $F(x) \to 0$. As $x \to +\infty$The events increase to $\Omega$So $F(x) \to 1$.

(3) As $x \to a^+$The events $\{X \leq x\}$ decrease to $\{X \leq a\}$Giving right-continuity. $\blacksquare$

A random variable is discrete if its range is countable. The probability mass function (PMF) is $p_X(x) = P(X = x)$.

Definition (Expected Value). For a discrete random variable:

$E[X] = \sum_{x} x\, p_X(x)$

Provided the sum converges absolutely.

Definition (Variance). $\mathrm{Var}(X) = E[(X - \mu)^2] = E[X^2] - (E[X])^2$ where $\mu = E[X]$.

Proposition 2.2 (Linearity of Expectation). $E[aX + bY] = aE[X] + bE[Y]$ for any random variables $X$, $Y$ and constants $a$, $b$.

Proof. Direct computation from the definition of expected value. For the discrete case:

$E[aX + bY] = \sum_{x,y} (ax + by)\, p_{X,Y}(x,y) = a\sum_x x\, p_X(x) + b\sum_y y\, p_Y(y) = aE[X] + bE[Y]$

$\blacksquare$

A random variable is continuous if its CDF is absolutely continuous, i.e., there exists a probability density function (PDF) $f_X$ such that

$F_X(x) = \int_{-\infty}^{x} f_X(t)\, dt$

Key properties:

1. $f_X(x) \geq 0$ for all $x$.
2. $\int_{-\infty}^{\infty} f_X(x)\, dx = 1$.
3. $P(a \leq X \leq b) = \int_a^b f_X(x)\, dx$.
4. $P(X = a) = 0$ for any single point $a$.

Discrete distributions:

Continuous distributions:

Definition. $X \sim N(\mu, \sigma^2)$ if $X$ has PDF $f(x) = \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)$.

Theorem 2.3 (Standardisation). If $X \sim N(\mu, \sigma^2)$Then $Z = (X - \mu)/\sigma \sim N(0, 1)$.

Proof. The CDF of $Z$: $P(Z \leq z) = P(X \leq \mu + \sigma z) = \int_{-\infty}^{\mu + \sigma z} \frac{1}{\sigma\sqrt{2\pi}} e^{-t^2/2}\, dt$. Substituting $u = (t - \mu)/\sigma$: $= \int_{-\infty}^{z} \frac{1}{\sqrt{2\pi}} e^{-u^2/2}\, du$Which is the CDF of $N(0, 1)$. $\blacksquare$

Theorem 2.4 (Moment Generating Function). If $X \sim N(\mu, \sigma^2)$Then

$M_X(t) = E[e^{tX}] = \exp\left(\mu t + \frac{\sigma^2 t^2}{2}\right)$

Proof. $M_X(t) = \int_{-\infty}^{\infty} e^{tx} \frac{1}{\sigma\sqrt{2\pi}} e^{-(x-\mu)^2/(2\sigma^2)}\, dx$. Completing the square in the exponent and evaluating the Gaussian integral gives the result. $\blacksquare$

Definition. The moment generating function (MGF) of $X$ is $M_X(t) = E[e^{tX}]$ (when it exists in a neighbourhood of $t = 0$).

Theorem 2.5. If the MGF exists in a neighbourhood of 0, it uniquely determines the distribution. Furthermore, $E[X^n] = M_X^{(n)}(0)$.

Problem. Let $X \sim \mathrm{Poisson}(3)$ and $Y \sim \mathrm{Poisson}(5)$ be independent. Find the distribution of $X + Y$.