Probability and Statistics

A probability space is a triple $(\Omega, \mathcal{F}, P)$ where:

• $\Omega$ is the sample space (set of all possible outcomes).
• $\mathcal{F}$ is a sigma-algebra (collection of events) on $\Omega$.
• $P : \mathcal{F} \to [0,1]$ is a probability measure.

Definition. A sigma-algebra (or $\sigma$-algebra) $\mathcal{F}$ on a set $\Omega$ is a collection of subsets Of $\Omega$ satisfying:

1. $\Omega \in \mathcal{F}$.
2. If $A \in \mathcal{F}$Then $A^c \in \mathcal{F}$ (closed under complementation).
3. If $A_1, A_2, A_3, \ldots \in \mathcal{F}$Then $\bigcup_{i=1}^{\infty} A_i \in \mathcal{F}$ (closed under countable unions).

Remark. From (2) and (3), $\mathcal{F}$ is also closed under countable intersections (by De Morgan”s laws). The pair $(\Omega, \mathcal{F})$ is called a measurable space.

Example 1.1. For any set $\Omega$The trivial sigma-algebra is \\{\\emptyset, \\Omega\\} and the power Set $\mathcal{P}(\Omega)$ is also a sigma-algebra.

Example 1.2. If $\Omega = \{1, 2, 3, 4, 5, 6\}$ (a fair die), then $\mathcal{F} = \mathcal{P}(\Omega)$ contains all $2^6 = 64$ subsets. This is the sigma-algebra used for finite Sample spaces.

Example 1.3. For $\Omega = \mathbb{R}$The Borel sigma-algebra $\mathcal{B}$ is the smallest sigma-algebra Containing all open intervals. It is generated by taking countable unions, intersections, and complements of open Sets. We write $(\mathbb{R}, \mathcal{B})$.

Proposition 1.0. The intersection of any collection of sigma-algebras on $\Omega$ is a sigma-algebra.

Proof. Let \\{\\mathcal{{'}F{}'}_\\alpha\\} be a collection of sigma-algebras. Then: (1) $\Omega \in \mathcal{F}_\alpha$ for all $\alpha$So $\Omega \in \bigcap_\alpha \mathcal{F}_\alpha$. (2) If $A \in \bigcap_\alpha \mathcal{F}_\alpha$Then $A \in \mathcal{F}_\alpha$ for all $\alpha$So $A^c \in \mathcal{F}_\alpha$ for all $\alpha$Hence $A^c \in \bigcap_\alpha \mathcal{F}_\alpha$. (3) Countable unions follow similarly. $\blacksquare$

Intuition. This proposition guarantees that for any collection of subsets $\mathcal{E}$There exists a smallest Sigma-algebra containing $\mathcal{E}$Called the sigma-algebra generated by $\mathcal{E}$Denoted $\sigma(\mathcal{E})$.

1. Non-negativity: $P(A) \geq 0$ for all $A \in \mathcal{F}$.
2. Normalisation: $P(\Omega) = 1$.
3. Countable additivity: If $A_1, A_2, \ldots$ are pairwise disjoint events, then

$P\left(\bigcup_{i=1}^{\infty} A_i\right) = \sum_{i=1}^{\infty} P(A_i)$

Proposition 1.1. $P(\emptyset) = 0$.

Proof. $\Omega = \Omega \cup \emptyset$ (disjoint union), so $P(\Omega) = P(\Omega) + P(\emptyset)$ Hence $P(\emptyset) = 0$. $\blacksquare$

Proposition 1.2 (Complement). $P(A^c) = 1 - P(A)$.

Proposition 1.3 (Monotonicity). If $A \subseteq B$Then $P(A) \leq P(B)$.

Proof. Write $B = A \cup (B \setminus A)$A disjoint union. By countable additivity, $P(B) = P(A) + P(B \setminus A) \geq P(A)$ since $P(B \setminus A) \geq 0$. $\blacksquare$

Proposition 1.4 (Inclusion-Exclusion). For any two events $A, B$:

$P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Proof. Write $A \cup B = A \cup (B \setminus A)$ as a disjoint union. Then $P(A \cup B) = P(A) + P(B \setminus A)$. Since $B = (B \setminus A) \cup (A \cap B)$ is also disjoint, $P(B) = P(B \setminus A) + P(A \cap B)$So $P(B \setminus A) = P(B) - P(A \cap B)$. Substituting gives the Result. $\blacksquare$

Proposition 1.5 (General Inclusion-Exclusion). For events $A_1, \ldots, A_n$:

$P\left(\bigcup_{i=1}^n A_i\right) = \sum_{i} P(A_i) - \sum_{i \lt j} P(A_i \cap A_j) + \sum_{i \lt j \lt k} P(A_i \cap A_j \cap A_k) - \cdots + (-1)^{n+1} P(A_1 \cap \cdots \cap A_n)$

Proposition 1.6 (Boole’s Inequality). $P(A \cup B) \leq P(A) + P(B)$. More generally,

$P\left(\bigcup_{i=1}^n A_i\right) \leq \sum_{i=1}^n P(A_i)$

Definition. The conditional probability of $A$ given $B$ (where $P(B) \gt 0$) is

$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$

Intuition. Conditioning on $B$ restricts the sample space to $B$ and rescales so that $P(B \mid B) = 1$.

Theorem 1.1 (Law of Total Probability). If $B_1, \ldots, B_n$ partition $\Omega$ with $P(B_i) \gt 0$:

$P(A) = \sum_{i=1}^n P(A \mid B_i) P(B_i)$

Proof. Since $B_1, \ldots, B_n$ partition $\Omega$We have $A = \bigcup_{i=1}^n (A \cap B_i)$ (disjoint union). By countable additivity:

$P(A) = \sum_{i=1}^n P(A \cap B_i) = \sum_{i=1}^n P(A \mid B_i)\, P(B_i)$

$\blacksquare$

Theorem 1.2 (Bayes’ Theorem). For events $A$ and $B$ with $P(B) \gt 0$:

$P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$

In the partition form:

$P(B_j \mid A) = \frac{P(A \mid B_j) P(B_j)}{\sum_{i=1}^n P(A \mid B_i) P(B_i)}$

Proof. By definition, $P(A \mid B) = P(A \cap B)/P(B)$ and $P(B \mid A) = P(A \cap B)/P(A)$. Solving the Second for $P(A \cap B) = P(B \mid A) P(A)$ and substituting into the first gives Bayes’ theorem. The partition Form follows by applying the law of total probability to the denominator $P(B)$. $\blacksquare$

Problem 1.1. A bag contains 4 red and 6 blue marbles. Two marbles are drawn without replacement. What is the Probability that both are red?

Problem 1.2. A disease affects 1% of a population. A test has sensitivity 95% ($P(\mathrm{positive} \mid \mathrm{disease}) = 0.95$) And specificity 90% ($P(\mathrm{negative} \mid \mathrm{healthy}) = 0.90$). If a person tests positive, what is the Probability they have the disease?

Proposition 1.7 (General Inclusion-Exclusion). For events $A_1, \ldots, A_n$:

$P\left(\bigcup_{i=1}^n A_i\right) = \sum_{k=1}^n (-1)^{k+1} \sum_{1 \leq i_1 \lt \cdots \lt i_k \leq n} P(A_{i_1} \cap \cdots \cap A_{i_k})$

Proof. By induction on $n$. The base case $n = 1$ is trivial. Assume the result holds for $n$ events. For $n + 1$ Events:

$P\left(\bigcup_{i=1}^{n+1} A_i\right) = P\left(\bigcup_{i=1}^n A_i\right) + P(A_{n+1}) - P\left(\bigcup_{i=1}^n A_i \cap A_{n+1}\right)$

Applying the induction hypothesis to both union terms and rearranging yields the formula for $n + 1$. $\blacksquare$

Problem 1.3. Three machines A, B, C produce items. Machine A produces 50% of items with 2% defective, B produces 30% with 1% defective, and C produces 20% with 3% defective. An item is found to be defective. What is the Probability it was produced by machine A?

:::caution Common Pitfall People often confuse $P(A \mid B)$ with $P(B \mid A)$. In medical testing, $P(\mathrm{disease} \mid \mathrm{positive})$ Is much lower than $P(\mathrm{positive} \mid \mathrm{disease})$ due to low base rates. Always apply Bayes’ Theorem rigorously. :::

A random variable is a measurable function $X : \Omega \to \mathbb{R}$. Measurability means that for every Borel Set $B \subseteq \mathbb{R}$, $\{X^{-1}(B) : \omega \in \Omega, X(\omega) \in B\} \in \mathcal{F}$.

• Discrete random variable: takes values in a countable set.
• Continuous random variable: has a probability density function (PDF).

Example 2.1 (Discrete). Roll a fair die. Define $X(\omega) = \omega$. Then $X$ takes values in $\{1, 2, 3, 4, 5, 6\}$ With $P(X = k) = 1/6$ for each $k$.

Example 2.2 (Discrete — Indicator). For any event $A$The indicator random variable $\mathbf{1}_A$ equals 1 if $A$ occurs and 0 otherwise. Then $E[\mathbf{1}_A] = P(A)$ and $\mathrm{Var}(\mathbf{1}_A) = P(A)(1 - P(A))$.

Example 2.3 (Continuous). Let $X \sim \mathrm{Uniform}(0, 1)$. Then $X$ is the identity on $(0, 1)$ with PDF $f(x) = 1$ for $x \in (0, 1)$ and $f(x) = 0$ otherwise.

Example 2.4 (Mixed). A random variable can be neither purely discrete nor purely continuous. For instance, if $X = 0$ with probability $1/2$ and $X \sim \mathrm{Exp}(1)$ with probability $1/2$Then $X$ has an atom at 0 And a continuous part on $(0, \infty)$.

The CDF of a random variable $X$ is

$F_X(x) = P(X \leq x)$

Theorem 2.1 (Properties of the CDF).

1. $\lim_{x \to -\infty} F(x) = 0$, $\lim_{x \to +\infty} F(x) = 1$.
2. $F$ is non-decreasing: if $a \lt b$Then $F(a) \leq F(b)$.
3. $F$ is right-continuous: $\lim_{x \to a^+} F(x) = F(a)$.
4. $P(a \lt X \leq b) = F(b) - F(a)$.
5. $P(X = a) = F(a) - F(a^-)$ where $F(a^-) = \lim_{x \to a^-} F(x)$.
6. $F$ has at most countably many points of discontinuity.

Proof. (1) By monotonicity of $P$: $F(x) = P(X \leq x) \leq P(\Omega) = 1$ and as $x \to \infty$ $\{X \leq x\} \uparrow \Omega$So by continuity from below, $F(x) \to 1$. Similarly as $x \to -\infty$ $\{X \leq x\} \downarrow \emptyset$ and $F(x) \to 0$.

(2) If $a \lt b$Then $\{X \leq a\} \subseteq \{X \leq b\}$So by monotonicity of $P$ $F(a) \leq F(b)$.

(3) Let $x_n \downarrow a$. Then $\{X \leq x_n\} \downarrow \{X \leq a\}$ (since $\{X \leq a\} = \bigcap_n \{X \leq x_n\}$). By continuity from above of $P$, $F(x_n) \to F(a)$.

(4) $P(a \lt X \leq b) = P(X \leq b) - P(X \leq a) = F(b) - F(a)$.

(5) $P(X = a) = P(X \leq a) - P(X \lt a) = F(a) - \lim_{x \uparrow a} F(x) = F(a) - F(a^-)$.

(6) Since $F$ is non-decreasing, it can have at most countably many jump discontinuities (the sum of all jumps must Be bounded by 1). $\blacksquare$

For a discrete random variable $X$ with values $\{x_1, x_2, \ldots\}$:

$f_X(x) = P(X = x) = \begin{cases} p_i & \mathrm{if} x = x_i \\ 0 & \mathrm{otherwise} \end{cases}$

Where $p_i \geq 0$ and $\sum_i p_i = 1$.

A random variable $X$ is continuous if there exists a function $f_X \geq 0$ such that

$P(a \leq X \leq b) = \int_a^b f_X(x)\, dx$

And $\int_{-\infty}^{\infty} f_X(x)\, dx = 1$.

Note: $f_X(x)$ is not a probability; it is a probability density. For continuous $X$, $P(X = x) = 0$ For any individual $x$.

Proposition 2.1. If $X$ is a continuous random variable with PDF $f_X$ and $g$ is a strictly monotone Differentiable function, then $Y = g(X)$ has PDF

$f_Y(y) = f_X(g^{-1}(y)) \cdot \left|\frac{d}{dy} g^{-1}(y)\right|$

Proof. Suppose $g$ is strictly increasing. Then $F_Y(y) = P(Y \leq y) = P(X \leq g^{-1}(y)) = F_X(g^{-1}(y))$. Differentiating: $f_Y(y) = f_X(g^{-1}(y)) \cdot (g^{-1})'(y)$. For decreasing $g$The inequality reverses, Introducing a minus sign. Both cases are captured by the absolute value. $\blacksquare$

Problem 2.1. Let $X \sim \mathrm{Uniform}(0, 1)$. Find the distribution of $Y = -\ln X$.

Problem 2.2. Let $X \sim N(0, 1)$. Find the distribution of $Y = X^2$.

Definition. The quantile function (or inverse CDF) of a random variable $X$ with CDF $F$ is

$F^{-1}(p) = \inf\{x : F(x) \geq p\}, \quad 0 \lt p \lt 1$

Remark. If $F$ is strictly increasing, then $F^{-1}(p)$ is the unique $x$ such that $F(x) = p$. For discrete Distributions, $F^{-1}$ is the generalised inverse.

Theorem 2.2 (Probability Integral Transform). If $X$ has a continuous CDF $F$Then $F(X) \sim \mathrm{Uniform}(0, 1)$.

Proof. For $u \in (0, 1)$: $P(F(X) \leq u) = P(X \leq F^{-1}(u)) = F(F^{-1}(u)) = u$. $\blacksquare$

Intuition. This theorem is the foundation of inverse transform sampling: to generate from any distribution With CDF $F$Draw $U \sim \mathrm{Uniform}(0, 1)$ and compute $X = F^{-1}(U)$.

Definition. Let $X_1, \ldots, X_n$ be i.i.d. With CDF $F$ and PDF $f$. The order statistics $X_{(1)} \leq X_{(2)} \leq \cdots \leq X_{(n)}$ are the sorted values.

Theorem 2.3. The PDF of the $k$-th order statistic $X_{(k)}$ is

$f_{X_{(k)}}(x) = \frac{n!}{(k-1)!(n-k)!}\, [F(x)]^{k-1}\, [1 - F(x)]^{n-k}\, f(x)$

Proof. For $X_{(k)} \leq x$ to hold, at least $k$ of the $X_i$ must be $\leq x$. The event $X_{(k)} \in (x, x + dx)$ Requires exactly $k - 1$ observations below $x$One in $(x, x + dx)$And $n - k$ above $x + dx$:

$f_{X_{(k)}}(x)\, dx = \binom{n}{k-1, 1, n-k}\, [F(x)]^{k-1}\, f(x)\, dx\, [1 - F(x)]^{n-k}$

Which gives the result after cancelling $dx$. $\blacksquare$

Problem 2.3. Let $X_1, X_2, X_3$ be i.i.d. $\mathrm{Uniform}(0, 1)$. Find the PDF of the median $X_{(2)}$.

Bernoulli. $X \sim \mathrm{Bernoulli}(p)$: $P(X = 1) = p$, $P(X = 0) = 1 - p$.

$E[X] = p, \quad \mathrm{Var}(X) = p(1 - p)$

$M_X(t) = 1 - p + pe^t$

Proof of MGF: $M_X(t) = E[e^{tX}] = e^{t \cdot 0}(1 - p) + e^{t \cdot 1} \cdot p = 1 - p + pe^t$. $\blacksquare$

Binomial. $X \sim \mathrm{Bin}(n, p)$: number of successes in $n$ independent Bernoulli trials.

$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}, \quad k = 0, 1, \ldots, n$

$E[X] = np, \quad \mathrm{Var}(X) = np(1-p)$

$M_X(t) = (1 - p + pe^t)^n$

Proof of MGF: Since $X = \sum_{i=1}^n X_i$ where $X_i \sim \mathrm{Bernoulli}(p)$ are independent:

$M_X(t) = \prod_{i=1}^n M_{X_i}(t) = (1 - p + pe^t)^n \quad \blacksquare$

Geometric. $X \sim \mathrm{Geom}(p)$: number of trials until the first success (counting the success).

$P(X = k) = (1 - p)^{k-1} p, \quad k = 1, 2, 3, \ldots$

$E[X] = \frac{1}{p}, \quad \mathrm{Var}(X) = \frac{1 - p}{p^2}$

$M_X(t) = \frac{pe^t}{1 - (1 - p)e^t}, \quad \mathrm{for} t \lt -\ln(1 - p)$

Proof that $E[X] = 1/p$:

$E[X] = \sum_{k=1}^{\infty} k(1-p)^{k-1}p = p \sum_{k=1}^{\infty} k(1-p)^{k-1} = p \cdot \frac{1}{(1 - (1-p))^2} = \frac{p}{p^2} = \frac{1}{p}$

Using the identity $\sum_{k=1}^{\infty} kr^{k-1} = 1/(1-r)^2$ for $|r| \lt 1$. $\blacksquare$

Negative Binomial. $X \sim \mathrm{NegBin}(r, p)$: number of trials until the $r$-th success.

$P(X = k) = \binom{k-1}{r-1} p^r (1-p)^{k-r}, \quad k = r, r+1, \ldots$

$E[X] = \frac{r}{p}, \quad \mathrm{Var}(X) = \frac{r(1-p)}{p^2}$

Hypergeometric. $X \sim \mathrm{Hypergeometric}(N, K, n)$: sampling $n$ items without replacement from a population Of $N$ containing $K$ “successes.”

$P(X = k) = \frac{\binom{K}{k}\binom{N-K}{n-k}}{\binom{N}{n}}, \quad k = \max(0, n + K - N), \ldots, \min(n, K)$

$E[X] = n \cdot \frac{K}{N}, \quad \mathrm{Var}(X) = n \cdot \frac{K}{N} \cdot \frac{N-K}{N} \cdot \frac{N-n}{N-1}$

Remark. The factor $(N - n)/(N - 1)$ is the finite population correction. When $n \ll N$The Hypergeometric is well-approximated by $\mathrm{Bin}(n, K/N)$.

Poisson. $X \sim \mathrm{Poisson}(\lambda)$: models rare events occurring at rate $\lambda$.

$P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}, \quad k = 0, 1, 2, \ldots$

$E[X] = \lambda, \quad \mathrm{Var}(X) = \lambda$

$M_X(t) = \exp\left(\lambda(e^t - 1)\right)$

Proof of MGF:

$M_X(t) = \sum_{k=0}^{\infty} e^{tk} \frac{e^{-\lambda} \lambda^k}{k!} = e^{-\lambda} \sum_{k=0}^{\infty} \frac{(\lambda e^t)^k}{k!} = e^{-\lambda} \cdot e^{\lambda e^t} = \exp(\lambda(e^t - 1))$

$\blacksquare$

Proof that $E[X] = \lambda$:

$E[X] = \sum_{k=0}^{\infty} k \frac{e^{-\lambda} \lambda^k}{k!} = e^{-\lambda} \sum_{k=1}^{\infty} \frac{\lambda^k}{(k-1)!} = e^{-\lambda} \lambda \sum_{k=0}^{\infty} \frac{\lambda^k}{k!} = \lambda$

$\blacksquare$

Uniform. $X \sim \mathrm{Uniform}(a, b)$:

$f(x) = \begin{cases} \frac{1}{b - a} & \mathrm{if} a \leq x \leq b \\ 0 & \mathrm{otherwise} \end{cases}$

$E[X] = \frac{a + b}{2}, \quad \mathrm{Var}(X) = \frac{(b-a)^2}{12}$

$M_X(t) = \begin{cases} \frac{e^{tb} - e^{ta}}{t(b - a)} & \mathrm{if} t \neq 0 \\ 1 & \mathrm{if} t = 0 \end{cases}$

Gamma. $X \sim \mathrm{Gamma}(\alpha, \lambda)$ (shape $\alpha \gt 0$Rate $\lambda \gt 0$):

$f(x) = \frac{\lambda^\alpha}{\Gamma(\alpha)} x^{\alpha - 1} e^{-\lambda x}, \quad x \gt 0$

Where $\Gamma(\alpha) = \int_0^\infty t^{\alpha - 1} e^{-t}\, dt$ is the Gamma function.

$E[X] = \frac{\alpha}{\lambda}, \quad \mathrm{Var}(X) = \frac{\alpha}{\lambda^2}$

$M_X(t) = \left(\frac{\lambda}{\lambda - t}\right)^\alpha, \quad t \lt \lambda$

Remark. Special cases: $\mathrm{Gamma}(1, \lambda) = \mathrm{Exp}(\lambda)$; $\mathrm{Gamma}(n/2, 1/2) = \chi^2_n$.

Theorem 3.0 (Sum of Independent Gammas). If $X \sim \mathrm{Gamma}(\alpha_1, \lambda)$ and $Y \sim \mathrm{Gamma}(\alpha_2, \lambda)$ are independent, then $X + Y \sim \mathrm{Gamma}(\alpha_1 + \alpha_2, \lambda)$.

Proof. $M_{X+Y}(t) = M_X(t)\, M_Y(t) = \left(\frac{\lambda}{\lambda - t}\right)^{\alpha_1} \left(\frac{\lambda}{\lambda - t}\right)^{\alpha_2} = \left(\frac{\lambda}{\lambda - t}\right)^{\alpha_1 + \alpha_2}$, which is the MGF of $\mathrm{Gamma}(\alpha_1 + \alpha_2, \lambda)$. $\blacksquare$

Chi-Squared. $X \sim \chi^2_k$ (chi-squared with $k$ degrees of freedom):

$f(x) = \frac{1}{2^{k/2}\, \Gamma(k/2)}\, x^{k/2 - 1}\, e^{-x/2}, \quad x \gt 0$

$E[X] = k, \quad \mathrm{Var}(X) = 2k$

$M_X(t) = (1 - 2t)^{-k/2}, \quad t \lt 1/2$

Remark. If $Z_1, \ldots, Z_k \sim N(0, 1)$ are independent, then $\sum_{i=1}^k Z_i^2 \sim \chi^2_k$.

Beta. $X \sim \mathrm{Beta}(\alpha, \beta)$:

$f(x) = \frac{x^{\alpha - 1}(1 - x)^{\beta - 1}}{B(\alpha, \beta)}, \quad 0 \lt x \lt 1$

Where $B(\alpha, \beta) = \frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha + \beta)}$ is the Beta function.

$E[X] = \frac{\alpha}{\alpha + \beta}, \quad \mathrm{Var}(X) = \frac{\alpha \beta}{(\alpha + \beta)^2(\alpha + \beta + 1)}$

Exponential. $X \sim \mathrm{Exp}(\lambda)$:

$f(x) = \begin{cases} \lambda e^{-\lambda x} & \mathrm{if} x \geq 0 \\ 0 & \mathrm{if} x \lt 0 \end{cases}$

$E[X] = \frac{1}{\lambda}, \quad \mathrm{Var}(X) = \frac{1}{\lambda^2}$

Theorem 3.1 (Memoryless Property). If $X \sim \mathrm{Exp}(\lambda)$Then for all $s, t \gt 0$:

$P(X \gt s + t \mid X \gt s) = P(X \gt t)$

Proof.

$P(X \gt s + t \mid X \gt s) = \frac{P(X \gt s + t)}{P(X \gt s)} = \frac{e^{-\lambda(s+t)}}{e^{-\lambda s}} = e^{-\lambda t} = P(X \gt t)$

$\blacksquare$

Remark. The exponential distribution is the only continuous distribution with the memoryless property. This Makes it the natural model for waiting times between Poisson events.

Normal (Gaussian). $X \sim N(\mu, \sigma^2)$:

$f(x) = \frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right), \quad x \in \mathbb{R}$

$E[X] = \mu, \quad \mathrm{Var}(X) = \sigma^2$

The standard normal $Z \sim N(0,1)$ has CDF denoted $\Phi(z)$. For any $X \sim N(\mu, \sigma^2)$:

$Z = \frac{X - \mu}{\sigma} \sim N(0, 1)$

$M_X(t) = \exp\left(\mu t + \frac{\sigma^2 t^2}{2}\right)$

Verification that $f$ integrates to 1. Consider $I = \int_{-\infty}^{\infty} e^{-x^2/2}\, dx$. Then $I^2 = \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} e^{-(x^2+y^2)/2}\, dx\, dy$. Switching to polar coordinates: $x = r\cos\theta$, $y = r\sin\theta$, $dx\, dy = r\, dr\, d\theta$.

$I^2 = \int_0^{2\pi}\int_0^{\infty} e^{-r^2/2}\, r\, dr\, d\theta = 2\pi \int_0^{\infty} e^{-r^2/2}\, r\, dr = 2\pi \left[-e^{-r^2/2}\right]_0^{\infty} = 2\pi$

So $I = \sqrt{2\pi}$Confirming that $\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty} e^{-x^2/2}\, dx = 1$. For the general $N(\mu, \sigma^2)$The substitution $z = (x - \mu)/\sigma$ reduces to the standard case.

Verification that $E[X] = \mu$ for $Z \sim N(0, 1)$: The integrand $z \cdot \phi(z)$ is an odd function of $z$So $\int_{-\infty}^{\infty} z\, \phi(z)\, dz = 0$. For $X = \mu + \sigma Z$: $E[X] = \mu + \sigma \cdot 0 = \mu$.

Verification that $\mathrm{Var}(Z) = 1$: Integration by parts with $u = z$, $dv = z\, \phi(z)\, dz$:

$E[Z^2] = \int_{-\infty}^{\infty} z^2 \phi(z)\, dz = \left[-z\phi(z)\right]_{-\infty}^{\infty} + \int_{-\infty}^{\infty} \phi(z)\, dz = 0 + 1 = 1$

Proof of MGF for $Z \sim N(0, 1)$: