Inner Product Spaces and Hilbert Spaces

An inner product space is a vector space $H$ with an inner product $\langle \cdot, \cdot \rangle : H \times H \to \mathbb{C}$ satisfying:

1. $\langle x, x\rangle \geq 0$ with equality iff $x = 0$.
2. $\langle x, y\rangle = \overline{\langle y, x\rangle}$.
3. $\langle \alpha x + \beta y, z\rangle = \alpha\langle x, z\rangle + \beta\langle y, z\rangle$.

Every inner product induces a norm: $\|x\| = \sqrt{\langle x, x\rangle}$.

Example 1. $\mathbb{C}^n$ with $\langle x, y\rangle = \sum_{i=1}^n x_i \overline{y_i}$.

Example 2. $L^2(\mu)$ with $\langle f, g\rangle = \int f \overline{g}\, d\mu$.

Example 3. $\ell^2$ with $\langle x, y\rangle = \sum_{i=1}^\infty x_i \overline{y_i}$.

Vectors $x, y \in H$ are orthogonal (written $x \perp y$) if $\langle x, y\rangle = 0$.

Theorem 2.1 (Pythagorean Theorem). If $x \perp y$, then $\|x + y\|^2 = \|x\|^2 + \|y\|^2$.

Theorem 2.2 (Parallelogram Law). In any inner product space:

$\|x + y\|^2 + \|x - y\|^2 = 2\|x\|^2 + 2\|y\|^2$

Theorem 2.3 (Polarization Identity). In a complex inner product space:

$\langle x, y\rangle = \frac{1}{4}\left(\|x + y\|^2 - \|x - y\|^2 + i\|x + iy\|^2 - i\|x - iy\|^2\right)$

Theorem 2.4 (Cauchy-Schwarz Inequality). $|\langle x, y\rangle| \leq \|x\| \cdot \|y\|$ with equality iff $x$ and $y$ are linearly dependent.

A Hilbert space is a complete inner product space.

Theorem 2.5 (Orthogonal Projection). Let $M$ be a closed subspace of a Hilbert space $H$. For every $x \in H$, there exists a unique $y \in M$ (the orthogonal projection of $x$ onto $M$) such that $x - y \perp M$. We write $y = P_M(x)$.

Theorem 2.6 (Orthogonal Decomposition). If $M$ is a closed subspace of $H$, then $H = M \oplus M^\perp$, where $M^\perp = \{x \in H : x \perp M\}$.

A set $\{e_i\}_{i \in I} \subseteq H$ is an orthonormal system if $\langle e_i, e_j\rangle = \delta_{ij}$.

Theorem 2.7 (Bessel’s Inequality). If $\{e_i\}_{i=1}^n$ is an orthonormal set, then $\sum_{i=1}^n |\langle x, e_i\rangle|^2 \leq \|x\|^2$.

Theorem 2.8. A Hilbert space is separable if and only if it admits a countable orthonormal basis.

Theorem 2.9 (Parseval’s Identity). If $\{e_n\}$ is an orthonormal basis for $H$, then for every $x \in H$:

$\|x\|^2 = \sum_{n=1}^{\infty} |\langle x, e_n\rangle|^2 \quad \text{and} \quad x = \sum_{n=1}^{\infty} \langle x, e_n\rangle e_n$

Theorem 2.10 (Riesz Representation). Let $H$ be a Hilbert space. For every bounded linear functional $\varphi \in H^*$, there exists a unique $y \in H$ such that $\varphi(x) = \langle x, y\rangle$ for all $x \in H$. Moreover, $\|\varphi\|_{H^*} = \|y\|_H$.

Proof. If $\varphi = 0$, take $y = 0$. Otherwise, $\ker(\varphi)$ is a closed subspace, so $H = \ker(\varphi) \oplus \ker(\varphi)^\perp$. Take $z \in \ker(\varphi)^\perp$ with $\|z\| = 1$. Then $y = \overline{\varphi(z)} \cdot z$ satisfies $\varphi(x) = \langle x, y\rangle$ for all $x$. Uniqueness follows from the polarization identity. $\blacksquare$

Corollary 2.11. Every Hilbert space is isometrically isomorphic to its dual: $H \cong H^*$ (anti-linearly).

Every Hilbert space is a Banach space, but the converse fails: $L^p$ for $p \neq 2$ is Banach but not Hilbert. The parallelogram law characterises normed spaces whose norm comes from an inner product.

• Assuming every Cauchy sequence converges in an inner product space. Completeness is an extra requirement. For example, $C([0,1])$ with the $L^2$ inner product is not complete.
• Confusing orthogonality with linear independence. Orthogonal vectors are always linearly independent, but linearly independent vectors need not be orthogonal.
• Forgetting that $M^\perp$ is closed even when $M$ is not. The orthogonal complement is always a closed subspace, regardless of whether $M$ itself is closed.
• Assuming $\langle x, y\rangle = \langle y, x\rangle$ in complex spaces. Conjugate symmetry means $\langle x, y\rangle = \overline{\langle y, x\rangle}$, not equality.

• Quantum mechanics: States are vectors in a Hilbert space; observables are self-adjoint operators; inner products give probability amplitudes.
• Signal processing: $L^2(\mathbb{R})$ is the space of finite-energy signals; orthonormal bases (Fourier, wavelet) enable efficient compression and denoising.
• Machine learning: Kernel methods map data into a reproducing kernel Hilbert space (RKHS) where inner products correspond to kernel evaluations.
• Numerical analysis: The Galerkin method projects PDE solutions onto finite-dimensional subspaces using orthogonal projection.

Problem 1. Let $H = L^2([0,1])$ with inner product $\langle f,g\rangle = \int_0^1 f(x)\overline{g(x)}\,dx$. Show that $\{e^{2\pi i n x}\}_{n\in\mathbb{Z}}$ is an orthonormal system.

Solution. $\langle e^{2\pi i n x}, e^{2\pi i m x}\rangle = \int_0^1 e^{2\pi i (n-m)x}\,dx = \delta_{nm}$ by orthogonality of complex exponentials. So the system is orthonormal. Completeness (that it forms a basis) is the statement of the Fourier series convergence theorem. $\blacksquare$

Problem 2. Let $M = \{f \in L^2([0,1]) : \int_0^1 f(x)\,dx = 0\}$. Find $M^\perp$.

Solution. $M^\perp = \{g \in L^2([0,1]) : \langle f,g\rangle = 0 \text{ for all } f \in M\}$. The constant function $h(x) = c$ satisfies $\langle f, h\rangle = c\int_0^1 f = 0$ for all $f \in M$. Conversely, if $g \in M^\perp$, write $g = \bar{g} + c$ where $\bar{g} \in M$ and $c = \int_0^1 g$. Then $0 = \langle \bar{g}, g\rangle = \|\bar{g}\|^2 + c\int \bar{g} = \|\bar{g}\|^2$, so $\bar{g}=0$ and $g$ is constant. Hence $M^\perp$ is the 1-dimensional space of constant functions. $\blacksquare$