Laws of large numbers
Section of Expectation and convergence, part of Stochastic processes
Laws of large numbers
| Type | Section |
| Taught in | Expectation and convergence |
| Part | Stochastic processes |
Let Y_1,Y_2,\ldots be independent and identically distributed, with \mathbb{E} Y_i=\mu, and write \overline Y_n=n^{-1}\sum_{i=1}^n Y_i.
Theorem 1 (Strong law) If \mathbb{E}|Y_1|<\infty, then \overline Y_n\to\mu almost surely.
This is the strong law stated on Functions of random variables and limit theorems. The proof under only \mathbb{E}|Y_1|<\infty needs more than the fourth-moment argument below. What that page already shows, and what is repeated here because the mode matters, is the weak law under a finite variance. If \operatorname{Var}(Y_i)=\sigma^2<\infty, Chebyshev’s inequality gives
\mathrm{P}(|\overline Y_n-\mu|>\varepsilon)\le\frac{\sigma^2}{n\varepsilon^2}\to 0.
The bound proves convergence in probability. It does not prove almost-sure convergence: \sum_n n^{-1} diverges, so the first Borel–Cantelli lemma cannot be applied to these bounds.
The central limit theorem is a refinement of the same average. It describes the distribution of \sqrt{n}\,(\overline Y_n-\mu), not the expectation of a function of that average, unless a further integrability condition is added.
Used in
See also
Monotone convergence, Fatou, and dominated convergence · Modes of convergence · Strong law · Uniform integrability · Inequalities and limits under the expectation · Four modes of convergence · Where the reverse implications fail · Characteristic functions