Law of large numbers

Theorem taught in Functions of random variables and limit theorems, part of Probability

Law of large numbers

Twenty simulated runs of 100 tosses of a fair coin. From Random experiments and probability.
Type Theorem
Taught in Functions of random variables and limit theorems
Part Probability
Worked examples 1
Questions using it 6

If the mean \mu exists, the strong law says

\mathbb{P}\Bigl(\lim_{n\to\infty} S_n/n=\mu\Bigr)=1.

If in addition the variance is finite, the weak law says that for every \varepsilon>0,

\lim_{n\to\infty}\mathbb{P}\bigl(|S_n/n-\mu|>\varepsilon\bigr)=0.

The weak law under a finite variance follows from Chebyshev’s inequality: \mathbb{P}(|S_n/n-\mu|>\varepsilon)\le \operatorname{Var}(S_n/n)/\varepsilon^2=\operatorname{Var}(X_1)/(n\varepsilon^2), which tends to zero.

Examples

The mean of repeated voltage readings

Question page: The mean of repeated voltage readings

Voltage readings X_1,\ldots,X_n are independent and \operatorname{N}(\mu,\sigma^2), with \mu and \sigma^2 unknown. Let \overline X=(X_1+\cdots+X_n)/n, and suppose x_1,\ldots,x_n are the observed values.

  1. What function of the observations estimates \mu?

  2. Show that \mathbb{E}\overline X=\mu.

  3. Show that \operatorname{Var}(\overline X)\to 0 as n\to\infty.

  4. State the distribution of \overline X.

  5. What do (b) and (c) say about the estimate in (a)?

  1. Use the sample average \overline x=(x_1+\cdots+x_n)/n.

  2. \mathbb{E}\overline X=n^{-1}(\mathbb{E} X_1+\cdots+\mathbb{E} X_n)=\mu.

  3. Independence gives \operatorname{Var}(\overline X)=\sigma^2/n, which tends to 0.

  4. An affine image of independent normals is normal, so \overline X\sim\operatorname{N}(\mu,\sigma^2/n).

  5. The estimate is unbiased at every n, and its variance disappears as more readings are taken, so \overline x concentrates on \mu.

Used in

See also

Transformation rule · Multivariate normal distribution · Affine images of normal vectors · Normal linear model · Central limit theorem

Glossary and index