Independent random variables

Definition taught in Joint distributions, part of Probability

Independent random variables

Type Definition
Taught in Joint distributions
Part Probability
Worked examples 2

Discrete random variables X_1,\ldots,X_n are independent when the joint mass function factors for every point:

\mathbb{P}(X_1=x_1,\ldots,X_n=x_n)=\prod_{k=1}^n \mathbb{P}(X_k=x_k).

Continuous random variables with joint density f are independent when f(x_1,\ldots,x_n)=\prod_{k=1}^n f_{X_k}(x_k). An infinite sequence is independent when every finite subcollection is.

Examples

A box of three dice

Worked example from Joint distributions

A box holds three dice. Die 1 is ordinary. Die 2 has no six but two fives. Die 3 has no five but two sixes. A die is chosen at random and thrown. Let X be the number of the die and Y the face shown.

Question. Find the joint mass function and both marginals. Are X and Y independent?

Solution. Each die is chosen with probability 1/3, and each of its six faces then shows with probability 1/6, so each face position has probability 1/18. A value printed on two faces gets 2/18=1/9.

y=1 y=2 y=3 y=4 y=5 y=6 f_X(x)
x=1 1/18 1/18 1/18 1/18 1/18 1/18 1/3
x=2 1/18 1/18 1/18 1/18 1/9 0 1/3
x=3 1/18 1/18 1/18 1/18 0 1/9 1/3
f_Y(y) 1/6 1/6 1/6 1/6 1/6 1/6 1

The marginals are the row and column sums. Remarkably, Y is uniform on \{1,\ldots,6\}, exactly as for a single fair die. Yet X and Y are not independent: \mathbb{P}(X=2,Y=6)=0, while f_X(2)f_Y(6)=1/18. Seeing a six rules out die 2. The marginals alone cannot reveal the dependence; only the joint table can.

Uncorrelated but dependent

Worked example from Joint distributions

Let X be uniform on \{-1,0,1\} and Y=X^2.

Question. Find \operatorname{Cov}(X,Y). Are X and Y independent?

Solution. \mathbb{E} X=0 and XY=X^3=X, so \operatorname{Cov}(X,Y)=\mathbb{E}(XY)-\mathbb{E} X\,\mathbb{E} Y=\mathbb{E} X-0=0. The variables are uncorrelated. They are far from independent: Y is a function of X, and \mathbb{P}(X=0,Y=1)=0 while \mathbb{P}(X=0)\mathbb{P}(Y=1)=\tfrac13\cdot\tfrac23. Covariance measures only linear dependence, and this dependence is symmetric, so it cancels.

Used in

See also

Random variable · Joint probability mass function · Joint probability density · Covariance · Expectation vector and covariance matrix · Conditional distribution · Tower property

Glossary and index