Covariance

Definition taught in Joint distributions, part of Probability

Covariance

Two hundred draws from standard bivariate normal laws with correlation 0, 0.4, 0.7 and -0.9. From Joint distributions.

Type	Definition
Taught in	Joint distributions
Part	Probability
Properties	1
Worked examples	2
Questions using it	1

The covariance of X and Y is \operatorname{Cov}(X,Y)=\mathbb{E}\bigl((X-\mathbb{E} X)(Y-\mathbb{E} Y)\bigr), when the expectation exists. Equivalently, \operatorname{Cov}(X,Y)=\mathbb{E}(XY)-(\mathbb{E} X)(\mathbb{E} Y).

Properties

Affine images of normal vectors

Main article: Affine images of normal vectors

If X_i\sim\operatorname{N}(\mu_i,\sigma_i^2) are independent and a,b_1,\ldots,b_n are constants, then

a+\sum_{i=1}^n b_i X_i \sim \operatorname{N}\Bigl(a+\sum_{i=1}^n b_i\mu_i,\;\sum_{i=1}^n b_i^2\sigma_i^2\Bigr).

The same conclusion holds for a jointly normal vector and any affine image, with the output variance \mathbf{b}^\top\Sigma\mathbf{b}.

Examples

Uncorrelated but dependent

Worked example from Joint distributions

Let X be uniform on \{-1,0,1\} and Y=X^2.

Question. Find \operatorname{Cov}(X,Y). Are X and Y independent?

Solution. \mathbb{E} X=0 and XY=X^3=X, so \operatorname{Cov}(X,Y)=\mathbb{E}(XY)-\mathbb{E} X\,\mathbb{E} Y=\mathbb{E} X-0=0. The variables are uncorrelated. They are far from independent: Y is a function of X, and \mathbb{P}(X=0,Y=1)=0 while \mathbb{P}(X=0)\mathbb{P}(Y=1)=\tfrac13\cdot\tfrac23. Covariance measures only linear dependence, and this dependence is symmetric, so it cancels.

Signs and their sum

Question page: Signs and their sum

Let U and V be independent, with \mathbb{P}(U=1)=\mathbb{P}(V=1)=1/4 and \mathbb{P}(U=-1)=\mathbb{P}(V=-1)=3/4. Set X=U/V and Y=U+V.

1. Give the joint probability mass function of (X,Y).

2. Calculate \operatorname{Cov}(X,Y).

Worked answer (click to show)

1. Since V=\pm1, 1/V=V and X=UV. The four sign pairs give

(U,V)	Probability	X	Y
(1,1)	1/16	1	2
(1,-1)	3/16	-1	0
(-1,1)	3/16	-1	0
(-1,-1)	9/16	1	-2

so \mathbb{P}(X=1,Y=2)=1/16, \mathbb{P}(X=-1,Y=0)=6/16 and \mathbb{P}(X=1,Y=-2)=9/16.

2. \mathbb{E} X=10/16-6/16=1/4, \mathbb{E} Y=\mathbb{E} U+\mathbb{E} V=2(1/4-3/4)=-1 and \mathbb{E}(XY)=2\times1/16-2\times9/16=-1. So

\operatorname{Cov}(X,Y)=-1-\tfrac14\times(-1)=-\tfrac34.

Used in

• Taught in Joint distributions, where the derivation and examples are.
• Also used in Functions of random variables and limit theorems
• Questions that use it (1) — A product density (answer)
• All 8 questions on Joint distributions

See also

Expectation · Expectation vector and covariance matrix · Multivariate normal distribution · Joint probability mass function · Independent random variables · Joint probability density · Conditional distribution · Tower property

Glossary and index