Conditional independence

Section of Graphical models, part of Statistical learning

Conditional independence

Type Section
Taught in Graphical models
Part Statistical learning
Questions using it 26

Conditional independence is a statement about the conditional distribution, not about the marginal correlation. If X,Y,Z are random variables with joint distribution P, then X is conditionally independent of Y given Z, written X\perp Y\mid Z, if for any measurable set A in the sample space of X there exists a conditional probability of A given Y and Z that is a function of Z alone. For discrete variables,

X\perp Y\mid Z \iff \mathbb{P}(X=x,Y=y\mid Z=z) =\mathbb{P}(X=x\mid Z=z)\,\mathbb{P}(Y=y\mid Z=z),

and when X,Y,Z have a joint density on a product measure,

\begin{aligned} X\perp Y\mid Z &\iff f_{XY\mid Z}(x,y\mid z)=f_{X\mid Z}(x\mid z)\,f_{Y\mid Z}(y\mid z)\\ &\iff f_{XYZ}(x,y,z)\,f_Z(z)=f_{XZ}(x,z)\,f_{YZ}(y,z), \end{aligned}

almost surely (for all z with f_Z(z)>0 if the densities are continuous). The following properties hold (Lauritzen, 1996).

And X\perp Y\mid Z is equivalent to each of the following, almost surely, with f the density of its arguments:

\begin{aligned} f(x,y,z)&=f(x,z)\,f(y,z)/f(z),\\ f(x\mid y,z)&=f(x\mid z),\\ f(x,z\mid y)&=f(x\mid z)\,f(z\mid y),\\ f(x,y,z)&=g(x,z)\,h(y,z)\quad\text{for some }g,h,\\ f(x,y,z)&=f(x\mid z)\,f(y,z). \end{aligned}

The third line follows from the second, because f(x,z\mid y)=f(x\mid y,z)\,f(z\mid y). The factorisation f(x,y,z)=g(x,z)\,h(y,z) is the form that generalises to graphs. Also, if C separates A from B, then X_A\perp X_B\mid X_C for a graphical model with the global Markov property below.

Used in

See also

A zero precision entry is conditional independence · Categorical and multinomial models · Markov random field · Precision matrix · Prior and posterior predictive distributions · Separating set · Undirected graph · Hammersley–Clifford

Glossary and index