Stationarity and detailed balance
Definition taught in Markov chains, Gibbs sampling, and Metropolis–Hastings, part of Bayesian statistics
Stationarity and detailed balance
| Type | Definition |
| Taught in | Markov chains, Gibbs sampling, and Metropolis–Hastings |
| Part | Bayesian statistics |
| Properties | 1 |
| Worked examples | 2 |
| Questions using it | 3 |
A distribution \pi is stationary if a state distributed as \pi remains distributed as \pi after one transition. The global balance equation is \pi(y)=\sum_x\pi(x)q(y\mid x) \quad\text{or}\quad \pi(y)=\int\pi(x)q(y\mid x)\,\mathrm{d} x. A stronger sufficient condition is detailed balance: \pi(x)q(y\mid x)=\pi(y)q(x\mid y) \quad\text{for all }x,y.
Properties
Detailed balance implies stationarity
Main article: Detailed balance implies stationarity
If a transition kernel q satisfies Equation (3) in Markov chains, Gibbs sampling, and Metropolis–Hastings for a probability distribution \pi, then \pi satisfies global balance and is a stationary distribution of the chain.
Examples
Four-state random walk
Worked example from Markov chains, Gibbs sampling, and Metropolis–Hastings
Question. Consider states 1,2,3,4 arranged in a line. At every state the chain stays put with probability 1/2. From an endpoint it moves to the only neighbour with probability 1/2; from an interior state it moves left or right with probability 1/4 each. With columns indexing the current state and rows the next state, construct the transition matrix and find the stationary distribution.
Solution. The transition matrix is
{\color{blue}\boldsymbol{Q}}= \begin{pmatrix} 1/2&1/4&0&0\\ 1/2&1/2&1/4&0\\ 0&1/4&1/2&1/2\\ 0&0&1/4&1/2 \end{pmatrix}.
Detailed balance between states 1 and 2 requires
\pi_1\frac12=\pi_2\frac14 \quad\Longrightarrow\quad \pi_2=2\pi_1.
Similarly,
\pi_2\frac14=\pi_3\frac14 \Longrightarrow\pi_3=\pi_2, \qquad \pi_3\frac14=\pi_4\frac12 \Longrightarrow\pi_3=2\pi_4.
Hence
{\boldsymbol{\pi}}=(\pi_1,\pi_2,\pi_3,\pi_4)^{\mathsf T} \propto(1,2,2,1)^{\mathsf T}.
Normalising by 1+2+2+1=6 gives
\boxed{{\boldsymbol{\pi}}=(1/6,2/6,2/6,1/6)^{\mathsf T}}.
C12. Markov property and invariance
Question page: C12. Markov property and invariance
What is the Markov property? What does it mean for a distribution to be invariant for a chain?
Definitions/notation. The Markov property says that the conditional distribution of the next state, given the whole past, depends only on the current state. A distribution \pi is invariant if one transition preserves it: X^{(t)}\sim\pi implies X^{(t+1)}\sim\pi; this alone does not establish convergence from every initial state.
Used in
- Taught in Markov chains, Gibbs sampling, and Metropolis–Hastings, where the derivation and examples are.
- Also used in Discrete-time Markov chains · Classes, recurrence, and reversibility · Continuous-time Markov chains
- Questions that use it (3) — A chain that never settles (answer) · Detailed balance · Question 3 (answer)
- All 31 questions on Markov chains, Gibbs sampling, and Metropolis–Hastings
See also
Detailed balance · Detailed balance implies stationarity · Metropolis–Hastings invariance · Stationary distribution · Markov chain · Systematic-scan Gibbs sampler · Warm-up is not a convergence proof · Metropolis–Hastings update