Four-state random walk
Section of Markov chains, Gibbs sampling, and Metropolis–Hastings, part of Bayesian statistics
Four-state random walk
| Type | Section |
| Taught in | Markov chains, Gibbs sampling, and Metropolis–Hastings |
| Part | Bayesian statistics |
Example 1 (Four-state random walk) Question. Consider states 1,2,3,4 arranged in a line. At every state the chain stays put with probability 1/2. From an endpoint it moves to the only neighbour with probability 1/2; from an interior state it moves left or right with probability 1/4 each. With columns indexing the current state and rows the next state, construct the transition matrix and find the stationary distribution.
Solution. The transition matrix is
{\color{blue}\boldsymbol{Q}}= \begin{pmatrix} 1/2&1/4&0&0\\ 1/2&1/2&1/4&0\\ 0&1/4&1/2&1/2\\ 0&0&1/4&1/2 \end{pmatrix}.
Detailed balance between states 1 and 2 requires
\pi_1\frac12=\pi_2\frac14 \quad\Longrightarrow\quad \pi_2=2\pi_1.
Similarly,
\pi_2\frac14=\pi_3\frac14 \Longrightarrow\pi_3=\pi_2, \qquad \pi_3\frac14=\pi_4\frac12 \Longrightarrow\pi_3=2\pi_4.
Hence
{\boldsymbol{\pi}}=(\pi_1,\pi_2,\pi_3,\pi_4)^{\mathsf T} \propto(1,2,2,1)^{\mathsf T}.
Normalising by 1+2+2+1=6 gives
\boxed{{\boldsymbol{\pi}}=(1/6,2/6,2/6,1/6)^{\mathsf T}}.
Used in
See also
Random walk with drift · Drift and scale of a random walk · Gambler’s ruin · Markov chain · Stationarity and detailed balance · Detailed balance implies stationarity · Systematic-scan Gibbs sampler · Warm-up is not a convergence proof