The limit of the three-state chain

Topic: Markov chains · more questions on this topic
‹ Two steps of a three-state chain Any two-state chain ›

A chain on \{1,2,3\} has transition matrix

P=\begin{pmatrix}0.5&0.5&0\\0.25&0.5&0.25\\0&0.5&0.5\end{pmatrix}.

Show that it is irreducible and aperiodic, and find its limiting distribution.

Worked answer (click to show)

State 2 can be reached from 1 and 3 and can reach both, so every pair of states communicates and the chain is irreducible. P_{11}=0.5>0, so state 1 can return in one step and has period 1; in an irreducible chain all states then have period 1.

The balance equations are \pi_1=0.5\pi_1+0.25\pi_2 and \pi_3=0.25\pi_2+0.5\pi_3, so \pi_1=\pi_3=\pi_2/2. With \pi_1+\pi_2+\pi_3=1,

\boldsymbol\pi=(1/4,\ 1/2,\ 1/4).