Two steps of a three-state chain
A chain on \{1,2,3\} has transition matrix
P=\begin{pmatrix}0.5&0.5&0\\0.25&0.5&0.25\\0&0.5&0.5\end{pmatrix}.
Starting from state 1, find the distribution of X_2, and the probability that X_1=2 and X_2=3.
Worked answer (click to show)
The distribution of X_2 is the first row of P^2:
(0.5\times0.5+0.5\times0.25,\ 0.5\times0.5+0.5\times0.5,\ 0.5\times0.25)=(0.375,\ 0.5,\ 0.125).
The path 1\to2\to3 has probability P_{12}P_{23}=0.5\times0.25=0.125. It is the only two-step path from 1 to 3, which is why it equals \mathbb{P}(X_2=3).