Two-state continuous-time chain
Section of Continuous-time Markov chains, part of Stochastic processes
Two-state continuous-time chain
| Type | Section |
| Taught in | Continuous-time Markov chains |
| Part | Stochastic processes |
Suppose the chain moves from 0 to 1 at rate \alpha>0 and from 1 to 0 at rate \beta>0. Then
Q=\begin{pmatrix}-\alpha & \alpha\\ \beta & -\beta\end{pmatrix}, \qquad \pi=\Bigl(\frac{\beta}{\alpha+\beta},\frac{\alpha}{\alpha+\beta}\Bigr).
Let u(t)=\mathbb{P}\bigl(X(t)=1\mid X(0)=0\bigr). In a short interval the chain in 0 enters 1 at rate \alpha, and the chain in 1 leaves at rate \beta, so
u'(t)=\alpha\bigl(1-u(t)\bigr)-\beta u(t)=\alpha-(\alpha+\beta)u(t),
with u(0)=0. The equilibrium of the ODE is \alpha/(\alpha+\beta). The deviation from that equilibrium decays as e^{-(\alpha+\beta)t}:
u(t)=\frac{\alpha}{\alpha+\beta}\bigl(1-e^{-(\alpha+\beta)t}\bigr).
Differentiating the formula recovers the ODE, and the value at 0 is 0. The stationary probability of state 1 is the limit as t\to\infty.
This finite irreducible chain converges even though its jump chain alternates between the two states. The jump chain is the periodic discrete chain with a=b=1 on Discrete-time Markov chains. Random holding times destroy the parity: the chain does not switch at every integer, so the continuous-time law cannot oscillate forever between the two states.
Used in
See also
Generator · Kolmogorov forward and backward equations · Uniformisation · Jump chain · Holding time