Uniformisation
Section of Continuous-time Markov chains, part of Stochastic processes
Uniformisation
| Type | Section |
| Taught in | Continuous-time Markov chains |
| Part | Stochastic processes |
Choose \gamma\ge\max_i\nu_i and set R=I+Q/\gamma. Then R is a transition matrix. Its off-diagonal entries q_{ij}/\gamma are non-negative, its diagonal entries are 1-\nu_i/\gamma\ge 0, and its rows sum to 1 because the rows of Q sum to 0. Since Q=\gamma(R-I),
e^{tQ}=e^{-\gamma t}\sum_{n=0}^{\infty}\frac{(\gamma t)^n}{n!} R^n.
A finite continuous-time transition law is a Poisson mixture of the discrete-time powers of R. Some steps of R are self-transitions inserted so that every state has the same potential exit rate \gamma. Those self-transitions are not the jumps of the embedded chain K. The matrix K is used when a jump actually occurs. The matrix R is used to write e^{tQ} as an average of discrete chains, which is a practical way to compute P(t) and a precise link between Poisson clocks and continuous-time chains.
The generator Q stores jump rates, with q_{ii}=-\nu_i so that each row sums to 0. The chain waits an \mathrm{Exp}(\nu_i) time and then jumps with probabilities q_{ij}/\nu_i. On a finite irreducible state space, P(t)=e^{tQ} converges to the unique solution of \pi Q=0.
Used in
See also
Generator · Kolmogorov forward and backward equations · Two-state continuous-time chain · Jump chain · Holding time