Simulating a Markov chain
Section of Markov chains, part of Probability
Simulating a Markov chain
| Type | Section |
| Taught in | Markov chains |
| Part | Probability |
One step of a finite-state chain is a discrete draw from the row P_{i,\cdot}, which is the discrete inverse-transform method applied to that row. A path is the iteration: given X_n=i, draw X_{n+1} from row i, independently of the earlier past.
A random walk on the integers is even simpler. Draw independent steps \xi_k\in\{-1,+1\} with \mathbb{P}(\xi_k=+1)=p, and set X_n=X_0+\xi_1+\cdots+\xi_n. The partial-sum representation makes the binomial count of rightward steps obvious.
Used in
See also
Finite-state Markov chain · Limiting distribution · Markov property · Chapman–Kolmogorov equations