Discrete-time Markov chain
Definition taught in Discrete-time Markov chains, part of Stochastic processes
Discrete-time Markov chain
| Type | Definition |
| Taught in | Discrete-time Markov chains |
| Part | Stochastic processes |
A process X_0,X_1,\ldots on a finite or countable state space S is a discrete-time Markov chain when
\mathbb{P}(X_{n+1}=j\mid X_n=i,X_{n-1},\ldots,X_0)=\mathbb{P}(X_{n+1}=j\mid X_n=i)=p_{ij}
for every n and every choice of states. The chain is time-homogeneous when p_{ij} does not depend on n. The transition matrix P=(p_{ij}) has non-negative entries, and every row sums to 1.
Used in
- Taught in Discrete-time Markov chains, where the derivation and examples are.
- All 3 questions on Discrete-time Markov chains