Markov assumption as a chosen state
Section of Discrete-time Markov chains, part of Stochastic processes
Markov assumption as a chosen state
| Type | Section |
| Taught in | Discrete-time Markov chains |
| Part | Stochastic processes |
Suppose tomorrow’s weather depends on today’s weather and also on whether it has already rained for three days. Two states cannot store a run of length three. A state such as (\text{today's weather},\text{current run length}) can. The state space gets larger, and the transition rule becomes local in time again. Compressing the past into a state makes the one-step calculation possible. Compressing too far drops information the next step uses.
Time homogeneity is a separate assumption. A chain can be Markov at every time with a matrix P_n that changes with n, for example a morning matrix and an evening matrix. The law is then
\mu_n=\mu_0 P_0 P_1\cdots P_{n-1},
not a power of one matrix. Solving \pi P=\pi for a single matrix does not describe that process.
Used in
See also
Discrete-time Markov chain · Stationary distribution · Transition matrix · Hitting probabilities · Spectral view for two states · Stationarity and the law from one start · Doubly stochastic matrix