C12. Markov property and invariance

Topic: Markov chains, Gibbs sampling, and Metropolis–Hastings · more questions on this topic
‹ C11 C13 ›

What is the Markov property? What does it mean for a distribution to be invariant for a chain?

Worked answer (click to show)

Definitions/notation. The Markov property says that the conditional distribution of the next state, given the whole past, depends only on the current state. A distribution \pi is invariant if one transition preserves it: X^{(t)}\sim\pi implies X^{(t+1)}\sim\pi; this alone does not establish convergence from every initial state.