Return and period
Section of Classes, recurrence, and reversibility, part of Stochastic processes
Return and period
| Type | Section |
| Taught in | Classes, recurrence, and reversibility |
| Part | Stochastic processes |
Let T_i^+=\inf\{n\ge 1: X_n=i\} be the first return time to i.
Definition 1 (Recurrence and period) State i is recurrent when \mathbb{P}_i(T_i^+<\infty)=1, and transient otherwise. A recurrent state is positive recurrent when \mathbb{E}_i T_i^+<\infty, and null recurrent when the return time is almost surely finite and has infinite mean. The period of i is
d(i)=\gcd\{n\ge 1: p_{ii}^{(n)}>0\}.
Period 1 means aperiodic.
Communicating states share a period. In a finite chain, every state in a closed irreducible class is positive recurrent, and every state that lies in no closed class is transient. A self-loop forces period 1 in its class. Aperiodicity does not require a self-loop at every state: two return lengths with gcd 1 are enough. The stepping-stone chain on Markov chains is aperiodic because it can return to stone 1 in both 2 and 3 steps.
Theorem 1 (Finite irreducible chains) A finite irreducible chain has a unique stationary distribution, and every entry of that distribution is strictly positive. If the chain is also aperiodic, then p_{ij}^{(n)}\to\pi_j for every i and j.
Aperiodicity belongs to the limit, not to the existence of \pi. Deterministic alternation between two states has \pi=(1/2,1/2) and P^n oscillating. The same split is the content of the limiting-distribution theorem on Markov chains: irreducibility and aperiodicity on a finite set are the hypotheses that turn \pi=\pi P into \lim_n (P^n)_{ij}=\pi_j.
Used in
See also
Recurrence and period · Finite irreducible chains · Detailed balance · Communicating classes · Global balance and detailed balance · Two families with an explicit stationary law · When a stationary law fails to be unique · Period and time averages