Modelling checklist for a stochastic process
Section of Stochastic processes, part of Stochastic processes
Modelling checklist for a stochastic process
| Type | Section |
| Taught in | Stochastic processes |
| Part | Stochastic processes |
Consider a help desk. Let N(t) count calls received by time t, and let Q(t) count calls that are waiting or being served at time t. The first path is non-decreasing. The second rises when a call arrives and falls when a call finishes. A Poisson model for N does not make Q Poisson. A model for Q also needs the service durations and a rule for what happens when calls overlap.
The state may be a number or a vector. If service times are exponential and there is one server, the current count can be a Markov state, because an exponential clock has no memory of how long it has already run. That memoryless property is on Random variables and distributions. If instead every service lasts exactly ten minutes, knowing that one call is in service does not say whether it departs in one second or in ten minutes. The state has to be enlarged by the elapsed service time, or by the remaining service time. The Markov property is a statement about a chosen state, not a property of the physical system on its own. The finite-state case of that property is on Markov chains. The hitting calculations, the class structure, and the continuous-time generator are developed later in this section.
Stationary increments and independent increments come apart in the same way. A random walk with independent identically distributed steps has both. A Poisson process whose rate changes with calendar time keeps independent increments over disjoint intervals, and loses stationary increments, because an interval at a busy hour is not distributed as an interval of the same length at a quiet hour. The repeated bit X_n=B has increments equal to zero after time 0, so the increments are simple, while the levels are perfectly dependent. Naming the property selects the theorem.
Example 1 (A walk after one hundred steps) Suppose the daily increments Y_i are independent, with \mathbb{P}(Y_i=1)=0.6 and \mathbb{P}(Y_i=-1)=0.4, and S_0=0.
Question. Find the mean and standard deviation of S_{100}. Say what that calculation does not answer.
Solution. Each step has mean \mathbb{E} Y_i=0.6-0.4=0.2 and second moment \mathbb{E} Y_i^2=1, so
\operatorname{Var}(Y_i)=1-(0.2)^2=0.96.
Independence gives \mathbb{E} S_{100}=100\times 0.2=20 and \operatorname{Var}(S_{100})=100\times 0.96=96, so the standard deviation is \sqrt{96}\approx 9.80. A normal approximation, from the central limit theorem, describes the position at the fixed time n=100. It does not say whether the path S_0,S_1,\ldots,S_{100} ever fell below -5. That is a question about the whole path. First-passage calculations for a walk are on Random walks and first passage.
Used in
See also
Stochastic process · Stationary and independent increments · What a stochastic process models · State, clock, and assumptions · Toolkit for stochastic processes · What a model can identify · Markov’s inequality · Chebyshev’s inequality