8 - STOCHASTIC PROCESSES The word stochastic is derived from the Greek OTOXOOTIKOS, meaning 'to aim at a target'. Stochastic processes involve state which changes in a random way. A Markov process is a particular kind of stochastic process. Using discrete time the state of the process at time n + 1 depends only on its state at time n. The classic example of a stochastic process is the random walk. . . Random Walk The simplest form of the random walk problem imagines a line marked out in unit steps or paces from some origin: + -3 -2 -1 0 1 2 + 3 A person or other object starts at the origin and then makes a sequence of steps, some to the right and some to the left, at random. It is reasonable to think of a sequence of turns. At each turn a weighted coin is tossed and if it lands heads one step is taken to the right and if it lands tails one step is taken to the left. In the analysis below assume: · Probability of a left-step (tails) is q · Probability of a right-step (heads) is p where p + q = 1 Consider a walk which consists of a total of n steps or turns. Let X be a random variable whose value, r, is the number of those n steps which are to the right. Given a total of n steps, each of which has a probability p of being a right-step, the probability of there being r right-steps is given by the Binomial distribution: P(X= r) = n r p (8.1) Usually one is interested in the net displacement. Call this k measured in net steps to the right of the origin. Clearly: Net displacement to the right = Total right-steps - Total left-steps Since the total number of left-steps is n - r the net displacement k can be expressed algebraically: k = r - (n-r) = 2r - n hence r = 1(n + k) Rewriting (8.1): P(X = {(n + k)) = ( n ) p?(n+k)q1 2(n-k) (n + k) (8.2) An incomplete interpretation is the following: for fixed n, this is the probability that the number of right-steps is such as to give a net displacement of k steps to the right. - 8.1 -
Interpretation of the Probability Since X is the number of right-steps, its value must be an integer. Therefore the probability P(X = {(n + k) requires the term (n + k) to be an integer. Accordingly n + k (and hence n - k) are even in the expression for the probability: P(X = {(n + k)) = ( n 1 (n + k ) - pz(n+k) q 2(n-k) Further, -n 6 k 6 n. This accords with common sense. If the total number of steps is 2 the net displacement must be one of the three possibilities: two steps to the left, back to the start, or two steps