Question

Example 3.17 Consider a Markov chain with transition matrix a b c a 0 1 0 P = b 1/2 0 1/2 c 1/3 1/3 1/3 From state a, find the expected return time E(Ta|X0 = a) using first-step analysis. Solution Let ex = E(Ta|X0 = x), for x = a, b, c. Thus, ea is the desired expected return time, and eb and ec are the expected first passage times to a for the chain started in b and c, respectively. For the chain started in a, the next state is b, with probability 1. From b, the further evolution of the chain behaves as if the original chain started at b. Thus, ea = 1 + eb. From b, the chain either hits a, with probability 1/2, or moves to c, where the chain behaves as if the original chain started at c. It follows that eb = 1/2 + 1/2(1 + ec). Similarly, from c, we have ec = 1/3 + 1/3(1 + eb) + 1/3(1 + ec). Solving the three equations gives ec = 8/3, eb = 7/3, and ea = 10/3. The desired expected return time is 10/3.

          Example 3.17 Consider a Markov chain with transition matrix
a b c
a 0 1 0
P = b 1/2 0 1/2
c 1/3 1/3 1/3
From state a, find the expected return time E(Ta|X0 = a) using first-step analysis.

Solution Let ex = E(Ta|X0 = x), for x = a, b, c. Thus, ea is the desired expected return time, and eb and ec are the expected first passage times to a for the chain started in b and c, respectively.
For the chain started in a, the next state is b, with probability 1. From b, the further evolution of the chain behaves as if the original chain started at b. Thus,
ea = 1 + eb.
From b, the chain either hits a, with probability 1/2, or moves to c, where the chain behaves as if the original chain started at c. It follows that
eb = 1/2 + 1/2(1 + ec).
Similarly, from c, we have
ec = 1/3 + 1/3(1 + eb) + 1/3(1 + ec).
Solving the three equations gives
ec = 8/3, eb = 7/3, and ea = 10/3.
The desired expected return time is 10/3.

Example 3.17 Consider a Markov chain with transition matrix
a b c
a 0 1 0
P = b 1/2 0 1/2
c 1/3 1/3 1/3
From state a, find the expected return time E(Ta|X0 = a) using first-step analysis.

Solution Let ex = E(Ta|X0 = x), for x = a, b, c. Thus, ea is the desired expected return time, and eb and ec are the expected first passage times to a for the chain started in b and c, respectively.
For the chain started in a, the next state is b, with probability 1. From b, the further evolution of the chain behaves as if the original chain started at b. Thus,
ea = 1 + eb.
From b, the chain either hits a, with probability 1/2, or moves to c, where the chain behaves as if the original chain started at c. It follows that
eb = 1/2 + 1/2(1 + ec).
Similarly, from c, we have
ec = 1/3 + 1/3(1 + eb) + 1/3(1 + ec).
Solving the three equations gives
ec = 8/3, eb = 7/3, and ea = 10/3.
The desired expected return time is 10/3.

Added by Nuria G.

Question

Please give Ace some feedback