Problem 1 A Markov chain Xn, n ge 0 with states 0, 1, 2,...

Problem 1 A Markov chain {X_n, n ge 0} with states 0, 1, 2, (i.e., $X_n = {0, 1, 2}$) has the transition probability matrix $egin{bmatrix} frac{1}{2} & frac{1}{3} & frac{1}{6} \ 0 & frac{1}{3} & frac{2}{3} \ frac{1}{2} & 0 & frac{1}{2} end{bmatrix}$ If $P{X_0 = 0} = P{X_0 = 1} = frac{1}{4}$, find $E[X_3]$.

Transcript

00:01 We are given the transition matrix for a markov chain with three states, and we are looking for the expected value of the third state.

00:13 So from the conditional probability, we know that we can use conditional expectation for this.

00:20 Specifically, we can say that this is the expected value of the expected value conditional of x3, given the initial state x0.

00:31 So the theorem we're using for that is the property of expectation that the expectation of some random variable, we can condition it on anything, any other random variable y, and take the nested expectation.

00:49 So let's calculate a couple of these expectations here.

00:54 So we can start with the expected value of x3 given that x not equals zero, the initial state being.

01:04 Zero and we'll do that for 0 1 and 2.

01:07 So how do we calculate an expectation? well, since we're discrete, our expectation is going to be a sum, and this sum is going to be over all the states.

01:16 So i goes from 0 to 2 of i, since we want an expected value, times the probability of that outcome.

01:25 So x3 being the state i, given that x not equals 0.

01:32 So for this, this probability is going to come from the 0 comma ith element of the third order transition matrix.

01:46 So i suppose that's something we should find first.

01:49 So we can do that over here.

01:51 P squared, if we multiply out our transition matrix by itself, comes out to one -third, one -third, one -third, one -half, 518th, 1 9th, 1 6th, 718th, 5 9th, and 1 3rd.

02:13 So that's the 2 -state transition matrix, and then p cubed.

02:16 We could get from multiplying p squared again by p, and this is the one we're looking for.

02:21 13 over 36, 4 over 9, 5 .12th, let's see, 11 .54ths, 4 over 27, 2 9th, 2 9ths...

Question

Please give Ace some feedback