Question

1. Let {X_n} be Markov chain with state space S and transition matrix P = (P_{i,j}). Define P_i to be the conditional probability that the Markov chain starts from i, that is, P_i(A) = P(A|X_0 = i) for all events A. Define the return time to i be R_i := min{n ? 1 : X_n = i} and let its p.m.f. under P_i be {f_i^{(n)}}_{n ? 0}. That is, we let f_i^{(n)} := P_i(R_i = n) for all n ? 0. Note that f_i^{(0)} = 0 since R_i is at least 1 by definition. (a) What does it mean by saying i is recurrent? What does it mean by saying j is transient? Write down both the intuitive meaning and the mathematical definition. (b) Show that P_i(R_i = 2) = P_{ii}^{(2)} - (P_{ii})^2, where P_{ii}^{(2)} is the 2-step transition probability from i to itself. (c) Compute P_i(R_i = 3) in terms of P.

          1. Let {X_n} be Markov chain with state space S and transition matrix P = (P_{i,j}). Define P_i to be the conditional probability that the Markov chain starts from i, that is, P_i(A) = P(A|X_0 = i) for all events A. Define the return time to i be
R_i := min{n ? 1 : X_n = i}
and let its p.m.f. under P_i be {f_i^{(n)}}_{n ? 0}. That is, we let f_i^{(n)} := P_i(R_i = n) for all n ? 0.
Note that f_i^{(0)} = 0 since R_i is at least 1 by definition.

(a) What does it mean by saying i is recurrent? What does it mean by saying j is transient? Write down both the intuitive meaning and the mathematical definition.
(b) Show that P_i(R_i = 2) = P_{ii}^{(2)} - (P_{ii})^2, where P_{ii}^{(2)} is the 2-step transition probability from i to itself.
(c) Compute P_i(R_i = 3) in terms of P.

1. Let Xn be Markov chain with state space S and transition matrix P = (Pi,j). Define Pi to be the conditional probability that the Markov chain starts from i, that is, Pi(A) = P(A|X0 = i) for all events A. Define the return time to i be
Ri := minn ? 1 : Xn = i
and let its p.m.f. under Pi be fi^(n)n ? 0. That is, we let fi^(n) := Pi(Ri = n) for all n ? 0.
Note that fi^(0) = 0 since Ri is at least 1 by definition.

(a) What does it mean by saying i is recurrent? What does it mean by saying j is transient? Write down both the intuitive meaning and the mathematical definition.
(b) Show that Pi(Ri = 2) = Pii^(2) - (Pii)^2, where Pii^(2) is the 2-step transition probability from i to itself.
(c) Compute Pi(Ri = 3) in terms of P.

Added by Elijah J.

Question

Please give Ace some feedback