3. Consider a Markov chain on Ω = {1,2,3,4,5,6} specified by the following transition
probability matrix.
$$P = \begin{bmatrix}
\frac{1}{3} & 0 & \frac{1}{3} & 0 & 0 & \frac{1}{3} \\
\frac{1}{2} & \frac{1}{4} & \frac{1}{4} & 0 & 0 & 0 \\
0 & 0 & 0 & 0 & 1 & 0 \\
\frac{1}{4} & \frac{1}{4} & \frac{1}{4} & 0 & 0 & \frac{1}{4} \\
0 & 0 & 1 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 & 0 & 1
\end{bmatrix}$$
(a) What are the classes of this Markov chain? Is the Markov chain irreducible?
(b) Which states are transient and which are recurrent?
(c) What is the period of each state of this Markov chain?
(d) Let Xo be the initial state with distribution πο = (1,1,1,0,0,0) corresponding to
the probability of being in states 1, 2, 3, 4, 5, 6 respectively. Let X0, X1, X2,... be
the Markov chain constructed using Pabove. What is E[X1]?
(e) What is Var(X1)?
(f) What is E[X3]?