Let X = {Xu : n ∈ Z+} be a time homogeneous Markov chain having the one-step transition matrix P given by
0.5
0.5
(points) Draw the directed graph of the above time homogeneous Markov chain. (points) Identify the class of states and also identify which classes are transient, positive null recurrent. (10 points) If we introduce for every i,j ∈ $, the absorption probability
P(absorption in S(j) | Xu = i)
P(Tsu | Xo = i)
find the value of jisur. (15 points) If we introduce for every i,j ∈ $ the values Tij given by the fraction of time X visits starting
find the values #ij, and obtain the matrix Il = (Tij | fisu)