Answer questions based on the following transition matrix
P = [1/3 2/3 0 0; 1/2 1/2 0 0; 1/4 1/2 0 1/4; 1/3 1/3 1/3 0]
a) Specify the classes of the following Markov chains (system states are numbered 1, 2, 3, 4), determine whether they are transient or recurrent.
b) What is the limiting probability of each state?
c) Suppose the process starts from state 4 at time zero, what is the expected time that the process is absorbed in a class of recurrent states?
d) Suppose the process starts from state 4 at time zero, what is the probability that it reaches state 1 earlier than state 2?