Question

1. Consider the following discrete-time Markov chain with $m + 1$ states and transition probability matrix $P = (p_{ij})$: for $j \in \{1, 2, \dots, m\}$, $p_{ij} = \begin{cases} p & (i, j) \in \{ (1, 2), (2, 3), \dots, (m - 1, m), (m, 1) \} \ p & (i, j) \in \{ (1, m), (2, 1), (3, 2), \dots, (m, m - 1) \} \ \frac{1}{m} & (i, j) \in \{ (m + 1, 1), (m + 1, 2), \dots, (m + 1, m - 1), (m + 1, m) \} \ 0 & \text{otherwise} \end{cases}$ Obtain a full classification of the states in this Markov chain (classes, recurrence, transience, periodicity) and when appropriate obtain the steady-state and/or stationary probabilities.

          1. Consider the following discrete-time Markov chain with $m + 1$ states and transition probability matrix $P = (p_{ij})$: for $j \in \{1, 2, \dots, m\}$,

$p_{ij} = \begin{cases} p & (i, j) \in \{ (1, 2), (2, 3), \dots, (m - 1, m), (m, 1) \} \
p & (i, j) \in \{ (1, m), (2, 1), (3, 2), \dots, (m, m - 1) \} \
\frac{1}{m} & (i, j) \in \{ (m + 1, 1), (m + 1, 2), \dots, (m + 1, m - 1), (m + 1, m) \} \
0 & \text{otherwise} \end{cases}$

Obtain a full classification of the states in this Markov chain (classes, recurrence, transience, periodicity) and when appropriate obtain the steady-state and/or stationary probabilities.

1. Consider the following discrete-time Markov chain with m + 1 states and transition probability matrix P = (pij): for j ∈{1, 2, …, m},

pij = p (i, j) ∈{ (1, 2), (2, 3), …, (m - 1, m), (m, 1) }p (i, j) ∈{ (1, m), (2, 1), (3, 2), …, (m, m - 1) }(1)/(m) (i, j) ∈{ (m + 1, 1), (m + 1, 2), …, (m + 1, m - 1), (m + 1, m) }0 otherwise

Obtain a full classification of the states in this Markov chain (classes, recurrence, transience, periodicity) and when appropriate obtain the steady-state and/or stationary probabilities.

Added by Mark B.

Question

Please give Ace some feedback