3. On any given day, Buffy is either cheerful (0), so-so (1), or gloomy (2). If she is cheerful today, then
she will be so-so tomorrow with probability 1. If she is so-so today, then she will be cheerful, so-so,
or gloomy tomorrow with respective probabilities 0.1, 0.5, 0.4. If she is gloomy today, then Buffy
will be so-so tomorrow with probability 1. The successive emotional status of Buffy constitutes a
Markov chain.
(a) Write down the transition probability matrix.
[2 points]
(b) Does there exist an integer $r \ge 1$ such that $P(X_r = j|X_0 = i) > 0$ for all $i, j = 0, 1, 2$?
Justify your answer.
[2 points]
(c) Do the limiting probabilities $\lim_{n \to \infty} P(X_n = j|X_0 = i)$, for $i, j = 0, 1, 2$ exist? If so find the
limits. Justify your answer.
[3 points]
