00:01
To begin here, i need to note that by numerate policies, i'm only supposed to answer one question at a time, with exceptions for things like multi -part problems.
00:12
But because of that, i'm going to be focusing on question five.
00:15
If question six is what you need help with, then i would suggest you re -upload the question focusing on problem six.
00:23
That being said, we have our markov chain.
00:26
We have our state space, one, two, three, four.
00:28
So we begin with our state one, where we know that the probability of transitioning from state one to state two is probability one.
00:40
If we are in state one, we will end up in state two when we transition.
00:45
If we're in state two, then we transition to state three with a probability of one -third, and we transition to state four with a probability of two -thirds.
01:05
If we're in state three, we transition to state one with a probability of one.
01:16
And if we are in state four, transition to state two with probability one over two.
01:25
And we transition to state three with probability one over two.
01:38
Now, for a recurrent state, we effectively have that we will return.
01:48
To that state infinitely many times.
02:08
Additionally, we need to note that if i is recurrent and j is in a communicating class with i, then j is recurrent as well.
02:29
Which means for figuring out which states are recurrent, it's ultimately going to be figuring out which of the states are in communicating classes...