00:01
Okay, i'm going to do a markov chain.
00:03
Markov chain describes a process where the probability of the future depends only on the present.
00:10
We don't look at past states to determine the probability markov chains are used all over the place and used for all sorts of things.
00:17
The example we're going to look at is the learning algorithm for artificial intelligence, and i'm going to do some analysis of the probability in that change.
00:27
The first step to move from the diagram into the analysis is to create the matrix.
00:33
So this is the probability matrix what i've done here.
00:36
And i put it in dash line so i can keep my rows and columns straight, so i don't accidentally read the wrong data.
00:42
But what i've done here is to take st one and put it in this first row.
00:47
There is if you look at the diagram, there's a 0.45 probability that state one will move back in on itself and go to st one again.
00:55
There's a 10.45 probability that it will move to st to there are no arrows that go to three or four or five or seven or nine.
01:03
But there is a point, oh, three probability that it goes to six and 60.7 that it goes to eight.
01:09
And so i worked my way across state, one state to state three.
01:13
It's very important to keep that orientation.
01:15
So in state two, there's only an arrow that goes to three and has a one probability state.
01:21
Three only has an arrow going to state for, and it's a one probability note that these numbers should add up to one over here because there is a one probability that something will happen.
01:33
Even if it stays where it is, something will happen, so these numbers ought to add up to one.
01:38
If they don't, that's good.
01:39
Check that you've mis entered something.
01:42
Another thing to notice is down here.
01:44
There's an identity matrix that the eighth state has a one probability of going to the eighth state, and the ninth state has a one probability of going to the ninth state and over on over on the diagram that looks like this, and there's a little one next to it.
02:02
Those are absorbing states.
02:04
That's where the process will eventually end up.
02:06
So you give it enough time and the process ends up in these absorbing states.
02:10
And once they get there, they stay there to make the math a little easier for us to handle, a little more useful than we will say that it loops back on itself, and that allows us to put these ones here.
02:23
So now that we've done that, we have identified are absorbing states.
02:29
We have a matrix ready to go, then we can move on and look at meantime to absorption.
02:34
In the meantime, to absorption is how long or more properly, how many steps will it take to get from a particular stage to an absorbing stage? now we could construct more matrices, not just with one step in the future, but a matrix that shows two steps or three steps or four steps or 100 steps into the future.
02:55
And we could look at cumulative probability and multiply by the number of steps.
02:59
It would be an enormous headache, and it would only give us an estimate.
03:03
But there is a theorem that allows us to determine exactly what those mean times to absorption are.
03:10
And that's this theorem.
03:12
That's m t t a.
03:13
And we're going to use that and determine exactly how long it's going to take us now.
03:19
Before i do that, it shows up.
03:20
It has this queue in the equation.
03:23
And the lovely thing is, that's just a chunk that i'm going to pull out of this matrix.
03:28
This matrix is in what we call the canonical form.
03:32
The canonical form has it arranged so that the absorbing states are down here in the bottom rose.
03:37
However many there are they would be in the bottom rose.
03:40
If they're not down there, if you don't have the identity matrix down here, you would need to rearrange the matrix.
03:46
But ours has already set up.
03:47
And that means that over here, this chunk that does not include the absorbing states that is called q.
03:57
We're going to use that right away.
03:59
This other matrix over here that shows the probability of moving to those absorbing states that's called r.
04:08
Then over here i have an identity matrix.
04:11
You should see that in the canonical form, and then over here i have zeros.
04:16
So that's what i'm going to use to determine the meantime, to absorption.
04:21
The first step is to pull out q.
04:24
And so i have q right here.
04:27
So that's my cue matrix...