I (author Carlton) have a six-room house whose configuration...

I (author Carlton) have a six-room house whose configuration is depicted in the accompanying diagram. When my sister and her family visit, I often play hide-and-seek with my young nephew, Lucas. Consider the following situation: Lucas counts to ten in Room $1,$ while I run and hide in Room $6 .$ Lucas" "strategy," as much as he has one, is such that standing in any room of the house, he is equally likely to next visit any of the adjacent rooms, regardless of where he has searched previously. (The exception, of course, is if he enters Room $6,$ in which case he discovers me and the round of hide-and-seek is over.) $$ \begin{array}{l}{\text { (a) Let } X_{n}=\text { the } n \text { th room Lucas visits (with } X_{0}=1, \text { his starting point). Construct the }} \\ {\text { one-step transition matrix for the corresponding Markov chain. }}\end{array} $$ $$ \begin{array}{l}{\text { (b) What is the probability that his third room-to-room transition will take him into Room } 2 ?} \\ {\text { (c) What is the fewest number of time steps (i.e., room transitions) required for Lucas to find }} \\ {\text { me? }} \\ {\text { (d) What is the probability that, after } 12 \text { time steps, he still hasn't found me? }}\end{array} $$

I (author Carlton) have a six-room house whose configuration is depicted in the accompanying
diagram. When my sister and her family visit, I often play hide-and-seek with my young
nephew, Lucas. Consider the following situation: Lucas counts to ten in Room $1,$ while I run and hide in Room $6 .$ Lucas" "strategy," as much as he has one, is such that standing in any room of the house, he is equally likely to next visit any of the adjacent rooms, regardless of where he has searched previously. (The exception, of course, is if he enters Room $6,$ in which case he
discovers me and the round of hide-and-seek is over.)
$$
\begin{array}{l}{\text { (a) Let } X_{n}=\text { the } n \text { th room Lucas visits (with } X_{0}=1, \text { his starting point). Construct the }} \\ {\text { one-step transition matrix for the corresponding Markov chain. }}\end{array}
$$
$$
\begin{array}{l}{\text { (b) What is the probability that his third room-to-room transition will take him into Room } 2 ?} \\ {\text { (c) What is the fewest number of time steps (i.e., room transitions) required for Lucas to find }} \\ {\text { me? }} \\ {\text { (d) What is the probability that, after } 12 \text { time steps, he still hasn't found me? }}\end{array}
$$

Key Concepts

Markov Chains

A Markov chain is a stochastic process that undergoes transitions from one state to another in a state space, with the property that the next state depends only on the current state (the Markov property). This process is typically analyzed using transition probabilities that dictate how likely it is to move from one state to any other state, making it an ideal framework for problems involving random movements among a set of states.

Transition Matrix

The transition matrix is a key tool in Markov chain analysis, encapsulating all the one-step transition probabilities between states in a matrix format. Each entry in the matrix represents the probability of moving from one specific state to another in a single time step, and it is essential for computing multi-step transition probabilities and for studying the long-term behavior of the chain.

State Space

The state space refers to the complete set of all possible states in the Markov chain. In stochastic models, understanding the state space is crucial because it defines the boundaries and possibilities of the process. This includes identifying both transient states (those that may be left and revisited) and any absorbing states, which play a special role in the dynamics.

Absorbing States

An absorbing state in a Markov chain is one that, once entered, cannot be left; the process terminates or remains in that state indefinitely. Recognizing absorbing states is important since they greatly influence the analysis of the chain, particularly in calculating the probability of eventually reaching an absorbing state and determining the expected number of steps to reach it.

Hitting Time (First Passage Time)

The hitting time, or first passage time, is defined as the number of transitions needed for the Markov chain to reach a specified state for the first time. This concept is vital in problems where one analyzes the minimum number of steps required for the process to reach an absorbing state or other state of interest, and it is used to compute various probability measures related to the progress of the chain.

Transcript

00:01 So in this problem, a house is described that has six rooms, and lucas is in the house, and he started in room one, and he, whatever room he's in, he has an equal probability of moving into any other adjacent room in his next move.

00:20 And we're given, so if x sub -n is the nth room that lucas visits, and room number one is the starting point, what is the one -step transition matrix for this markov chain? we're given x sub 0 is 1.

00:40 So the single step transition matrix can be made in this regard.

00:57 So if we look at the diagram in the problem, start with room 1.

01:01 First of all, one thing we can do is we know that he never stays in the same room that he's in.

01:07 So he always transitions into another adjacent room.

01:10 So all these self -loop transition probabilities are zero.

01:24 In room 1, there's only one possibility.

01:26 He can only go into room two from there.

01:29 So we know for sure that that's where he's going.

01:32 So therefore, all of the other transitions must be zero.

01:42 And if he's in room two, he has access to room one, three, and four.

01:48 And he has an equal likelihood of going into any of those rooms, based on the way that the question is formulated.

01:55 So it's one over three, one over three, and one over three.