(a) Alice and Bob are wandering around randomly,...

(a) Alice and Bob are wandering around randomly, independently of each other, in a house with $M$ rooms, labeled $1,2, \ldots, M .$ Let $d_{i}$ be the number of doors in room $i$ (leading to other rooms, not leading outside). At each step, Alice moves to another room by choosing randomly which door to go through (with equal probabilities). Bob does the same, independently. The Markov chain they each follow is irreducible and aperiodic. Let $A_{n}$ and $B_{n}$ be Alice's room and Bob's room at time $n$, respectively, for $n=0,1,2, \ldots$ Find $\lim _{n \rightarrow \infty} P\left(A_{n}=i, B_{n}=j\right)$. (b) With setup as in (a), let $p_{i j}$ be the transition probability for going from room $i$ to room $j$. Let $t_{i k}$ be the expected first time at which Alice and Bob are in the same room, if Alice starts in room $i$ and Bob starts in room $k$ (note that $t_{i k}=0$ for $i=k$ ). Provide a system of linear equations, the solution of which would yield $t_{i k}$ for all rooms $i, k$.

(a) Alice and Bob are wandering around randomly, independently of each other, in a house with $M$ rooms, labeled $1,2, \ldots, M .$ Let $d_{i}$ be the number of doors in room $i$ (leading to other rooms, not leading outside). At each step, Alice moves to another room by choosing randomly which door to go through (with equal probabilities). Bob does the same, independently. The Markov chain they each follow is irreducible and aperiodic. Let $A_{n}$ and $B_{n}$ be Alice's room and Bob's room at time $n$, respectively, for $n=0,1,2, \ldots$
Find $\lim _{n \rightarrow \infty} P\left(A_{n}=i, B_{n}=j\right)$.
(b) With setup as in (a), let $p_{i j}$ be the transition probability for going from room $i$ to room $j$. Let $t_{i k}$ be the expected first time at which Alice and Bob are in the same room, if Alice starts in room $i$ and Bob starts in room $k$ (note that $t_{i k}=0$ for $i=k$ ).

Provide a system of linear equations, the solution of which would yield $t_{i k}$ for all rooms $i, k$.

Key Concepts

Markov Chain Properties

A fundamental property of Markov chains is that if they are irreducible (any state can be reached from any other) and aperiodic (the chain does not cycle with a fixed period), then they possess a unique stationary distribution. This guarantees that, as time progresses, the probability distribution over the states converges to this stationary distribution regardless of the initial state.

Stationary Distribution for Random Walks on Graphs

In a random walk on a graph (or a house with multiple rooms connected by doors), the stationary distribution is typically related to the connectivity (degree) of each node (room). Specifically, the probability of being in a room converges to a value proportional to the number of doors it has, meaning state i has probability d??? divided by the sum of all degrees. This reflects the fact that more connected rooms are visited more frequently.

Joint Distribution of Independent Processes

When two agents independently follow the same Markov chain, their long?term joint distribution is given by the product of their individual stationary distributions. This concept emerges from the independence of the agents' movements; in the limit, the probability that one agent is in room i and the other in room j is simply the product of the probabilities for room i and room j individually.

Expected Hitting Times and Recurrence Relations

The expected time until an event occurs in a Markov process, such as two independent processes meeting (hitting time), is characterized by recurrence relations derived from conditioning on the next state. If the process starts in a configuration where the two agents are in different states, the expected time to meet can be set up recursively by considering the one time-unit taken plus the expected time from the new pair of states, with the base case being zero when the agents are in the same state.

System of Linear Equations for Meeting Times

To compute the expected meeting time (or hitting time) for two independent Markov chains, one can set up a system of linear equations. For any pair of distinct states i and k, the expected time t?? until the processes meet satisfies the equation t?? = 1 + ? (over j) ? (over l) p?? · p?? · t??, with the condition t?? = 0 for any state i. This system, which has one equation per pair of states, captures the probabilistic evolution of the two processes until they coincide.

Please give Ace some feedback