For a Markov chain with state space S and for any...

For a Markov chain with state space $S$ and for any nonnegative integer $t$, let $$ \bar{\Delta}(t)=\max _{x, y \in S}\left\|p_{x}^{t}-p_{y}^{t}\right\| $$ Assume also that the Markov chain has a stationary distribution. (a) Prove $\bar{\Delta}(s+t) \leq \bar{\Delta}(s) \bar{\Delta}(t)$ for any positive integers $s$ and $t$. (b) Prove $\Delta(s+t) \leq \Delta(s) \bar{\Delta}(t)$ for any positive integers $s$ and $t$. (c) Prove $$ \Delta(t) \leq \bar{\Delta}(t) \leq 2 \Delta(t) $$ for any positive integer $t$.

For a Markov chain with state space $S$ and for any nonnegative integer $t$, let
$$
\bar{\Delta}(t)=\max _{x, y \in S}\left\|p_{x}^{t}-p_{y}^{t}\right\|
$$
Assume also that the Markov chain has a stationary distribution.
(a) Prove $\bar{\Delta}(s+t) \leq \bar{\Delta}(s) \bar{\Delta}(t)$ for any positive integers $s$ and $t$.
(b) Prove $\Delta(s+t) \leq \Delta(s) \bar{\Delta}(t)$ for any positive integers $s$ and $t$.
(c) Prove
$$
\Delta(t) \leq \bar{\Delta}(t) \leq 2 \Delta(t)
$$
for any positive integer $t$.

Key Concepts

Markov Chain

A Markov chain is a type of stochastic process characterized by a set of states and transition probabilities between those states, where the future evolution depends only on the current state and not on the preceding sequence of events. This memoryless property is fundamental in analyzing time-evolving random systems and their long?term behavior.

Stationary Distribution

A stationary distribution for a Markov chain is a probability distribution that remains unchanged under the chain’s transition dynamics. If the chain is started in this distribution, it will maintain the same distribution at all subsequent times, making it crucial for understanding equilibrium and convergence properties.

Total Variation Distance

Total variation distance is a metric for quantifying the difference between two probability distributions. In the context of Markov chains, it is used to measure how far the distribution at a given time is from another distribution, such as the stationary distribution, and plays an important role in analyzing the rate at which the chain converges.

Submultiplicative Property

The submultiplicative property refers to the concept that the distance between distributions at a combined time can be bounded by the product of distances over shorter time intervals. This property, evident in inequalities like ?(s + t) ? ?(s) * ?(t), is essential in understanding how discrepancies in probability distributions contract over time, thus providing insight into the mixing behavior of the chain.

Coupling Method

The coupling method is a technique used to analyze the convergence rates of Markov chains by constructing a joint process of two copies of the chain started from different states. By showing that the probability of the copies being unequal decreases over time, one can derive bounds on the total variation distance, thereby connecting coupling with the chain’s convergence properties.

Transcript

00:01 So in this question we are asked to show this proof, which says that for a sample space or state space of 1 through s, show that the m plus n step transition probability of going from i to j is equal to the summation for all k of the m step transition probability of going from i to k times the n -step transition probability of going from k to j.

00:40 And so they give us a hint in the question to start with this probability here.

00:49 So both of these represent the m plus n -step transition probability of going from i to j.

01:03 So we can also write this as the probability.

01:35 So this is the probability of going from state i to state j at n plus m transitions or at the n plus mth state but also having gone through state k at the nth state state so it's a probability of going to state j after n plus m transitions and also going to state k after n transitions given that we start at state i so we only want to show that we go to some intermediate state k and that can be any state at all in the sample space.

02:20 And so we want the probability of the union of these probabilities for all k.

02:29 Or rather, that's the union of these paths for all k.

02:33 So this is basically the probability of going through any path from i to k and then to j.

02:46 So this is k as an element of the state space.

02:51 So now since each of these paths is a unique path, no path is a subset of another path.

03:03 If you take one of the unique paths, you cannot have taken the other paths.

03:09 Which means that the probability of the union of all these paths can be written as the summation of the probabilities of the individual paths.

03:21 So this is the summation of k for all k is an element of s, probability of a right of a right.

03:32 Arriving at state j after n plus m transitions, and also arriving at state k after n transitions.

03:44 Conditional on having started at state i.

03:51 And now using the chain rule for probability, we can rewrite this as follows...

Please give Ace some feedback