Chapter 11, Coupling of Markov Chains Video Solutions, Probability and Computing

Problem 1

Write a program that takes as input two positive integers $n_{1}$ and $n_{2}$ and two real numbers $p_{1}, p_{2}$ with $0 \leq p_{1}, p_{2} \leq 1$. The output of your program should be the variation distance between the binomial random variables $B\left(n_{1}, p_{1}\right)$ and $B\left(n_{2}, p_{2}\right)$, rounded to the nearest thousandth. Use your program to compute the variation distance between the following pairs of distributions: $B(20,0.5)$ and $B(20,0.49) ; B(20,0.5)$ and $B(21,0.5) ;$ and $B(21,0.5)$ and $B(21,0.49)$.

Bryan Lynn

Numerade Educator

01:39

Problem 2

Consider the Markov chain for shuffling cards, where at each step a card is chosen uniformly at random and moved to the top. Suppose that, instead of running the chain for a fixed number of steps, we stop the chain at the first step where every card has been moved to the top at least once. Show that, at this stopping time, the state of the chain is uniformly distributed on the $n !$ possible permutations of the cards.

Dominador Tan

Numerade Educator

01:39

Problem 3

Consider the Markov chain for shuffling cards, where at each step a card is chosen uniformly at random and moved to the top. Show that, if the chain. is run for only $(1-\varepsilon) n \ln n$ steps for some constant $\varepsilon>0$, then the variation distance is $1-o(1)$.

Dominador Tan

Numerade Educator

04:10

Problem 4

(a) Consider the Markov chain given by the transition matrix
$$
P=\left[\begin{array}{ccccc}
1 / 2 & 0 & 1 / 2 & 0 & 0 \\
0 & 1 / 2 & 1 / 2 & 0 & 0 \\
1 / 4 & 1 / 4 & 0 & 1 / 4 & 1 / 4 \\
0 & 0 & 1 / 2 & 1 / 2 & 0 \\
0 & 0 & 1 / 2 & 0 & 1 / 2
\end{array}\right]
$$
Explain why Theorem $11.5$ is not useful when applied directly to P. Then apply Theorem $11.5$ to the Markov chain with transition matrix $\mathbf{P}^{2}$, and explain the implications for the convergence of the original Markov chain to its stationary distribution.
(b) Consider the Markov chain given by the transition matrix
$$
\mathbf{P}=\left[\begin{array}{ccccc}
1 / 2 & 0 & 1 / 2 & 0 & 0 \\
0 & 1 / 2 & 1 / 2 & 0 & 0 \\
1 / 5 & 1 / 5 & 1 / 5 & 1 / 5 & 1 / 5 \\
0 & 0 & 1 / 2 & 1 / 2 & 0 \\
0 & 0 & 1 / 2 & 0 & 1 / 2
\end{array}\right]
$$
Apply Theorem $11.5$ to P. Then apply Theorem $11.5$ to the Markov chain with transition matrix $\mathbf{P}^{2}$, and explain the implications for the convergence of the original Markof chain to its stationary distribution. Which application gives better bounds on the variation distance?

Mahnoor Amin

Numerade Educator

05:54

Problem 5

Suppose I repeatedly roll a standard six-sided die and obtain a sequencé of independent random variables $X_{1}, X_{2} \ldots .$ where $X_{i}$ is the outcome of the $i$ th roll. Let
$$
Y_{j}=\sum_{i=1}^{j} X_{t} \bmod 10
$$
be the sum of the first $j$ rolls considered modulo 10 . The sequence $Y_{j}$ forms a Markow chain. Determine its stationary distribution, and determine a bound on $\tau(\varepsilon)$ for this chain. (Hint: One approach is to use the method of Exercise 11.4.)

Ahmad Reda

Numerade Educator

04:37

Problem 6

Theorem $11.5$ is useful only if there exists a nonzero entry in at least one column of the transition matrix P of the Markov chain. Argue that for any finite, aperiodic, irreducible Markov chain, there exists a time $T$ such that every entry of $\mathbf{P}^{T}$ is nonzero. Explain how this can be used in conjunction with Theorem 11.5.

Robin Corrigan

Numerade Educator

26:40

Problem 7

A technique we use repeatedly in the chapter is to define a distance function $d_{t}$ that represent the distance between the two states of our coupling after $t$ steps, and then show that when $d_{t}>0$ there exists a $\beta<1$ such that
$$
\mathbf{E}\left[d_{t+1} \mid d_{t}\right] \leq \beta d_{t}
$$
(a) Under this condition, give an upper bound for $\tau(\varepsilon)$ in terms of $\beta$ and $d^{*}$, where $d^{*}$ is the maximum distance over all possible pairs of initial states for the coupling.
(b) Suppose that instead we have
$$
\mathbf{E}\left[d_{t+1} \mid d_{t}\right] \leq d_{t} .
$$
Suppose we have the additional conditions that $d_{t+1}$ is one of $d_{t}, d_{t}-1$, or $d_{t}+1$ and that $\operatorname{Pr}\left(d_{r} \neq d_{t+1}\right) \geq \gamma$. Give an upper bound for $\tau(\varepsilon)$ in terms of $\varepsilon, d^{*}$, and $\gamma$. Your answer should by polynomial in $d^{*}$ and $1 / \gamma$. (Hint: Think of $d_{t}$ as being similar to a random walk on the line.)
(c) Using (a) and (b), show that the mixing time of the coloring chain of Section 11.5 is polynomial in the number of vertices in the graph and $\ln (1 / \varepsilon)$, even when the number of colors is only $2 \Delta$.
(d) By extending the argument of part (b), show that the mixing time of the Markov chain for independent sets given in Section $11.6$ is polynomial in the number of vertices in the graph and $\ln (1 / \varepsilon)$.

Chris Trentman

Numerade Educator

02:35

Problem 8

Consider the random walk on a non-bipartite, connected graph on $n$ vertices, where each vertex has the same degree $d>n / 2$. Show that
$$
\tau(\varepsilon) \leq \frac{\ln \varepsilon}{\ln (1-(2 d-n) / d)}
$$

Wen Zheng

Numerade Educator

10:03

Problem 9

Consider a Markov chain on $n$ points $[0, n-1]$ lying in order on a circle. At each step, the chain stays at the current point with probability $1 / 2$ or moves to the next point in the clockwise direction with probability $1 / 2$. Find the stationary distribution and show that, for any $\varepsilon>0$, the mixing time $\tau(\varepsilon)$ is $O\left(n^{2} \ln (1 / \varepsilon)\right)$.

Robin Corrigan

Numerade Educator

10:48

Problem 10

In Section 11.2.3, we suggested the following coupling $Z_{t}=\left(X_{t}, Y_{t}\right)$. First choose a transition for the chain $X_{t}$, with $v \in X_{t}$ and $w \in V$. If $v \in Y_{t}$, use the same vertices $v$ and $w$ for the transition of the chain $Y_{f}$; otherwise, choose uniformly at random a vertex $v^{\prime} \in Y_{t}-X_{t}$ and then perform the transition in the chain $Y_{t}$ with the pair $v^{\prime}$ and $w$. Show that this is a valid coupling that satisfies Definition $11.3$.

Brian Lin

Numerade Educator

03:20

Problem 11

Show that the Markov chain for sampling all independent sets of size exactly $k \leq n / 3(\Delta+1)$ in a graph with $n$ nodes and maximum degree $\Delta$, as defined in Section 11.2.3, is ergodic and has a uniform stationary distribution.

James Kiss

Numerade Educator

26:40

Problem 12

We wish to improve the coupling technique used in Section 11.2.3 in order to obtain a better bound. The improvement here is related to the technique used to prove Theorem 11.8. As with the coupling in Section 11.2.3, if an attempt is made to move $v \in X_{t}-Y_{I}$ to a vertex $w$ then the same attempt is made with the matched vertex in the other chain. If, however, an attempt is made to move a vertex $v \in X_{t} \cap Y_{t}$ in both chains, we no longer attempt to make the same move.
(a) Assume there exists a set $S_{1}$ of exactly $d_{t}(\Delta+1)$ distinct vertices that are members of or neighbors of vertices in $X_{t}-Y_{t}$ and, likewise, a set $S_{2}$ of exactly $d_{t}(\Delta+1)$ distinct vertices that are members of or neighbors of vertices in $Y_{t}-X_{i} ;$ assume further that $S_{2}$ and $S_{1}$ are disjoint. Suppose that we match up the vertices in $S_{1}$ and $S_{2}$ in a one-to-one fashion. Argue that the moves can be coupled so that, when one chain attempts and fails to move $v$ to a vertex in $S_{1}$ in one chain, it also attempts and fails to move $v$ to the matching vertex in $S_{2}$ in the other chain. Similarly, argue that the moves can be coupled so that, when one chain attempts and succeeds in moving $v$ to a vertex in $S_{1}$ in one chain, it also attempts and succeeds in moving $v$ to the matching vertex in $S_{2}$ in the other chain. Show that the coupling gives
$$
\operatorname{Pr}\left(d_{t+1}=d_{t}+1\right) \leq \frac{k-d_{t}}{k} \frac{d_{t}(\Delta+1)}{n}
$$
(b) In the general case, $S_{1}$ and $S_{2}$ are not necessarily disjoint or of equal size. Show that in this case, by pairing up failing moves as much as possible, the number of choices for $w$ that can increase $d_{t}$ is $\max \left(\left|S_{1}\right|,\left|S_{2}\right|\right) \leq d_{t}(\Delta+1)$. Then argue that
$$
\operatorname{Pr}\left(d_{t+1}=d_{t}+1\right) \leq \frac{k-d_{l}}{k} \frac{d_{t}(\Delta+1)}{n}
$$
holds in all cases. Use this coupling to obtain a polynomial bound on $\tau(\varepsilon)$ that holds for any $k \leq$
(c) Use this coup

Chris Trentman

Numerade Educator

07:01

Problem 13

For a Markov chain with state space $S$ and for any nonnegative integer $t$, let
$$
\bar{\Delta}(t)=\max _{x, y \in S}\left\|p_{x}^{t}-p_{y}^{t}\right\|
$$
Assume also that the Markov chain has a stationary distribution.
(a) Prove $\bar{\Delta}(s+t) \leq \bar{\Delta}(s) \bar{\Delta}(t)$ for any positive integers $s$ and $t$.
(b) Prove $\Delta(s+t) \leq \Delta(s) \bar{\Delta}(t)$ for any positive integers $s$ and $t$.
(c) Prove
$$
\Delta(t) \leq \bar{\Delta}(t) \leq 2 \Delta(t)
$$
for any positive integer $t$.

Robin Corrigan

Numerade Educator

03:42

Problem 14

Consider the following variation on shuffling for a deck of $n$ cards. At each step, two specific cards are chosen uniformly at random from the deck, and their positions are exchanged. (It is possible both choices give the same card, in which case no change occurs.) (a) Argue that the following is an equivalent process: at each step, a specific card is chosen uniformly at random from the deck, and a position from $[1, n]$ is chosen uniformly at random; then the card at position $i$ exchanges positions with the specific card chosen.
(b) Consider the coupling where the two choices of card and position are the same for both copies of the chain. Let $X_{r}$ be the number of cards whose positions are different in the two copies of the chain. Show that $X_{t}$ is nonincreasing over time.
(c) Show that
$$
\operatorname{Pr}\left(X_{t+1} \leq X_{t}-1 \mid X_{t}>0\right) \geq\left(\frac{X_{t}}{n}\right)^{2}
$$
(d) Argue that the expected time until $X_{t}$ is 0 is $O\left(n^{2}\right)$, regardless of the starting state of the two chains.

Amany Waheeb

Numerade Educator

08:15

Problem 15

Modify the arguments of Lemma $10.3$ and Lemma $10.4$ to show that, if we have an FPAUS for proper colorings for any $c \geq \Delta+2$, then we also have an FPRAS for this value of $c$.

Oswaldo Jiménez

Numerade Educator

11:11

Problem 16

Consider the following simple Markov chain whose states are independent sets in a graph $G=(V, E) .$ To compute $X_{i+1}$ from $X_{i}$ :
- choose a vertex $v$ uniformly at random from $V$, and flip a fair coin;
- if the flip is heads and $v \in X_{i}$, then $X_{t+1}=X_{i} \backslash\{v\}$;
- if the flip is heads and $v \notin X_{i}$, then $X_{i+1}=X_{i}$;
- if the flip is tails, $v \notin X_{i}$, and adding $v$ to $X_{i}$ still gives an independent set, then $X_{i+1}=X_{i} \cup\{v\} ;$
- if the flip is tails and $v \in X_{i}$, then $X_{i+1}=X_{i} .$
(a) Show that the stationary distribution of this chain is uniform over all independent sets,
(b) We consider this Markov chain specifically on cycles and line graphs. For a line graph with $n$ vertices, the vertices are labeled 1 to $n$, and there is an edge from 1 to 2,2 to $3, \ldots, n-1$ to $n$. A cycle graph on $n$ vertices is the same with the addition of an edge from $n$ to 1 .

Devise a coupling $\left(X_{1}, Y_{f}\right)$ for this Markov chain such that, on line graphs and cycle graphs, if $d_{l}=\left|X_{t}-Y_{l}\right|+\left|Y_{t}-X_{t}\right|$ is the number of vertices on which the two independent sets disagree, then at each step the coupling is at least as likely to reduce $d_{t}$ as to increase $d_{t *}$
(c) With the coupling from part (b), argue that you can use this chain to obtain an FPAUS for independent sets on a cycle graph or line graph. You may want to use Exercise 11.7.
(d) For the special cases of line graphs and cycle graphs, we can derive exact formulas for the number of independent sets. Derive exact formulas for these cases and prove that your formulas are correct. (Hint: You may want to express your results in terms of Fibonacci numbers.)

Ruirui Liu

Numerade Educator

23:03

Problem 17

For integers $a$ and $b$, an $a \times b$ grid is a graph whose vertices are all ordered pairs of integers $(x, y)$ with $0 \leq x<a$ and $0 \leq y<b$. The edges of the graph connect all pairs of distinct vertices $(x, y)$ and $\left(x^{\prime}, y^{\prime}\right)$ such that $\left|x-x^{\prime}\right|+\left|y-y^{\prime}\right|=$ 1. That is, every vertex is connected to the neighbors up, down, left, and right of it, where vertices on the boundary are connected to the relevant points only. Consider the following problems on the graph given by the $10 \times 10$ grid.
(a) Implement an FPAUS to generate an $\varepsilon$-uniform proper 10 -coloring of the graph, where $\varepsilon$ is given as an input. Discuss how many steps your Markov chain runs for, what your starting state is, and any other relevant details.
(b) Using your FPAUS as a subroutine, implement an FPRAS to generate an $(\varepsilon, \delta)$ )approximation to the number of proper 10 -colorings of the graph. Test your code by running it to obtain a $(0.1,0.001)$-approximation. (Note: This may take a significant amount of time to run.) Discuss the ordering you choose on the edges, how many samples are required at each step, how many steps of the Markov chain you perform in total throughout the process, and any other relevant details.

Chris Trentman

Numerade Educator

01:54

Problem 18

In Section 11.2.3 we considered the following Markov chain on independent sets: a move is made from the independent set $X_{t}$ by choosing a vertex $v \in$ $X_{t}$ uniformly at random and picking a vertex $w$ uniformly at random from the graph. If $X_{t}-\{v\}+\{w\}$ is an independent set, then $X_{t+1}=X_{t}-\{v\}+\{w\}$; otherwise; $X_{t+1}=X_{t}$. We have shown that the chain converges quickly to its stationary distribuition via bounding $\tau(\varepsilon)$ by an expression that is polynomial in $n$ and $\ln (1 / \varepsilon)$ whenever $k \leq n / 2(\Delta+1)$. Use the idea of path coupling to simplify the proof.

Nick Johnson

Numerade Educator

01:39

Problem 19

In Section 11.5, we considered a simple Markov chain for coloring. Suppose that we can apply the path coupling technique. (You do not need to show this.) In this case, we can just consider the case where $d_{t}=1$. Give a simpler argument that, when $d_{t}=1$ and $c>2 \Delta, \mathbf{E}\left[d_{t+1} \mid d_{r}\right] \leq \beta d_{t}$ for some $\beta<1$. Also show that, when $d_{t}=1$ and $c=2 \Delta, \mathbf{E}\left[d_{t+1} \mid d_{t}\right] \leq d_{t} .$

Dominador Tan

Numerade Educator