• Home
  • Textbooks
  • Probability and Computing
  • Balls, Bins, and Random Graphs

Probability and Computing

Michael Mitzenmacher , Eli Upfal

Chapter 5

Balls, Bins, and Random Graphs - all with Video Answers

Educators

Chapter Questions

01:16

Problem 1

For what values of $n$ is $(1+1 / n)^{n}$ within $1 \%$ of e? Within $0.0001 \%$ of e? Similarly, for what values of $n$ is $(1-1 / n)^{n}$ within $1 \%$ of $1 / \mathrm{e}$ ? Within $0.0001 \%$ ?

Nick Johnson
Nick Johnson
Numerade Educator
02:20

Problem 2

Suppose that Social Security numbers were issued uniformly at random, with replacement. That is, your Social Security number would consist of just nine randomly generated digits, and no check would be made to ensure that the same number was not issued twice. Sometimes, the last four digits of a Social Security number are used as a password. How many people would you need to have in a room before it was more likely than not that two had the same last four digits? How many numbers could be issued before it would be more likely than not that there is a duplicate number? How would you answer these two questions if Social Security numbers had 13 digits? Try to give exact numerical answers.

Jeff Vermeire
Jeff Vermeire
Numerade Educator
02:37

Problem 3

Suppose that balls are thrown randomly into $n$ bins. Show, for some constant $c_{1}$, that if there are $c_{1} \sqrt{n}$ balls then the probability that no two land in the same bin is at most 1/e. Similarly, show for some constant $c_{2}$ (and sufficiently large $n$ ) that, if there are $c_{2} \sqrt{n}$ balls, then the probability that no two land in the same bin is at least 1/2. Make these constants as close to optimal as possible. Hint: You may want to use the facts that
$$
\mathrm{e}^{-x} \geq 1-x
$$
and
$$
\mathrm{e}^{-x-x^{2}} \leq 1-x \quad \text { for } x \leq \frac{1}{2}
$$

Nick Johnson
Nick Johnson
Numerade Educator
02:12

Problem 4

In a lecture hall containing 100 people, you consider whether or not there are three people in the room who share the same birthday. Explain how to calculate this probability exactly, using the same assumptions as in our previous analysis.

Maxime Rossetti
Maxime Rossetti
Numerade Educator
01:21

Problem 5

Let $X$ be a Poisson random variable with mean $\mu$, representing the number of errors on a page of this book. Each error is independently a grammatical error
119 with probability $p$ and a spelling error with probability $1-p$. If $Y$ and $Z$ are random variables representing the number of grammatical and spelling errors (respectively) on a page of this book, prove that $Y$ and $Z$ are Poisson random variables with means $\mu p$ and $\mu(1-p)$, respectively. Also, prove that $Y$ and $Z$ are independent.

Nick Johnson
Nick Johnson
Numerade Educator
01:07

Problem 6

Use the Taylor expansion
$$
\ln (1+x)=x-\frac{x^{2}}{2}+\frac{x^{3}}{3}-\frac{x^{4}}{4}+\cdots
$$
to prove that, for any $x$ with $|x| \leq 1$,
$$
\mathrm{e}^{x}\left(1-x^{2}\right) \leq 1+x \leq \mathrm{e}^{x}
$$

Carson Merrill
Carson Merrill
Numerade Educator
02:37

Problem 7

Suppose that $n$ balls are thrown independently and uniformly at random into $n$ bins.
(a) Find the conditional probability that bin 1 has one ball given that exactly one ball fell into the first three bins.
(b) Find the conditional expectation of the number of balls in bin 1 under the condition that bin 2 received no balls.
(c) Write an expression for the probability that bin 1 receives more balls than bin $2 .$

Nick Johnson
Nick Johnson
Numerade Educator
View

Problem 8

Our analysis of Bucket sort in Section 5.2.2 assumed that $n$ elements were chosen independently and uniformly at random from the range $\left[0,2^{k}\right) .$ Suppose instead that $n$ elements are chosen independently from the range $\left[0,2^{k}\right)$ according to a distribution with the property that any number $x \in\left[0,2^{k}\right)$ is chosen with probability at most $a / 2^{k}$ for some fixed constant $a>0$. Show that, under these conditions, Bucket sort still requires linear expected time.

Victor Salazar
Victor Salazar
Numerade Educator
02:47

Problem 9

Consider the probability that every bin receives exactly one ball when $n$ balls are thrown randomly into $n$ bins.
(a) Give an upper bound on this probability using the Poisson approximation.
(b) Determine the exact probability of this event.
(c) Show that these two probabilities differ by a multiplicative factor that equals the probability that a Poisson random variable with parameter $n$ takes on the value $n$. Explain why this is implied by Theorem $5.6$.

Nick Johnson
Nick Johnson
Numerade Educator
View

Problem 10

Consider throwing $m$ balls into $n$ bins, and for convenience let the bins be numbered from 0 to $n-1$. We say there is a $k$-gap starting at bin $i$ if bins $i, i+1, \ldots, i+k-1$ are all empty.
(a) Determine the expected number of $k$-gaps.
(b) Prove a Chernoff-like bound for the number of $k$-gaps. (Hint: If you let $X_{i}=1$ when there is a $k$-gap starting at bin $i$, then there are dependencies between $X_{i}$ and $X_{i+1}$; to avoid these dependencies, you might consider $X_{i}$ and $X_{i+k}$ )
120

Victor Salazar
Victor Salazar
Numerade Educator
02:43

Problem 11

The following problem models a simple distributed system wherein agents contend for resources but "back off" in the face of contention. Balls represent agents, and bins represent resources.

The system evolves over rounds. Every round, balls are thrown independently and uniformly at random into $n$ bins. Any ball that lands in a bin by itself is served and removed from consideration. The remaining balls are thrown again in the next round. We begin with $n$ balls in the first round, and we finish when every ball is served.
(a) If there are $b$ balls at the start of a round, what is the expected number of balls at the start of the next round?
(b) Suppose that every round the number of balls served was exactly the expected number of balls to be served. Show that all the balls would be served in $O(\log \log n)$ rounds. (Hint: If $x_{j}$ is the expected number of balls left after $j$ rounds, show and use that $x_{j+1} \leq x_{j}^{2} / n$.)

Amany Waheeb
Amany Waheeb
Numerade Educator
02:07

Problem 12

Suppose that we vary the balls-and-bins process as follows. For convenience let the bins be numbered from 0 to $n-1$. There are $\log _{2} n$ players. Each player randomly chooses a starting location $\ell$ uniformly from $[0, n-1]$ and then places one ball in each of the bins numbered $\ell \bmod n, \ell+1 \bmod n, \ldots, \ell+n / \log _{2} n-1 \bmod n$. Argue that the maximum load in this case is only $O(\log \log n / \log \log \log n)$ with probability that approaches 1 as $n \rightarrow \infty$.

Nick Johnson
Nick Johnson
Numerade Educator
03:09

Problem 13

We prove that if $Z$ is a Poisson random variable of mean $\mu$, where $\mu \geq$ 1 is an integer, then $\operatorname{Pr}(Z \geq \mu) \geq 1 / 2$ and $\operatorname{Pr}(Z \leq \mu) \geq 1 / 2$.
(a) Show that $\operatorname{Pr}(Z=\mu+h) \geq \operatorname{Pr}(Z=\mu-h-1)$ for $0 \leq h \leq \mu-1$.
(b) Using part (a), argue that $\operatorname{Pr}(Z \geq \mu) \geq 1 / 2$.
(c) Show that $\operatorname{Pr}(Z=\mu-h) \geq \operatorname{Pr}(Z=\mu+h+1)$ for $0 \leq h \leq \mu$.
(d) Determine a lower bound on $\operatorname{Pr}(Z=\mu-h)-\operatorname{Pr}(Z=\mu+h+1)$.
(e) Determine an upper bound on $\operatorname{Pr}(Z \geq 2 \mu+2)$.
(f) Using parts (c) $-(\mathrm{e})$, argue that $\operatorname{Pr}(Z \leq \mu) \geq 1 / 2$.

SS
Sagar Singh
Numerade Educator
04:25

Problem 14

(a) In Theorem $5.7$ we showed that, for any nonnegative functions $f$,
$$
\mathbf{E}\left[f\left(Y_{1}^{(m)}, \ldots, Y_{n}^{(m)}\right)\right] \geq \mathbf{E}\left[f\left(X_{1}^{(m)}, \ldots, X_{n}^{(m)}\right)\right] \operatorname{Pr}\left(\sum Y_{i}^{(m)}=m\right)
$$
Prove that if $\mathbf{E}\left[f\left(X_{1}^{(m)}, \ldots, X_{n}^{(m)}\right)\right]$ is monotonically increasing in $m$, then
$$
\mathbf{E}\left[f\left(Y_{1}^{(m)}, \ldots, Y_{n}^{(m)}\right)\right] \geq \mathbf{E}\left[f\left(X_{1}^{(m)}, \ldots, X_{n}^{(m)}\right)\right] \operatorname{Pr}\left(\sum Y_{i}^{(m)} \geq m\right),
$$
again under the condition that $f$ is nonnegative. Make a similar statement for the case when $\mathbf{E}\left[f\left(X_{1}^{(m)}, \ldots, X_{n}^{(m)}\right)\right]$ is monotonically decreasing in $m$.
(b) Using part (a) and Exercise 5.13, Prove Theorem 5.10.

Nick Johnson
Nick Johnson
Numerade Educator
03:23

Problem 15

We consider another way to obtain Chemoff-like bounds in the setting of balls and bins without using Theorem 5.7. Consider $n$ balls thrown randomly into $n$ bins. Let $X_{i}=1$ if the $i$ th bin is empty and 0 otherwise. Let $X=\sum_{i=1}^{n} X_{i}$. Let $Y_{i}, i=1, \ldots, n$, be independent Bernoulli random variables that are 1 with probability $p=(1-1 / n)^{n}$. Let $Y=\sum_{i=1}^{n} Y_{i}$.
(a) Show that $\mathbf{E}\left[X_{1} X_{2} \cdots X_{k}\right] \leq \mathbf{E}\left[Y_{1} Y_{2} \cdots Y_{k}\right]$ for any $k \geq 1$.
(b) Show that $\mathbf{E}\left[\mathrm{e}^{t X}\right] \leq \mathbf{E}\left[\mathrm{e}^{t Y}\right]$ for all $t \geq 0$. (Hint: Use the expansion for $\mathrm{e}^{x}$ and compare $\mathbf{E}\left[X^{k}\right]$ to $\left.\mathbf{E}\left[Y^{k}\right] .\right)$
(c) Derive a Chernoff bound for $\operatorname{Pr}(X \geq(1+\delta) \mathbf{E}[X])$.

SS
Sagar Singh
Numerade Educator
17:27

Problem 16

Let $G$ be a random graph generated using the $G_{n, p}$ model.
(a) A clique of $k$ vertices in a graph is a subset of $k$ vertices such that all $\left(\begin{array}{l}k \\ 2\end{array}\right)$ edges between these vertices lie in the graph. For what value of $p$, as a function of $n$, is the expected number of cliques of five vertices in $G$ equal to 1 ?
(b) A $K_{3,3}$ graph is a complete bipartite graph with three vertices on each side. In other words, it is a graph with six vertices and nine edges; the six distinct vertices are arranged in two groups of three, and the nine edges connect each of the nine pairs of vertices with one vertex in each group. For what value of $p$, as a function of $n$, is the expected number of $K_{3,3}$ subgraphs of $G$ equal to 1 ?
(c) For what value of $p$, as a function of $n$, is the expected number of Hamiltonian cycles in the graph equal to 1 ?

Chris Trentman
Chris Trentman
Numerade Educator
02:47

Problem 17

Theorem $5.7$ shows that any event that occurs with small probability in the balls-and-bins setting where the number of balls in each bin is an independent Poisson random variable also occurs with small probability in the standard balls-and-bins model. Prove a similar statement for random graphs: Every event that happens with small probability in the $G_{n, p}$ model also happens with small probability in the $G_{n, N}$ model for $N=\left(\begin{array}{l}n \\ 2\end{array}\right) p$.

Nick Johnson
Nick Johnson
Numerade Educator
02:35

Problem 18

An undirected graph on $n$ vertices is disconnected if there exists a set of $k<n$ vertices such that there is no edge between this set and the rest of the graph. Otherwise, the graph is said to be connected. Show that there exists a constant $c$ such that if $N \geq c n \log n$ then, with probability $O\left(\mathrm{e}^{-n}\right)$, a graph randomly chosen from $G_{n, N}$ is connected.

WZ
Wen Zheng
Numerade Educator
00:26

Problem 19

Prove Theorem $5.14 .$

Vikash Ranjan
Vikash Ranjan
Numerade Educator
17:27

Problem 20

(a) Let $f(n)$ be the expected number of random edges that must be added before an empty undirected graph with $n$ vertices becomes connected. (Connectedness is defined in Exercise 5.18.) That is, suppose that we start with a graph on $n$ vertices with zero edges and then repeatedly add an edge, chosen uniformly at random from all edges not currently in the graph, until the graph becomes connected. If $X_{n}$ represents the number of edges added, then $f(n)=\mathbf{E}\left[X_{n}\right]$.

Write a program to estimate $f(n)$ for a given value of $n$. Your program should track the connected components of the graph as you add edges until the graph becomes connected. You will probably want to use a disjoint set data structure, a topic covered in standard undergraduate algorithms texts. You should try $n=100,200,300,400,500$, $600,700,800,900$, and 1000 . Repeat each experiment 100 times, and for each value of $n$ compute the average number of edges needed. Based on your experiments, suggest a function $h(n)$ that you think is a good estimate for $f(n)$.
(b) Modify your program for the problem in part (a) so that it also keeps track of isolated vertices. Let $g(n)$ be the expected number of edges added before there are no more isolated vertices. What seems to be the relationship between $f(n)$ and $g(n) ?$

Chris Trentman
Chris Trentman
Numerade Educator
06:25

Problem 21

In hashing with open addressing, the hash table is implemented as an array and there are no linked lists or chaining. Each entry in the array either contains one hashed item or is empty. The hash function defines, for each key $k$, a probe sequence $h(k, 0), h(k, 1), \ldots$ of table locations. To insert the key $k$, we first examine the sequence of table locations in the order defined by the key's probe sequence until we find an empty location; then we insert the item at that position. When searching for an item in the hash table, we examine the sequence of table locations in the order defined by the key's probe sequence until either the item is found or we have found an empty location in the sequence. If an empty location is found, this means the item is not present in the table.

An open-address hash table with $2 n$ entries is used to store $n$ items. Assume that the table location $h(k, j)$ is uniform over the $2 n$ possible table locations and that all $h(k, j)$ are independent.
(a) Show that, under these conditions, the probability of an insertion requiring more than $k$ probes is at most $2^{-k}$.
(b) Show that, for $i=1,2, \ldots, n$, the probability that the $i$ th insertion requires more than $2 \log n$ probes is at most $1 / n^{2}$.

Let the random variable $X_{i}$ denote the number of probes required by the $i$ th insertion. You have shown in part (b) that $\operatorname{Pr}\left(X_{i}>2 \log n\right) \leq 1 / n^{2}$. Let the random variable $X=$ $\max _{1 \leq i \leq n} X_{i}$ denote the maximum number of probes required by any of the $n$ insertions.
(c) Show that $\operatorname{Pr}(X>2 \log n) \leq 1 / n$.
(d) Show that the expected length of the longest probe sequence is $\mathbf{E}[X]=O(\log n)$.

Robin Corrigan
Robin Corrigan
Numerade Educator
01:57

Problem 22

Bloom filters can be used to estimate set differences. Suppose you have a set $X$ and I have a set $Y$, both with $n$ elements. For example, the sets might represent our 100 favorite songs. We both create Bloom filters of our sets, using the same number of bits $m$ and the same $k$ hash functions. Determine the expected number of bits where our Bloom filters differ as a function of $m, n, k$, and $|X \cap Y|$. Explain how this could be used as a tool to find people with the same taste in music more easily than comparing lists of songs directly.

Nick Johnson
Nick Johnson
Numerade Educator
01:57

Problem 23

Suppose that we wanted to extend Bloom filters to allow deletions as well as insertions of items into the underlying set. We could modify the Bloom filter to be an array of counters instead of an array of bits. Each time an item is inserted into a Bloom filter, the counters given by the hashes of the item are increased by one. To delete an item, one can simply decrement the counters. To keep space small, the counters should be a fixed length, such as 4 bits.

Explain how errors can arise when using fixed-length counters. Assuming a setting where one has at most $n$ elements in the set at any time, $m$ counters, $k$ hash functions, and counters with $b$ bits, explain how to bound the probability that an error occurs over the course of $t$ insertions or deletions.

Nick Johnson
Nick Johnson
Numerade Educator
01:57

Problem 24

Suppose that you built a Bloom filter for a dictionary of words with $m=2^{b}$ bits. A co-worker building an application wants to use your Bloom filter but has only $2^{b-1}$ bits available. Explain how your colleague can use your Bloom filter to avoid rebuilding a new Bloom filter using the original dictionary of words.

Nick Johnson
Nick Johnson
Numerade Educator
01:16

Problem 25

For the leader election problem alluded to in Section 5.5.4, we have $\boldsymbol{n}$ users, each with an identifier. The hash function takes as input the identifier and outputs a $b$-bit hash value, and we assume that these values are independent and uniformly distributed. Each user hashes its identifier, and the leader is the user with the smallest hash value. Give lower and upper bounds on the number of bits $b$ necessary to ensure that a unique leader is successfully chosen with probability $p$. Make your bounds as tight as possible.

Vysakh M
Vysakh M
Numerade Educator
03:42

Problem 26

Consider Algorithm 5.2, the modified algorithm for finding Hamiltonian cycles. We have shown that the algorithm can be applied to find a Hamiltonian cycle with high probability in a graph chosen randomly from $G_{n, p}$, when $p$ is known and sufficiently large, by initially placing edges in the edge lists appropriately. Argue that the algorithm can similarly be applied to find a Hamiltonian cycle with high probability on a graph chosen randomly from $G_{n, N}$ when $N=c_{1} n \ln n$ for a suitably large constant $c_{1}$. Argue also that the modified algorithm can be applied even when $p$ is not known in advance as long as $p$ is at least $c_{2} \ln n / n$ for a suitably large constant $c_{2}$.

Hunza Gilgit
Hunza Gilgit
Numerade Educator