• Home
  • Textbooks
  • Probability and Computing
  • Balanced Allocations

Probability and Computing

Michael Mitzenmacher , Eli Upfal

Chapter 14

Balanced Allocations - all with Video Answers

Educators


Chapter Questions

02:47

Problem 1

(a) For Theorems $14.1$ and $14.4$, the statement of the proof is for the case that ties are broken randomly. Argue informally that, if the bins are numbered from 1 to $n$ and if ties are broken in favor of the lower-numbered bin, then the theorems still bold.
(b) Argue informally that the theorems apply to any tie-breaking mechanism that has no knowledge of the bin choices made by balls that have not yet been placed.

Nick Johnson
Nick Johnson
Numerade Educator
05:46

Problem 2

Consider the following variant of the balanced allocation paradigm: $n$ balls are placed sequentially in $n$ bins, with the bins labeled from 0 to $n-1$. Each ball chooses a bin $i$ uniformly at random, and the ball is placed in the least loaded of bins $i, i+1 \bmod n, i+2 \bmod n, \ldots, i+d-1 \bmod n$. Argue that, when $d$ is a constant, the maximum load grows as $\Theta(\ln n / \ln \ln n)$. That is, the balanced allocation paradigm does not yield an $O(\ln \ln n)$ result in this case.

Mengchun Cai
Mengchun Cai
Numerade Educator
03:38

Problem 3

Explain why, with 2-way chaining, the expected time to insert an item and to search for an item in a hash table of size $n$ with $n$ items is $O(1)$. Consider two cases: the search is for an item that is in the table; and the search is for an item that is not in the table.

Eric Mockensturm
Eric Mockensturm
Numerade Educator
02:07

Problem 4

Consider the following variant of the balanced allocation paradigm: $n$ balls are placed sequentially in $n$ bins. Each ball comes with $d$ choices, chosen independently and uniformly at random from the $n$ bins. When a ball is placed, we are also allowed to move balls among these $d$ bins to equalize their load as much as possible. Show that the maximum load is still at least $\ln \ln n / \ln d-O(1)$ with probability $1-o(1 / n)$ in this case.

Nick Johnson
Nick Johnson
Numerade Educator
02:37

Problem 5

Suppose that in the balanced allocation setup there are $n$ bins, but the bins are not chosen uniformly at random. Instead, the bins have two types: $1 / 3$ of the bins are type $A$ and $2 / 3$ of the bins are type $B$. When a bin is chosen at random, each of the type-A bins is chosen with probability $2 / n$ and each of the type-B bins is chosen with probability $1 / 2 n$. Prove that the maximum load of any bin when each ball has $d$ bin choices is still $\ln \ln n / \ln d+O$ (1).

Nick Johnson
Nick Johnson
Numerade Educator
00:50

Problem 6

Consider a parallel version of the balanced allocation paradigm in which we have $n / k$ rounds, where $k$ new balls arrive in each round. Each ball is placed in the least loaded of its $d$ choices, where in this setting the load of each bin is the load at the end of the previous round. Ties are broken randomly. Note that the $k$ new balls cannot affect each other's placement. Give an upper bound on the maximum load as a function of $n, d$, and $k$.

Hunza Gilgit
Hunza Gilgit
Numerade Educator
02:14

Problem 7

We have shown that sequentially throwing $n$ balls into $n$ bins randomly, using two bin choices for each ball, yields a maximum load of $\ln \ln n / \ln 2+O$ (1) with high probability. Suppose that, instead of placing the balls sequentially, we had access to all of the $2 n$ choices for the $n$ balls, and suppose we wanted to place each ball into one of its choices while minimizing the maximum load. In this setting, with high probability, we can obtain a maximum load that is constant.

Write a program to explore this scenario. Your program should take as input a parameter $k$ and implement the following greedy algorithm. At each step, some subset of the balls are active; initially, all balls are active. Repeatedly find a bin that has at least one but no more than $k$ active balls that have chosen it, assign these active balls to that bin; and then remove these balls from the set of active balls. The process stops either when there are no active balls remaining or when there is no suitable bin. If the algorithm stops with no active balls remaining, then every bin is assigned no more than $k$ balls.
Try running your program with 10,000 balls and 10,000 bins. What is the smallest value of $k$ for which the program terminates with no active balls remaining at least four out of five times? If your program is fast enough, try experimenting with more trials. Also, if your program is fast enough, try answering the same question for 100,000 balls and 100,000 bins.

Hunza Gilgit
Hunza Gilgit
Numerade Educator
02:14

Problem 8

The following problem models a simple distributed system where agents contend for resources and back off in the face of contention. As in Exercise 5.11, balls represent agents and bins represent resources.

The system evolves over rounds. In the first part of every round, balls are thrown independently and uniformly at random into $n$ bins. In the second part of each round, each bin in which at least one ball has landed in that round serves exactly one ball from that round. The remaining balls are thrown again in the next round. We begin with $n$ balls in the first round, and we finish when every ball is served.
(a) Show that, with probability $1-o(1 / n)$, this approach takes at most $\log _{2} \log _{2} n+$ $O$ (1) rounds. (Hint: Let $b_{k}$ be the number of balls left after $k$ rounds; show that $b_{k+1} \leq c\left(b_{k}\right)^{2} / n$, for a suitable constant $c$ with high probability, as long as $b_{k+1}$ is sufficiently large.)
(b) Suppose that we modify the system so that a bin accepts a ball in a round if and only if that ball was the only ball to request that bin in that round. Show that, again with probability $1-o(1 / n)$, this approach takes at most $\log _{2} \log _{2} n+O(1)$ rounds.

Hunza Gilgit
Hunza Gilgit
Numerade Educator
01:15

Problem 9

The natural way to simulate experiments with balls and bins is to create an array that stores the load at each bin. To simulate $1,000,000$ balls being placed into $1,000,000$ bins would require an array of $1,000,000$ counters. An alternative approach is to keep an array that records in the $j$ th cell the number of bins with load $j$. Explain how this could be used to simulate placing $1,000,000$ balls into $1,000,000$ bins using the standard balls-and-bins paradigm and the balanced allocation paradigm with much less space.

Trinity Steen
Trinity Steen
Numerade Educator
View

Problem 10

Write a program to compare the performance of the standard ballsand-bins paradigm and the balanced allocation paradigm. Run simulations placing $n$ balls into $n$ bins, with each ball having $d=1,2,3$, and 4 random choices. You should try $n=10,000, n=100,000$, and $n=1,000,000$. Repeat each experiment at least 100 times and compute the expectation and variance of the maximum load for each value of $d$ based on your trials. You may wish to use the idea of Exercise $14.9 .$

Shu Naito
Shu Naito
Numerade Educator
07:47

Problem 11

Write a simulation showing how the balanced allocation paradigm can improve performance for distributed queueing systems. Consider a bank of $n$ FIFO queues with a Poisson arrival stream of customers to the entire bank of rate $\lambda n$ per second, where $\lambda<1$. Upon entry a customer chooses a queue for service, and the service time for each customer is exponentially distributed with mean 1 second. You should compare two settings: (i) where each customer chooses a queue independently and uniformly at random from the $n$ queues for service; and (ii) where each customer chooses two queues independently and uniformly at random from the $n$ queues and waits at the queue with fewer customers, breaking ties randomly. Notice that the first setting is equivalent to having a bank of $n M / M / 1$ FIFO queues, each with Poisson arrivals of rate $\lambda<1$ per second. You may find the discussion in Exercise $8.26$ helpful in constructing your simulation.

Your simulation should run for $t$ seconds, and it should return the average (over all customers that have completed service) of the time spent in the system as well as the average (over all customers that have arrived) of the number of customers found waiting in the queue they selected for service. You should present results for your simulations for $n=100$ and for $t=10,000$ seconds, with $\lambda=0.5,0.8,0.9$, and $0.99$.

Susan Schaub
Susan Schaub
Numerade Educator
05:10

Problem 12

Write a program to compare the performance of the following variation of the standard balls-and-bins paradigm and the balanced allocation paradigm. Initially $n$ points are placed uniformly at random on the boundary of a circle of circumference 1. These $n$ points divide the circle into $n$ arcs, which correspond to bins. We now place $n$ balls into the bins as follows: each ball chooses $d$ points on the boundary of the circle, uniformly at random. These $d$ points correspond to the arcs (or, equivalently, bins) that they lie on. The ball is placed in the least loaded of the $d$ bins, breaking ties in favor of the smallest arc.
Run simulations placing $n$ balls into $n$ bins for the cases $d=1$ and $d=2$. You should try $n=1,000, n=10,000$, and $n=100,000$. Repeat each experiment at least 100 times; for each run, the $n$ initial points should be re-chosen. Give a chart showing the number of times the maximum load was $k$, based on your trials for each value of $d$.
You may note that some arcs are much larger than others, and therefore when $d \ddot$ 1 the maximum load can be rather high. Also, to find which bin each ball is placed in may require implementing a binary search or some other additional data structure to quickly map points on the circle boundary to the appropriate bin.

Morgan Cheatham
Morgan Cheatham
Numerade Educator
02:46

Problem 13

There is a small but interesting improvement that can be made to the balanced allocation scheme we have described. Again we will place $n$ balls into $n$ bins; We assume here than $n$ is even. Suppose that we divide the $n$ bins into two groups of size $n / 2$. We call the two groups the left group and the right group. For each ball, we independently choose one bin uniformly at random from the left and one bin uniformly at random from the right. We put the ball in the least loaded bin, but if there is a tie we always put the ball in the bin from the left group. With this scheme, the maximum load is reduced to $\ln \ln n / 2 \ln \phi+O(1)$, where $\phi=(1+\sqrt{5}) / 2$ is the golden ratio. This improves the result of Theorem $14.1$ by a constant factor. (Note the two changes to otr original scheme: the bins are split into two groups, and ties are broken in a consistent way; both changes are necessary to obtain the improvement we describe.)
(a) Write a program to compare the performance of the original balanced allocation paradigm with this variation. Run simulations placing $n$ balls into $n$ bins, with each ball having $d=2$ choices. You should try $n=10,000, n=100,000$, and $n=1,000,000$. Repeat each experiment at least 100 times and compute the expectation and variance of the maximum load based on your trials. Describe the extent of the improvement of the new variation.
(b) Adapt Theorem $14.1$ to prove this result. The key idea in how the theorem's proof must change is that we now require two sequences, $\beta_{i}$ and $\gamma_{i}$. Similar to Theorem $14.1, \beta_{i}$ represents a desired upper bound on the number of bins on the left with load at least $i$, and $\gamma_{2}$ is a desired upper bound on the number of bins on the right with load at least $i$. Argue that choosing
$$
\beta_{i+1}=\frac{c_{1} \beta_{t} \gamma_{i}}{n^{2}} \quad \text { and } \quad \gamma_{i+1}=\frac{c_{2} \beta_{i+1} \gamma_{i}}{n^{2}}
$$
for some constants $c_{1}$ and $c_{2}$ is suitable (as long as $\beta_{i}$ and $\gamma_{t}$ are large enough that Chernoff bounds may apply).

Now let $F_{k}$ be the $k$ th Fibonacci number. Apply induction to show that, for sufficiently large $i, \beta_{i} \leq n c_{3} c_{4}^{F_{2 t}}$ and $\gamma_{i} \leq n c_{3} c_{4}^{F_{2 i+1}}$ for some constants $c_{3}$ and $c_{4}$. Following Theorem 14.1, use this to prove the $\ln \ln n / 2 \ln \phi+O(1)$ upper bound.
(c) This variation can easily be extended to the case of $d>2$ choices by splitting the $n$ bins into $d$ ordered groups, choosing one bin uniformly at random from each group, and breaking ties in favor of the group that comes first in the ordering. Suggest what would be the appropriate upper bound on the maximum load for this case, and give an argument backing your suggestion. (You need not give a complete formal proof.)

Hossam Mohamed
Hossam Mohamed
Numerade Educator