Chapter 13, Pairwise Independence and Universal Hash Functions Video Solutions, Probability and Computing

Problem 1

A fair coin is flipped $n$ times. Let $X_{i j}$, with $1 \leq i<j \leq n$, be 1 if the $i$ th and $j$ th flip landed on the same side; let $X_{i j}=0$ otherwise. Show that the $X_{i j}$ are pairwise independent but not independent.

Bryan Lynn

Numerade Educator

03:51

Problem 2

(a) Let $X$ and $Y$ be numbers that are chosen independently and uniformly at random from $\{0,1, \ldots, n\}$. Let $Z$ be their sum modulo $n+1$. Show that $X, Y$, and $Z$ are pairwise independent but not independent.
(b) Extend this example to give a collection of random variables that are $k$-wise independent but not $(k+1)$-wise independent.

James Kiss

Numerade Educator

04:33

Problem 3

For any family of hash functions from a finite set $U$ to a finite set $V$, show that, when $h$ is chosen at random from that family of hash functions, there exists a pair of elements $x$ and $y$ such that
$$
\operatorname{Pr}(h(x)=h(y)) \geq \frac{1}{|V|}-\frac{1}{|U|} .
$$
This result should not depend on how the function $h$ is chosen from the family.

Amany Waheeb

Numerade Educator

05:11

Problem 4

Show that, for any discrete random variable $X$ that takes on values in the range $[0,1], \operatorname{Var}[X] \leq 1 / 4$ .

Amany Waheeb

Numerade Educator

04:39

Problem 5

Suppose we have a randomized algorithm Test for testing whether a string appears in a language $L$ that works as follows. Given an input $x$, the algorithm Test chooses a random integer $r$ uniformly from the set $S=\{0,1, \ldots, p-1\}$ for some prime $p$. If $x$ is in the language, then $\operatorname{Test}(x, r)=1$ for at least half of the possible values of $r$. A value of $r$ such that Test $(x, r)=1$ is called a witness for $x$. If $x$ is not in the language, then Test $(x, r)=0$ always.

If we run the algorithm Test twice on an input $x \in L$ by choosing two numbers $r_{1}$ and $r_{2}$ independently and uniformly from $S$ and evaluating Test $\left(x, r_{1}\right)$ and Test $\left(x, r_{2}\right)$, then we find a witness with probability at least $3 / 4$. Argue that we can obtain a witness with probability at least $1-1 / t$ using the same amount of randomness by letting $s_{i}=$ $r_{1} i+r_{2} \bmod p$ and evaluating Test $\left(x, s_{i}\right)$ for values $0 \leq i \leq t<p .$

Chris Trentman

Numerade Educator

View

Problem 6

Our analysis of Bucket sort in Section 5.2.2 assumed that $n$ elements were chosen independently and uniformly at random from the range $\left[0,2^{k}\right) .$ Suppose instead that $n$ elements are chosen uniformly from the range $\left[0,2^{k}\right)$ in such a way that they are only pairwise independent. Show that, under these conditions, Bucket sort still requires linear expected time.

Victor Salazar

Numerade Educator

02:14

Problem 7

(a) We have shown that the maximum load when $n$ items are hashed into $n$ bins using a hash function chosen from a 2-universal family of hash functions is at most $\sqrt{2 n}$ with probability at least $1 / 2$. Generalize this argument to $k$-universal hash functions. That is, find a value such that the probability that the maximum load is larger than that value is at most $1 / 2$.
(b) In Lemma $5.1$ we showed that, under the standard balls-and-bins model, the maximum load when $n$ balls are thrown independently and uniformly at random into $n$ bins is at most $3 \ln n / \ln \ln n$ with probability $1-1 / n$. Find the smallest value of $k$ such that the maximum load is at most $3 \ln n / \ln \ln n$ with probability at least $1 / 2$ when choosing a hash function from a $k$-universal family.

Hunza Gilgit

Numerade Educator

04:15

Problem 8

We can generalize the problem of finding a large cut to finding a large $k$-cut. A $k$-cut is a partition of the vertices into $k$ disjoint sets, and the value of a cut is the weight of all edges crossing from one of the $k$ sets to another. In Section 13.1.2 we considered 2-cuts when all edges had the same weight 1, and we showed how to derandomize the standard randomized algorithm using collections of $n$ pairwise independent bits. Explain how this derandomization could be generalized to obtain a polynomial time algorithm for 3 -cuts, and give the running time for your algorithm. (Hint: You may want to use a hash function of the type found in Section 13.3.2.)

Chris Trentman

Numerade Educator

07:31

Problem 9

Suppose we are given $m$ vectors $\bar{v}_{1}, \bar{v}_{2}, \ldots, \bar{v}_{m} \in\{0,1\}^{\ell}$ such that any $k$ of the $m$ vectors are linearly independent modulo 2 . Let $\bar{v}_{i}=\left(v_{i, 1}, v_{i, 2}, \ldots, v_{i, \ell}\right)$. Let $\bar{u}$ be chosen uniformly at random from $\{0,1\}^{\ell}$, and let $X_{l}=\sum_{J=1}^{\ell} v_{i, j} u_{j} \bmod 2 .$ Show that the $X_{i}$ are uniform, $k$-wise independent bits.

Ameer Said

Numerade Educator

03:42

Problem 10

We examine a specific way in which 2-universal hash functions differ from completely random hash functions. Let $S=\{0,1,2, \ldots, k\}$, and consider a hash function $h$ with range $\{0,1,2, \ldots, p-1\}$ for some prime $p$ much larger than $k$ : Consider the values $h(0), h(1), \ldots, h(k) .$ If $h$ is a completely random hash function; then the probability that $h(0)$ is smaller than any of the other values is roughly $1 /(k+1)$. (There may be a tie for the smallest value, so the probability that any $h(i)$ is the unique uniformly from the family
$$
\mathcal{H}=\left\{h_{a, b} \mid 0 \leq a, b \leq p-1\right\}
$$
of Section 13.3.2. Estimate the probability that $h(0)$ is smaller than $h(1), \ldots, h(k)$ by randomly choosing 10,000 hash functions from $h$ and computing $h(x)$ for all $x \in S$. Run this experiment for $k=32$ and $k=128$, using primes $p=5,023,309$ and $p=$ $10,570,849$. Is your estimate close to $1 /(k+1)$ ?

Hunza Gilgit

Numerade Educator

06:25

Problem 11

In a multi-set, each element can appear multiple times. Suppose that we have two multi-sets, $S_{1}$ and $S_{2}$, consisting of positive integers. We want to test if the two sets are the "same" - that is, if each item appears the same number of times in each set. One way of doing this is to sort both sets and then compare the sets in sorted order. This takes $O(n \log n)$ time if each multi-set contains $n$ elements.
(a) Consider the following algorithm. Hash each element of $S_{1}$ into a hash table with $c n$ counters; the counters are initially 0 , and the $i$ th counter is incremented each time the hash value of an element is $i$. Using another table of the same size and using the same hash function, do the same for $S_{2}$. If the $i$ th counter in the first table matches the $i$ th counter in the second table for all $i$, report that the sets are the same, and otherwise report that the sets are different.

Analyze the running time and error probability of this algorithm, assuming that the hash function is chosen from a 2-universal family. Explain how this algorithm can be extended to a Monte Carlo algorithm, and analyze the trade-off between its running time and its error probability.
(b) We can also design a Las Vegas algorithm for this problem. Now each entry in the hash table corresponds to a linked list of counters. Each entry holds a list of the number of occurrences of each element that hashes to that location; this list can be kept in sorted order. Again, we create a hash table for $S_{1}$ and a hash table for $S_{2}$, and we test after hashing if the resulting tables are equal.

Argue that this algorithm requires only linear expected time using only linear space.

Robin Corrigan

Numerade Educator

03:23

Problem 12

In Section 13.3.1 we showed that the family
$$
\mathcal{H}=\left\{h_{a, b} \mid 1 \leq a \leq p-1,0 \leq b \leq p\right\}
$$
is 2-ùniversal when $p \geq n$, where
$$
h_{a, b}(x)=((a x+b) \bmod p) \bmod n .
$$
Consider now the hash functions
and the family
$$
h_{a}(x)=(a x \bmod p) \bmod n
$$
Give an example to show that $\mathcal{H}^{\prime}$ is mot $2-\mathrm{diniversal}$. The
Give an example to show that $\mathcal{H}^{\prime}$ is not 2 -aniversal. Then prove that $\mathcal{H}^{\prime}$ is almost 2 -universal in the following sense: for any $x, y \in\{0,1,2, \ldots, p-1\}$, if $h$ is chosen uniformly at random from $\mathcal{H}^{\prime}$ then
$$
\operatorname{Pr}(h(x)=h(y)) \leq \frac{2}{n}
$$

Sagar Singh

Numerade Educator

02:22

Problem 13

In describing count-min filters, we assumed that the data stream consisted of pairs of the form $\left(i_{t}, c_{t}\right)$, where $i_{t}$ was an item and $c_{t}>0$ an integer count increment. Suppose that one were also allowed to decrement the count for an item, so that the stream could include pairs of the form $\left(i_{t}, c_{t}\right)$ with $c_{t}<0$. We could require that the total count for an item $i$,
$$
\operatorname{Count}(i, T)=\sum_{t: i_{t}=i, 1 \leq t \leq T} c_{t},
$$
always be positive.
Explain how you could modify or otherwise use count-min filters to find heavy hitters in this situation.

Bryan Lynn

Numerade Educator