Question

Q 5 (10 points) A commonly used model of the distribution of terms in a collection is Zipf's law. It is stated as, if $t_1$ is the most common term in the collection, $t_2$ is the next most common, and so on, then the collection frequency $cf_i$, of the $i$th most common term is proportional to $1/i$. That is, $cf_i \propto \frac{1}{i}$ So if the most frequent term occurs $cf_1$ times, then the second most frequent term has half as many occurrences, the third most frequent term a third as many occurrences, and so on. The intuition is that frequency decreases very rapidly with rank. Equivalently, we can write Zipf's law as $cf_i = ci^k$ or as $log\,cf_i = log\,c +$ k $log\,i$ where $k = -1$ and c is a constant (for English, $c = 0.1$). If we ignored all occurrences of the five most frequent terms in the collection, what proportion of term occurrences would be removed from the collection?

          Q 5 (10 points)
A commonly used model of the distribution of terms in a collection is
Zipf's law. It is stated as, if $t_1$ is the most common term in the collection,
$t_2$ is the next most common, and so on, then the collection frequency $cf_i$,
of the $i$th most common term is proportional to $1/i$. That is,
$cf_i \propto \frac{1}{i}$
So if the most frequent term occurs $cf_1$ times, then the second most
frequent term has half as many occurrences, the third most frequent
term a third as many occurrences, and so on. The intuition is that
frequency decreases very rapidly with rank.
Equivalently, we can write Zipf's law as $cf_i = ci^k$ or as $log\,cf_i = log\,c +$
k $log\,i$ where $k = -1$ and c is a constant (for English, $c = 0.1$).
If we ignored all occurrences of the five most frequent terms in the
collection, what proportion of term occurrences would be removed from
the collection?

Q 5 (10 points)
A commonly used model of the distribution of terms in a collection is
Zipf's law. It is stated as, if t1 is the most common term in the collection,
t2 is the next most common, and so on, then the collection frequency cfi,
of the ith most common term is proportional to 1/i. That is,
cfi ∝(1)/(i)
So if the most frequent term occurs cf1 times, then the second most
frequent term has half as many occurrences, the third most frequent
term a third as many occurrences, and so on. The intuition is that
frequency decreases very rapidly with rank.
Equivalently, we can write Zipf's law as cfi = ci^k or as log cfi = log c +
k log i where k = -1 and c is a constant (for English, c = 0.1).
If we ignored all occurrences of the five most frequent terms in the
collection, what proportion of term occurrences would be removed from
the collection?

Added by Eric A.

Question

Please give Ace some feedback