2. The entropy of a discrete random variable X is defined as (use base e for all log operations unless specified otherwise):
$H(X) = - \sum_{x \in X} P(x) \log P(x)$
(a) Compute the entropy of the distribution $P(x) = \text{Multinoulli}([0.2, 0.3, 0.5])$. [3 pts]
(b) Compute the entropy of the uniform distribution $P(x) = \frac{1}{m} \forall x \in [1, m]$. [3 pts]
(c) Consider the entropy of the joint distribution P(X, Y):
$H(X, Y) = - \sum_{x \in X} \sum_{y \in Y} P(x, y) \log P(x, y)$
How does this entropy relate to H(X) and H(Y), (i.e. the entropies of the marginal distributions) when X and Y are independent? [4 pts]