Question

2 (10 points) Entropy and Conditional Entropy Given a discrete random variable X with possible values \{x_1, \dots, x_n\} and probability mass function P(X), the formula to compute entropy is $$ H(X) = -\sum_{i=1}^n P(X = x_i) \ln P(X = x_i). $$ For Y with possible values \{y_1, \dots, y_m\}, the conditional entropy of X conditioned on random variable Y is defined as $$ H(X|Y) = -\sum_{j=1}^m \sum_{i=1}^n P(X = x_i, Y = y_j) \ln \frac{P(X = x_i, Y = y_j)}{P(Y = y_j)} $$ The joint probability mass function of the random variables X and Y is given by the following table. \begin{tabular}{c|c|c|c} & X = x_1 & X = x_2 & X = x_3 \\ \hline Y = y_1 & 0.2 & 0.1 & 0.1 \\ Y = y_2 & 0.3 & 0.2 & 0.1 \\ \end{tabular} 1. Compute H(X) 2. Compute H(X|Y) 3. Compute H(X|Y = y_1). Note that the entropy of X knowing Y = y_1 is defined as: $H(X|Y = y_1) = -\sum_{i=1}^n P(X = x_i|Y = y_1) \ln P(X = x_i|Y = y_1)$.

          2 (10 points) Entropy and Conditional Entropy
Given a discrete random variable X with possible values \{x_1, \dots, x_n\} and probability
mass function P(X), the formula to compute entropy is
$$
H(X) = -\sum_{i=1}^n P(X = x_i) \ln P(X = x_i).
$$
For Y with possible values \{y_1, \dots, y_m\}, the conditional entropy of X conditioned on
random variable Y is defined as
$$
H(X|Y) = -\sum_{j=1}^m \sum_{i=1}^n P(X = x_i, Y = y_j) \ln \frac{P(X = x_i, Y = y_j)}{P(Y = y_j)}
$$
The joint probability mass function of the random variables X and Y is given by
the following table.
\begin{tabular}{c|c|c|c}
 & X = x_1 & X = x_2 & X = x_3 \\
\hline
Y = y_1 & 0.2 & 0.1 & 0.1 \\
Y = y_2 & 0.3 & 0.2 & 0.1 \\
\end{tabular}

1. Compute H(X)
2. Compute H(X|Y)
3. Compute H(X|Y = y_1). Note that the entropy of X knowing Y = y_1 is defined
as: $H(X|Y = y_1) = -\sum_{i=1}^n P(X = x_i|Y = y_1) \ln P(X = x_i|Y = y_1)$.

$2 (10 points) Entropy and Conditional Entropy Given a discrete random variable X with possible values {x1, …, xn} and probability mass function P(X), the formula to compute entropy is H(X) = -∑i=1^n P(X = xi) ln P(X = xi). For Y with possible values {y1, …, ym}, the conditional entropy of X conditioned on random variable Y is defined as H(X|Y) = -∑j=1^m ∑i=1^n P(X = xi, Y = yj) ln(P(X = xi, Y = yj))/(P(Y = yj)) The joint probability mass function of the random variables X and Y is given by the following table. X = x1 X = x2 X = x3 Y = y1 0.2 0.1 0.1 Y = y2 0.3 0.2 0.1 1. Compute H(X) 2. Compute H(X|Y) 3. Compute H(X|Y = y1). Note that the entropy of X knowing Y = y1 is defined as: H(X|Y = y1) = -∑i=1^n P(X = xi|Y = y1) ln P(X = xi|Y = y1).$

Added by Martin O.

Question

Please give Ace some feedback