ame: Al-Zahraa Salim
Sig.
ID: 13287073
1. a. Define, explain the importance and derive the Gibbs inequality.
b. Let X and Y be two random variables with $p(X) = [\frac{1}{6}, \frac{1}{6}, \frac{1}{6}, \frac{1}{6}, \frac{1}{6}, \frac{1}{6}]$ and $q(X) = [\frac{i}{21}, i = 1, 2, 3, 4, 5, 6]$. Find the value of
Gibbs inequality. Interpret your results.
2. a. Discuss the relation between the self-information, entropy and conditional
entropy for the case of two random variables.
a. Prove and explain the following:
$H(X, Y, Z) = H(X) + H(Y|X) + H(Z|XY)$.