Question 2 (20 marks)
(a) Prove that $I(x_i, x_j) = I(x_i) + I(x_j)$, if $x_i$ and $x_j$ are independent. [4 marks]
(b) Table 1 shows the discrete memoryless system with symbols $x_i$, $i = 1, 2, 3, 4$.
Show that all codes except code B satisfy Kraft's inequality.
[Hint: Kraft's inequality $\sum_{i=1}^{N} 2^{-l_i} \le 1$]
\begin{tabular}{|c|c|c|c|c|}
\hline
$x_i$ & Code A & Code B & Code C & Code D \\
\hline
$x_1$ & 00 & 0 & 0 & 0 \\
$x_2$ & 01 & 10 & 11 & 100 \\
$x_3$ & 10 & 11 & 100 & 110 \\
$x_4$ & 11 & 110 & 110 & 111 \\
\hline
\end{tabular}