Chapter 4, Reliable Communication Through Unreliable Channels Video Solutions, Foundations of coding: theory and applications of error-correcting codes

Problem 1

Suppose that a memoryless binary information source (2.1) has probabilities $p$ (for 0 ) and $q=1-p$ (for 1 ). Let $n$ symbols be emitted.
(1) Prove that the probability that the binary word of length $n$ has 1 on positions $i_1, i_2, \ldots, i_k$ and 0 on the remaining ones is $p^k q^{n-k}$.
(2) Conclude that the probability that the word has 1 on exactly $k$ positions is $\left(\begin{array}{l}n \\ k\end{array}\right) p^k q^{n-k}$.

Check back soon!

Problem 2

Verify the following error estimates for codes in a binary symmetric channel of error probability $p$ :
(1) Prove that the probability that in a word of length $n$, precisely $i$ bits are corrupted is $\left(\begin{array}{l}n \\ i\end{array}\right) p^i q^{n-i}$.
(2) Let $K$ be a binary block code of length $n$ and minimum distance $2 t$ or $2 t+1$. Prove that
$$
P_{\text {ear }}(K) \leq \sum_{i=t+1}^n\left(\begin{array}{c}
n \\
i
\end{array}\right) p^i q^{n-i} .
$$
(3) Verify that the estimate in (2) holds with equality for the repetition codes.

Check back soon!

Problem 3

In a binary symmetric channel of error probability $p=0.1$ :
(1) find the length of a repetition code $K$ with $P_{\text {err }}(K)<10^{-4}$;
(2) find $P_{\text {err }}\left(K_6^*\right)$ for the code $K_6^*$ of 4.3 .

Check back soon!

03:05

Problem 4

Prove that the mutual information $I(X, Y)$ has the following properties:
(1) $I(X, Y) \geq 0$, with equality holding only if the inputs and outputs are stochastically independent.
(2) $I(X, Y) \leq H(S)$, with equality holding only if the outputs completely determine the inputs.

Nick Johnson

Numerade Educator

Problem 5

Let $S$ be an information source at the input of an information channel, and $\bar{S}$ the information source at the output, given by (4.7.2). Prove that
$$
I(X, Y)=H(S)+H(\bar{S})-H(S, \bar{S}),
$$
where $H(S, \bar{S})$ is the entropy of the input-output pairs [with probabilities given in (4.7.1)].

Check back soon!