Suppose that several information sources generate symbols at random from a five-letter alphabet {A,
B, C, D, E} with different probabilities. We will try different encoding schemes for encoding these
symbols in binary. Given the probabilities and an encoding scheme, you will compute the
expected length of the encoding of n letters generated by the information source. You may use
any rigorous method of analysis you like, but must show your work and justify your answer.
(a) Suppose that the symbols occur with probabilities Pr[A]=0.4, Pr[B] = 0.2, Pr[C] = 0.2, Pr[D]
= 0.1, and Pr[E] = 0.1, and the coding scheme encodes these symbols into binary codes as
follows:
A
001
B
010
C
011
D
100
E
101
What is the expected length of the encoding for these probabilities and encoding?