Section 1
Classical Information and Shannon Entropy
Prove that Shannon entropy is always positive or zero.
Create a fourth scenario that is strictly in between the general pdf in Equation (10.9) and the uniform distribution. In other words, determine a pdf for the four-symbol source whose entropy verifies $1.75<H<2$.
Find a pdf for the four-symbol source so that the entropy will be less than 1 but strictly positive.