(a) (5 points) Imagine that a system A? has probability $P_r^{(1)}$ of being found in a state $r$ and a system A? has probability $P_s^{(2)}$ of being found in a state $s$. Then one has
$S_1 = -k_B \sum_r P_r^{(1)} \ln P_r^{(1)}$ and $S_2 = -k_B \sum_r P_s^{(2)} \ln P_s^{(2)}$.
Each state of the composite system A consisting of A? and A? can then be labeled by the pair of numbers $r$ and $s$; let the probability of A being found in this state be denoted by $P_{rs}$. Then its entropy is defined by:
$S = -k_B \sum_r \sum_s P_{rs} \ln P_{rs}$ (5)
If A? and A? are weakly interacting so that they are statistically independent, then $P_{rs} = P_r^{(1)} P_s^{(2)}$.
Show that under these circumstances the entropy is simply additive, i.e., $S = S_1 + S_2$.
(Hint: Recall the normalisation condition you explored in the Assignment 2 so that $\sum_r P_r^{(1)} = \sum_s P_s^{(2)} = 1$.)
(b) (8 points) Following the previous question, assume that A? and A? are not weakly interacting so that $P_{rs} \neq P_r^{(1)} P_s^{(2)}$. One has, of course, the general relations:
$P_r^{(1)} = \sum_s P_{rs}$ and $P_s^{(2)} = \sum_r P_{rs}$.
Furthermore, all the probabilities are properly normalised so that
$\sum_r P_r^{(1)} = \sum_s P_s^{(2)} = \sum_r \sum_s P_{rs} = 1$
Show that
$S - (S_1 + S_2) = k_B \sum_r \sum_s P_{rs} \ln \frac{P_{rs}}{P_r^{(1)} P_s^{(2)}}$ (6)
(c) (7 points) By using the inequality $-\ln x \ge -x + 1$, show that
$S \le S_1 + S_2$ (7)
where the equals sign holds only if $P_{rs} = P_r^{(1)} P_s^{(2)}$ for all $r$ and $s$. This means that the existence of correlations between the systems, due to the interaction between them, leads to a situation less random than that where the systems are completely independent of each other.
