c) As discussed in class, the maximum entropy distribution $P^{(k)}(x_1, x_2, \dots, x_N)$ \\
for $N$ binary variables (take values 0 or 1) constrained by the $k$th ($k \le N$) \\
order marginals has the form: \\
$\exp \left[ \theta_0 + \sum_i \theta_i x_i + \sum_{i<j} \theta_{ij} x_i x_j + \dots + \sum_{i<j<\dots<l} \theta_{ij\dots l} x_i x_j \dots x_l \right],$ \\
1 \\
where $x_i x_j \dots x_l$ is a product of $k$ factors. Now consider the XOR case as in \\
part b) with $N = 3$ variables where all the $N$th marginals are known, i.e., \\
all numerical values of the full joint probability $P(x_1, x_2, x_3)$ are known. \\
Can you solve for $\theta_i$, $\theta_{ij}$ and $\theta_{ijk}$ in this case? If yes, show their numerical \\
values; If not, explain what is the problem and how to resolve it. (6p)
