Question

c) As discussed in class, the maximum entropy distribution $P^{(k)}(x_1, x_2, \dots, x_N)$ \\ for $N$ binary variables (take values 0 or 1) constrained by the $k$th ($k \le N$) \\ order marginals has the form: \\ $\exp \left[ \theta_0 + \sum_i \theta_i x_i + \sum_{i<j} \theta_{ij} x_i x_j + \dots + \sum_{i<j<\dots<l} \theta_{ij\dots l} x_i x_j \dots x_l \right],$ \\ 1 \\ where $x_i x_j \dots x_l$ is a product of $k$ factors. Now consider the XOR case as in \\ part b) with $N = 3$ variables where all the $N$th marginals are known, i.e., \\ all numerical values of the full joint probability $P(x_1, x_2, x_3)$ are known. \\ Can you solve for $\theta_i$, $\theta_{ij}$ and $\theta_{ijk}$ in this case? If yes, show their numerical \\ values; If not, explain what is the problem and how to resolve it. (6p)

          c) As discussed in class, the maximum entropy distribution $P^{(k)}(x_1, x_2, \dots, x_N)$ \\
for $N$ binary variables (take values 0 or 1) constrained by the $k$th ($k \le N$) \\
order marginals has the form: \\
$\exp \left[ \theta_0 + \sum_i \theta_i x_i + \sum_{i<j} \theta_{ij} x_i x_j + \dots + \sum_{i<j<\dots<l} \theta_{ij\dots l} x_i x_j \dots x_l \right],$ \\
1 \\
where $x_i x_j \dots x_l$ is a product of $k$ factors. Now consider the XOR case as in \\
part b) with $N = 3$ variables where all the $N$th marginals are known, i.e., \\
all numerical values of the full joint probability $P(x_1, x_2, x_3)$ are known. \\
Can you solve for $\theta_i$, $\theta_{ij}$ and $\theta_{ijk}$ in this case? If yes, show their numerical \\
values; If not, explain what is the problem and how to resolve it. (6p)

c) As discussed in class, the maximum entropy distribution P^(k)(x1, x2, …, xN)

for N binary variables (take values 0 or 1) constrained by the kth (k ≤ N)

order marginals has the form:

[ θ0 + xi + ∑i<jθij xi xj + … + ∑i<j<…<lθij… l xi xj … xl ],

1

where xi xj … xl is a product of k factors. Now consider the XOR case as in

part b) with N = 3 variables where all the Nth marginals are known, i.e.,

all numerical values of the full joint probability P(x1, x2, x3) are known.

Can you solve for , θij and θijk in this case? If yes, show their numerical

values; If not, explain what is the problem and how to resolve it. (6p)

Added by Ricardo A.

Question

Please give Ace some feedback