follows the Gaussian mixture distribution with \( p(\mathbf{x})= \) \( \sum_{k=1}^{K} \pi_{k} \mathcal{N}\left(\mathbf{x} \mid \boldsymbol{\mu}_{k}, \boldsymbol{\Sigma}_{k}\right) \). Where \( \boldsymbol{\Theta}=\{\boldsymbol{\pi}, \boldsymbol{\mu}, \boldsymbol{\Sigma}\}, \boldsymbol{\pi}= \) \( \left\{\pi_{1}, \ldots, \pi_{K}\right\}, \boldsymbol{\mu}=\left\{\boldsymbol{\mu}_{1}, \ldots, \boldsymbol{\mu}_{K}\right\}, \boldsymbol{\Sigma}=\left\{\boldsymbol{\Sigma}_{1}, \ldots, \boldsymbol{\Sigma}_{K}\right\} \) and latent variable \( z \sim \) Categorical \( (\boldsymbol{\pi}) \), where \( \boldsymbol{\pi} \geq \) \( 0, \quad \sum_{k} \pi_{k}=1 \)
1. Likelihood Decomposition. Show that
\[
\ln p(\mathcal{D} ; \boldsymbol{\Theta}) \geq \mathcal{L}(q ; \boldsymbol{\Theta}), \forall \mathbf{q}, \boldsymbol{\Theta}
\]
where
\[
\mathcal{L}(q ; \boldsymbol{\Theta})=\sum_{n=1}^{N} \mathbb{E}_{q_{n}\left(z^{(n)}\right)}\left[\ln \left(\frac{p\left(\mathbf{x}^{(n)}, z^{(n)} ;\right)}{q_{n}\left(z^{(n)}\right)}\right)\right],
\]
and \( \mathbf{q}(\mathbf{z})=\prod_{n=1}^{N} q_{n}\left(z^{(n)}\right) \). And, write down the gap between \( \ln p(\mathcal{D} ; \boldsymbol{\Theta}) \) and \( \mathcal{L}(\mathbf{q} ; \boldsymbol{\Theta}) \). (6 points)
2. E-Step Derivation. With given \( \boldsymbol{\Theta}=\{\boldsymbol{\pi}, \boldsymbol{\mu}, \boldsymbol{\Sigma}\} \), update \( \mathbf{q}(\mathbf{z}) .(3 \) points \( ) \)
