Question

follows the Gaussian mixture distribution with $ p(\mathbf{x})= $ $ \sum_{k=1}^{K} \pi_{k} \mathcal{N}\left(\mathbf{x} \mid \boldsymbol{\mu}_{k}, \boldsymbol{\Sigma}_{k}\right) $. Where $ \boldsymbol{\Theta}=\{\boldsymbol{\pi}, \boldsymbol{\mu}, \boldsymbol{\Sigma}\}, \boldsymbol{\pi}= $ $ \left\{\pi_{1}, \ldots, \pi_{K}\right\}, \boldsymbol{\mu}=\left\{\boldsymbol{\mu}_{1}, \ldots, \boldsymbol{\mu}_{K}\right\}, \boldsymbol{\Sigma}=\left\{\boldsymbol{\Sigma}_{1}, \ldots, \boldsymbol{\Sigma}_{K}\right\} $ and latent variable $ z \sim $ Categorical $ (\boldsymbol{\pi}) $, where $ \boldsymbol{\pi} \geq $ $ 0, \quad \sum_{k} \pi_{k}=1 $ 1. Likelihood Decomposition. Show that \[ \ln p(\mathcal{D} ; \boldsymbol{\Theta}) \geq \mathcal{L}(q ; \boldsymbol{\Theta}), \forall \mathbf{q}, \boldsymbol{\Theta} \] where \[ \mathcal{L}(q ; \boldsymbol{\Theta})=\sum_{n=1}^{N} \mathbb{E}_{q_{n}\left(z^{(n)}\right)}\left[\ln \left(\frac{p\left(\mathbf{x}^{(n)}, z^{(n)} ;\right)}{q_{n}\left(z^{(n)}\right)}\right)\right], \] and $ \mathbf{q}(\mathbf{z})=\prod_{n=1}^{N} q_{n}\left(z^{(n)}\right) $. And, write down the gap between $ \ln p(\mathcal{D} ; \boldsymbol{\Theta}) $ and $ \mathcal{L}(\mathbf{q} ; \boldsymbol{\Theta}) $. (6 points) 2. E-Step Derivation. With given $ \boldsymbol{\Theta}=\{\boldsymbol{\pi}, \boldsymbol{\mu}, \boldsymbol{\Sigma}\} $, update $ \mathbf{q}(\mathbf{z}) .(3 $ points $ ) $

          follows the Gaussian mixture distribution with \( p(\mathbf{x})= \) \( \sum_{k=1}^{K} \pi_{k} \mathcal{N}\left(\mathbf{x} \mid \boldsymbol{\mu}_{k}, \boldsymbol{\Sigma}_{k}\right) \). Where \( \boldsymbol{\Theta}=\{\boldsymbol{\pi}, \boldsymbol{\mu}, \boldsymbol{\Sigma}\}, \boldsymbol{\pi}= \) \( \left\{\pi_{1}, \ldots, \pi_{K}\right\}, \boldsymbol{\mu}=\left\{\boldsymbol{\mu}_{1}, \ldots, \boldsymbol{\mu}_{K}\right\}, \boldsymbol{\Sigma}=\left\{\boldsymbol{\Sigma}_{1}, \ldots, \boldsymbol{\Sigma}_{K}\right\} \) and latent variable \( z \sim \) Categorical \( (\boldsymbol{\pi}) \), where \( \boldsymbol{\pi} \geq \) \( 0, \quad \sum_{k} \pi_{k}=1 \)
1. Likelihood Decomposition. Show that
\[
\ln p(\mathcal{D} ; \boldsymbol{\Theta}) \geq \mathcal{L}(q ; \boldsymbol{\Theta}), \forall \mathbf{q}, \boldsymbol{\Theta}
\]
where
\[
\mathcal{L}(q ; \boldsymbol{\Theta})=\sum_{n=1}^{N} \mathbb{E}_{q_{n}\left(z^{(n)}\right)}\left[\ln \left(\frac{p\left(\mathbf{x}^{(n)}, z^{(n)} ;\right)}{q_{n}\left(z^{(n)}\right)}\right)\right],
\]
and \( \mathbf{q}(\mathbf{z})=\prod_{n=1}^{N} q_{n}\left(z^{(n)}\right) \). And, write down the gap between \( \ln p(\mathcal{D} ; \boldsymbol{\Theta}) \) and \( \mathcal{L}(\mathbf{q} ; \boldsymbol{\Theta}) \). (6 points)
2. E-Step Derivation. With given \( \boldsymbol{\Theta}=\{\boldsymbol{\pi}, \boldsymbol{\mu}, \boldsymbol{\Sigma}\} \), update \( \mathbf{q}(\mathbf{z}) .(3 \) points \( ) \)

$follows the Gaussian mixture distribution with p(𝐱)= ∑k=1^Kπk𝒩(𝐱|μk, Σk). Where Θ={π, μ, Σ}, π= {π1, …, πK}, μ={μ1, …, μK}, Σ={Σ1, …, ΣK} and latent variable z ∼ Categorical (π), where π≥ 0, ∑kπk=1 1. Likelihood Decomposition. Show that ln p(𝒟 ; Θ) ≥ℒ(q ; Θ), ∀𝐪, Θ where ℒ(q ; Θ)=∑n=1^N𝔼qn(z^(n))[ln((p(𝐱^(n), z^(n) ;))/(qn(z^(n))))], and 𝐪(𝐳)=∏n=1^N qn(z^(n)). And, write down the gap between ln p(𝒟 ; Θ) and ℒ(𝐪 ; Θ). (6 points) 2. E-Step Derivation. With given Θ={π, μ, Σ}, update 𝐪(𝐳) .(3 points )$

Added by James S.

Question

Please give Ace some feedback