Predictions of a Generative Multinomial Model
1 point possible (graded)
Consider using a multinomial generative model \( M \) for the task of binary classification consisting of two classes which are denoted by + (positive class) and - (negative class).
Let the parameters of \( M \) that maximize the likelihood of training data for the positive class be denoted by \( \theta^{+} \) and for the negative class be denoted by \( \theta^{-} \).
Also, suppose that we classify a new document \( D \) to belong to the positive class iff
\[
\log \frac{P\left(D \mid \theta^{+}\right)}{P\left(D \mid \theta^{-}\right)} \geq 0
\]
where \( P(D \mid \theta) \) stands for the probability that document \( D \) is generated using a multinomial distribution with parameters \( \theta \).
Which of the following option(s) is/are true about this generative classifier? Choose all that apply from the statements below:
A document is classified as positive iff \( P\left(D \mid \theta^{+}\right) \geq P\left(D \mid \theta^{-}\right) \)
A document is classified as positive iff \( P\left(D \mid \theta^{+}\right)<P\left(D \mid \theta^{-}\right) \)
The generative classifier \( M \) can be shown to be equivalent to a linear classifier given by
\[
\sum_{w \in W}\left(\operatorname{count}(w) \theta_{w}^{\prime}\right) \geq 0 \text { where } \theta_{w}^{\prime}=\log \frac{\theta_{w}^{+}}{\theta_{w}^{-}}
\]
The generative classifier \( M \) can be shown to be equivalent to a linear classifier given by
\[
\sum_{w \in W}\left(\operatorname{count}(w) \theta_{w}^{\prime}\right) \geq 0 \text { where } \theta_{w}^{\prime}=\log \frac{\theta_{w}^{-}}{\theta_{w}^{-}}
\]
