1. Consider the following MLP
W^(1)T
X
where
W^(2)T
f^(1)
b^(1)
f^(2)
b^(2)
a^(2)
a^(3)
W^(1) = \begin{bmatrix} 0 & 1 & 0 \\ -1 & 2 & 1 \end{bmatrix}
W^(2) = \begin{bmatrix} -1 & 2 & -1 \\ 1 & 2 & 0 \end{bmatrix}
b^(1) = \begin{bmatrix} 0 \\ -1 \\ 0 \end{bmatrix}, b^(2) = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix},
Also assume that the components of $f^(1)$, and $f^(2)$ are linear, i.e. is $f_1(x) = f_2(x) = x$.
The second layer is followed by a softmax layer, and the output of each neuron in the
softmax layer is calculated as:
a_j^(3) = \frac{e^{a_j^(2)}}{e^{a_1^(2)} + e^{a_2^(2)} + e^{a_3^(2)}}
where $a_j^(3)$ is the output of $j^{th}$ neuron in the third layer and $a_j^(2)$ is the output of the
$j^{th}$ neuron in the second layer.
Answer the following questions:
(a) How many neurons are there in the first, and second layers, respectively?
(b) How many classes ($K$) does the classification problem performed by this network
involve?
(c) Explain why this network is exactly equivalent to performing multinomial re-
gression using linear logits and write down $p_k(x) = Pr(Y = k|X = x)$ for
k = 1,..., K.
