(1) If $y = \sigma(z)$, show that $\frac{dy}{dz} = \sigma(z)(1 - \sigma(z)) = y(1 - y)$.
(2) Consider a network with 2 input nodes, one hidden layer with 2 nodes, and 2 output
nodes. The weights and the bias values are given by $W_1$, $W_2$, $b_1$ and $b_2$:
$\begin{bmatrix} 1 & -1 \\ -3 & -2 \end{bmatrix}$
$W_1 =$
$W_2 = \begin{bmatrix} -2 & 0 \\ 0 & 3 \end{bmatrix}$
$b_1 = (0, 1)$ $b_2 = (1, -2)$.
The activation function at each node is the sigmoid function.
Suppose we want to train the network using input $x = (1, -2)$ with target $t = (1, 0)$.
(a) Calculate the output vector $y = (y_1, y_2)$ for the input $x$.
(b) Compute the sum-of-squares loss for the input $x$.
(c) Do the weight updates for each edge weight and bias according to the NEURAL
NETWORK TRAINING ALGORITHM with sum-of-squares loss and sigmoid activation
functions.
(d) After the updates, feed $x$ into the updated network to get the output.
(e) Compute the sum-of-squares loss for the input $x$ and compare with (b) (it should
have decreased).
