1.
Obtain the derivative, $\frac{\partial \sigma}{\partial x}$, for $\sigma(x) = \frac{1}{1 + e^{-x}}$.
2.
Obtain the derivative, $\frac{\partial \sigma}{\partial x}$, for $\sigma(x) = \tanh(x)$.
3.
For $x = \{x_1, x_2, ..., x_k\}$ and $\sigma(x) = \text{softmax} (x) = \begin{bmatrix} s_1\\s_2\\.\\.\\s_k \end{bmatrix}$, where $s_i = \frac{e^{x_i}}{\sum_k e^{x_k}}$, obtain the
derivative, $\frac{\partial s_i}{\partial x_j}$.
4.
Obtain the derivative, $\frac{\partial L}{\partial x_i}$, for $L = -\sum_j y_j \ln s_j$, where $s_i = \frac{e^{x_i}}{\sum_k e^{x_k}}$ and $\sum_j y_j = 1$.
