provide code in python numpy
2. Optimization in Neural Networks using Steepest Descent Consider a simple neural network with a single hidden layer for binary classification. The network has three parameters to optimize: weights (w.w2.w3) and biases b).The objective is to minimize the binary cross-entropy loss function Lw,w2,w3,b with respect to these parameters. The binary cross-entropy loss function is given by:
N
LW1,W2,W3,b=-
i=1
1 wherez=wx+wx+wx+b,z= N is the number of samples,yis 1+e the true label,and log represents the natural logarithm.
Page 2/3
a Briefly explain the structure of the above neural network and the importance of optimization in training
(b) Discuss how the steepest descent method can be adapted for optimizing neural network parameters.
c Use the small synthetic dataset for binary classification.
x1 x2 1.2 0.7 2 0.5 0.2 3 0.3 0.5 1.0 0.5 5 0.7 0.1 6 0.2 0.3 7 0.3 0.4 8 -0.5 0.6 9 0.8 0.2 10 0.4 0.4 11 0.2 0.3 12 0.6 0.3 13 0.1 0.2 14 0.4 0.1 15 -0.7 0.5 16 0.9 0.4 17 -0.6 0.3 18 0.1 0.2 19 -0.4 -0.1
1 0 1
0
0
0
0
0
0
0
1
d Implement the revised steepest descent algorithm to optimize the weights, biases and learning rate in a simple neural network