Why is it important that to apply the Relu function after a Convolutional layer in a CNN architecture... [3 marks]
Added by Patrick O.
Close
Step 1
**Non-linearity:** Convolutional layers perform linear operations (convolution and bias addition). Without a non-linear activation function like ReLU, the entire CNN would essentially be a linear model, regardless of how many layers it has. Linear models have Show more…
Show all steps
Your feedback will help us improve your experience
Sri K and 61 other AP CS educators are ready to help you.
Ask a new question
Labs
Want to see this concept in action?
Explore this concept interactively to see how it behaves as you change inputs.
Key Concepts
Recommended Videos
Consider a three-layer fully-connected network with n1, n2, and n3 neurons in the three layers respectively. Inputs are fed into the first layer. The loss is the mean squared error, and the non-linearity is a sigmoid function. Let the label vector be t of size n3. Let each layer output vector be y and input vector be x, both of size ni. Let the weight matrix between layer i and layer i + 1 be W(i+1). The j-th element in yi is defined by yj = o(c(i-1)W(i-1),j-2,3), the same for the weight connecting the k-th and l-th neuron in layers i and i + 1, which is defined by W(i+1)kl. You do not need to consider bias in this problem. Input #1 Output #1 Input #2 Output #2 Input #3 Output #3 Input #4 Output #4 Figure 2: A three-layer fully-connected MLP network: Here is a summary of the notation: o(c) denotes the activation function for L2 and L3. There is no activation applied to the input layer. o(c(i-1)W(i-1),j-2,3) = o(c(i-1)W(i-1),j-2,3).
Sri K.
Breanna O.
Akash M.
Recommended Textbooks
Computer Science and Information Technology
Introduction to Programming Using Python
Computer Science - An Overview
Transcript
18,000,000+
Students on Numerade
Trusted by students at 8,000+ universities
Watch the video solution with this free unlock.
EMAIL
PASSWORD