L1 regularization forces the weights to decay towards zero (but not exactly zero) whereas in L2 we penalize the absolute value of the weights and the weights may be reduced to zero here.
Added by Jamie G.
Step 1
This means that some weights may be reduced to exactly zero, effectively removing them from the model. Show more…
Show all steps
Your feedback will help us improve your experience
Akash M and 83 other AP CS educators are ready to help you.
Ask a new question
Labs
Want to see this concept in action?
Explore this concept interactively to see how it behaves as you change inputs.
Key Concepts
Recommended Videos
Akash M.
Problem (6pt) Recall that a linear threshold function or linear classifier is given by: If (wo + Ewixi) < 0 then the class is negative, otherwise it is positive. Assume that 1 is true and 0 is false. Consider a function over n binary features, defined as follows: If at least k variables are false, where k < n is a constant, then the class is positive, otherwise the class is negative. Can you represent this function using a linear threshold function? If your answer is YES, then give a precise numerical setting of the weights. Otherwise, clearly explain why this function cannot be represented using a linear threshold function.
Rashmi S.
Derive a gradient descent training rule for a single-unit neuron with output o, defined as: o = w0 + w1(x1 + x1^2) + ··· + wn(xn + xn^2) where x1, x2, ..., xn are the inputs, w1, w2, ..., wn are the corresponding weights, and w0 is the bias weight. You can assume an identity activation function, i.e. f(x) = x. Show all steps of your derivation and the final result for weight update. You can assume a learning rate of η.
Recommended Textbooks
Computer Science and Information Technology
Introduction to Programming Using Python
Computer Science - An Overview
Transcript
Watch the video solution with this free unlock.
EMAIL
PASSWORD