Problem 5 (2 points) The function for elastic net regularization in sklearn has a L1_ratio parameter which describes the weight of the L1 and L2 penalty terms relative to each other. The ? parameter describes the strength of the regularization terms. The formulation of the objective function follows: ||Xw – y||?² + ? · L1_ratio · ||w||? + ? · (1 – L1_ratio) · ||w||?² If the L1_ratio term is 0, what is the formulation equivalent to? 1. Standard LLS 2. LLS with L1 regularization 3. LLS with L2 regularization 4. LLS with L1 and L2 regularization
Added by Benjamin R.
Close
Step 1
Step 1: When the L1 ratio term is 0, the formulation becomes: ||Xw - y||^2 + a * (1 - L1_ratio) * ||w||_2 Show more…
Show all steps
Your feedback will help us improve your experience
Adi S and 52 other Calculus 3 educators are ready to help you.
Ask a new question
Labs
Want to see this concept in action?
Explore this concept interactively to see how it behaves as you change inputs.
Key Concepts
Recommended Videos
You are given 3 data points with two features and one label as follows: X1 X2 Y 1 1 1.05 0.7 3 0.81 2 0.5 2.045 Suppose you have a linear regression model: y = w1x1 + w2x2, please calculate and report the least square error, the L1 regularization, and the L2 regularization terms when (i) w1 = 1, w2 = 0 (ii) w1 = 1, w2 = 0.02 If you set the regularization coefficient C = 1, which of the above two weights is preferred by the Lasso and which is preferred by Ridge regression? Could you use this example to explain why Lasso prefers sparse models?
Adi S.
Neural Network. Consider a three-layer neural network to learn a function f : X → Y, where X = [X1, X2] consists of two features. The weights w1,...,w6 can be arbitrary. There are two possible choices for the function implemented by each unit in this network: - S: sigmoid function, S(z) = 1/(1+exp(-z)) - L: linear function, L(z) = cz where in both cases z = Σi wi Xi. Assign proper activation functions (S or L) to each unit in the following graph so that we can generate functions of the form f(X1, X2) = 1/(1+exp(β1 X1 + β2 X2)) at the output of the neural network Y. Derive β1 and β2 as a function of w1,...,w6.
Shaiju T.
Given data {(x_i, y_i) ∈ ℐ^d % #177;1} : i ∈ [1, n]}, logistic regression is another popular classification method in Machine Learning, which amounts to the following minimization problem: min_{w ∈ ℐ^d} {f(w) := 1/n ∑_{i=1}^n log(1 + e^{-y_i✨w, x_i✩}) + λ/2 ||w||^2}, where λ > 0 is a regularization parameter. Work out the gradient function ∇f(w) and the Hessian function ∇^2f(w). Show that its gradient function ∇f(w) is Lipschitz continuous with constant L ≤ 1/n ∑_{i=1}^n ||x_i||^2 + λ, i.e. ||∇f(w) - ∇f(w')|| ≤ (1/n ∑_{i=1}^n ||x_i||^2 + λ) ||w - w'||, ∀ w, w' ∈ ℐ^d. (Recall that in Homework 3 we have shown the objective function f of the logistic regression is convex) Write down the pseudo-code of Stochastic gradient descent (SGD) for logistic regression. (Bonus question) Consider the average of iterates, i.e. w_T = 1/T ∑_{t=1}^T w_t where {w_t : t ∈ [1, T]} is generated by SGD with step sizes η_t = 1/(λt). Prove that the following convergence rate for SGD in logistic regression: E(f_2(w_T)) - min_{w ∈ ℐ^d} f_2(w) = O(log T / T).
Babar A.
Recommended Textbooks
Calculus: Early Transcendentals
Thomas Calculus
Transcript
18,000,000+
Students on Numerade
Trusted by students at 8,000+ universities
Watch the video solution with this free unlock.
EMAIL
PASSWORD