Im sorry i cannot assist with that request 38128

Name: im sorry i cannot assist with that request 38128
Uploaded: 2022-03-21T18:19:12-08:00
Duration: 3 min 1 s
Channel: Babar Ali
Description: im sorry i cannot assist with that request 38128

Babar Ali and 73 other subject AP CS educators are ready to help you.

Ask a new question

Labs

Want to see this concept in action?

NEW

Explore this concept interactively to see how it behaves as you change inputs.

View Labs

Key Concepts

Key Concept

Premium Feature

Explore the core concept behind this problem.

Key Concept

Premium Feature

Explore the core concept behind this problem.

Recommended Videos

given-data-xiyi-rd-x-l-i-ln-logistic-regression-is-another-popular-classification-method-in-machine-learning-which-amounts-to-the-following-minimization-problem-min-1-fw-2-log1-e-yiwx-zilwl-40103

Given data {(x_i, y_i) ∈ ℐ^d % #177;1} : i ∈ [1, n]}, logistic regression is another popular classification method in Machine Learning, which amounts to the following minimization problem: min_{w ∈ ℐ^d} {f(w) := 1/n ∑_{i=1}^n log(1 + e^{-y_i✨w, x_i✩}) + λ/2 ||w||^2}, where λ > 0 is a regularization parameter. Work out the gradient function ∇f(w) and the Hessian function ∇^2f(w). Show that its gradient function ∇f(w) is Lipschitz continuous with constant L ≤ 1/n ∑_{i=1}^n ||x_i||^2 + λ, i.e. ||∇f(w) - ∇f(w')|| ≤ (1/n ∑_{i=1}^n ||x_i||^2 + λ) ||w - w'||, ∀ w, w' ∈ ℐ^d. (Recall that in Homework 3 we have shown the objective function f of the logistic regression is convex) Write down the pseudo-code of Stochastic gradient descent (SGD) for logistic regression. (Bonus question) Consider the average of iterates, i.e. w_T = 1/T ∑_{t=1}^T w_t where {w_t : t ∈ [1, T]} is generated by SGD with step sizes η_t = 1/(λt). Prove that the following convergence rate for SGD in logistic regression: E(f_2(w_T)) - min_{w ∈ ℐ^d} f_2(w) = O(log T / T).

Babar A.

question-principled-method-for-learning-the-step-size-in-gradient-descent-in-class-discussed-that-when-we-gradient-descent-muinimye-target-function-w-with-respect-the-step-size-ak-at-iterati-19301

Question: Principled method for learning the step size in gradient descent: In class, we discussed that when we perform gradient descent to minimize the target function J(w) with respect to the step size a(k) at each iteration, it is a crucial hyperparameter. We further said that we can experimentally determine a(k) through cross-validation. There is actually a principled way for computing the optimal a(k) in each iteration, and we are going to derive the expression for that. (a) (0.5 point) According to Taylor series expansion, for a differentiable function f(x) around a point x0, we have: f(x) = f(x0) + (Vf(x0))T(x - x0) + (1/2)(x - x0)TH(x - x0) where Vf(x0) is the gradient vector and H is the Hessian matrix of f evaluated at x0. Let w(k) be the value of f at the kth iteration of gradient descent. Show that the second-order Taylor expansion of the target function J(w) around w(k) is the following: J(w) = J(w(k)) + (VJ(w(k)))T(w - w(k)) + (1/2)(w - w(k))TH(w - w(k)) (b) (1 point) Show that the above expression of J(w) evaluated at w(k + 1) (i.e. at the (k + 1)th gradient descent iteration) can be written as: J(w(k + 1)) = J(w(k)) - a(k)(VJ(w(k)))T(VJ(w(k))) + 3a(k)(VJ(w(k)))TH(w - w(k)) + a(k)(w - w(k))TH(w - w(k)) Tip: Take into account the gradient descent update rule w(k + 1) = w(k) - a(k)VJ(w(k)). (c) (1 point) Show that minimizing the above expression with respect to a(k) and setting it to zero results in: a(k) = (VJ(w(k)))T(VJ(w(k))) / (VJ(w(k)))TH(w - w(k))(VJ(w(k)))T The above expression gives the closed-form solution for the optimal a(k) in each iteration (i.e. the a(k) that minimizes the target function for the next iteration). (d) (Bonus) What is the cost of computing a(k) at each iteration using the above expression?

Sri K.

a-15-points-using-our-forward-and-backward-variables-akz-pyi-yk-xk-bx2-pykls-_-yt-xk-x-for-a-sequence-of-observations-yi-yt-of-length-t-1the-state-transition-matrix-tij-pxkl-ij-xk-ti-and-the-39072

(a) (15 points) Using our forward and backward variables ̑k(x) = P(Y1, ..., Yk, Xk = x) ̢k(x) = P(Yk+1, ..., Yt | Xk = x). for a sequence of observations Y1, ..., Yt of length t > 1, the state transition matrix Tij = P(Xk+1 = xj | Xk = xi) and the observation matrix Mij = P(Yk = yj | Xk = xi), write down how you will compute the following probabilities (1) P(Xk+1 = xj | Xk = xi, Y1, ..., Yt), (2) P(Xk = xi | Xk+1 = xj, Y1, ..., Yt), (3) P(Xk-1 = xi, Xk = xj, Xk+1 = xl | Y1, ..., Yt). You do not need to consider the boundary cases like k ∈ {1, t}. (b) (5 points) Viterbi's algorithm finds the most likely state trajectory of the HMM's associated Markov chain given a sequence of observations Y1, ..., Yt. Explain why, in general, the solution of the decoding problem, i.e., (x1*, ..., xt*) = argmax P(X1 = x1, ..., Xt = xt | Y1, ..., Yt). (x1, ..., xt) is not the same as the sequence obtained by most likely state at each time computed by the smoothing, i.e., (̂x1, ..., ̂xt) where ̂xk = argmax P(Xk = x | Y1, ..., Yt). x Give an example where the two are the same.

Sri K.