Question

Problem 3: Given our objective function $E_{in}(w) = \frac{1}{N}(w^TX^TXw - 2w^TX^Ty + y^Ty)$, we obtained the gradient of $E_{in}$ as: $\nabla E_{in}(w) = \frac{2}{N}(X^TXw - X^Ty)$ To help you understand how we compute the gradient of $E_{in}$ in linear regression, prove that: $\nabla_w (w^TAw) = (A + A^T)w$ In this equation, A is a matrix of size (d+1) by (d+1). w is the weight vector of size (d+1) by 1. This will help you see how we obtain the first item in the gradient.

          Problem 3:
Given our objective function $E_{in}(w) = \frac{1}{N}(w^TX^TXw - 2w^TX^Ty + y^Ty)$, we obtained the gradient of $E_{in}$ as:
$\nabla E_{in}(w) = \frac{2}{N}(X^TXw - X^Ty)$
To help you understand how we compute the gradient of $E_{in}$ in linear regression, prove that:
$\nabla_w (w^TAw) = (A + A^T)w$
In this equation, A is a matrix of size (d+1) by (d+1). w is the weight vector of size (d+1) by 1.
This will help you see how we obtain the first item in the gradient.

Show more…

Problem 3:
Given our objective function Ein(w) = (1)/(N)(w^TX^TXw - 2w^TX^Ty + y^Ty), we obtained the gradient of Ein as:
∇ Ein(w) = (2)/(N)(X^TXw - X^Ty)
To help you understand how we compute the gradient of Ein in linear regression, prove that:
(w^TAw) = (A + A^T)w
In this equation, A is a matrix of size (d+1) by (d+1). w is the weight vector of size (d+1) by 1.
This will help you see how we obtain the first item in the gradient.

Added by Michael G.

University Physics with Modern Physics

Hugh D. Young 14th Edition

Instant Answer

Solved by Expert Mukesh Devi

Step 1

First, let's expand the equation âˆ‡wwAw: âˆ‡wwAw = (xTxw - xTy)(xTxw - xTy)w Now, let's simplify this expression: âˆ‡wwAw = (xTxw - xTy)(xTxw)w - (xTxw - xTy)(xTy)w Expanding further: âˆ‡wwAw = (xTxw)(xTxw)w - (xTy)(xTxw)w - (xTxw)(xTy)w + (xTy)(xTy)w Now, Show more…

Show all steps

lock

Thanks for your feedback!

Problem 3: Given our objective function Ein = -(wxXw - 2WXy + yy), we obtained the gradient of Ein as: âˆ‡Ein(w) = xTxw - xTy To help you understand how we compute the gradient of Ein in linear regression, prove that: âˆ‡wwAw = A + Aw In this equation, A is a matrix of size (d+1) by (d+1), and w is the weight vector of size (d+1) by 1. This will help you see how we obtain the first item in the gradient.

Play audio

Feedback

Powered by NumerAI

Mukesh Devi and 89 other subject Physics 103 educators are ready to help you.

Ask a new question

Labs

Want to see this concept in action?

NEW

Explore this concept interactively to see how it behaves as you change inputs.

View Labs

Key Concepts

Recommended Videos

Recommended Textbooks

Ace is your personal tutor. It breaks down any question with clear steps so you can learn.

Start Using Ace

Continue solving this problem

Create a free account to:

View full step-by-step solution
Ask follow-up questions with Ace AI
Save progress and study later