Question

1. (a) (5 marks) Derive the gradients of linear regression with 5 variables using expanded equation form and vector form and write the gradient descent update rule for the weights using Mean Squared Error (MSE) as loss function. Typically, gradient descent is run on the entire dataset (full gradient descent) or mini-batches (batched gradient descent). This problem is simple enough that you should be able to get results using per example iterations and MSE for every $y_i$, individually without summing it over mini-batches or the full dataset. Expanded equation form: y = w$_1$x$_1$ + w$_2$x$_2$ + w$_3$x$_3$ + w$_4$x$_4$ + w$_5$x$_5$ + b Vector form: w = [w$_1$, w$_2$, w$_3$, w$_4$, w$_5$]$^T$, x = [x$_1$, x$_2$, x$_3$, x$_4$, x$_5$]$^T$, b y = w$^T$x + b Here, w are the weights and b is the offset. Mean Squared Error: $\mathcal{L}(w; \{(x^{(i)}, y^{(i)})\}) = \frac{1}{N} \sum_{i=1}^{N} (\hat{y}^{(i)} - y^{(i)})^2$, i = 1,2,...,N $= \frac{1}{N} \sum_{i=1}^{N} (w^Tx^{(i)} - y^{(i)})^2$, i = 1,2,...,N Here \{(x$^{(i)}$, y$^{(i)}$); i = 1,2,...,N\} are the labeled data samples available for the optimization. $\hat{y}_i$ are the predictions from the linear regression model. Submit the derivation of the gradients for the above 5-dimensional example, both in the expanded equation form as well as the vector form.

          1. (a) (5 marks) Derive the gradients of linear regression with 5 variables using expanded
equation form and vector form and write the gradient descent update rule for the weights
using Mean Squared Error (MSE) as loss function. Typically, gradient descent is run
on the entire dataset (full gradient descent) or mini-batches (batched gradient descent).
This problem is simple enough that you should be able to get results using per example
iterations and MSE for every $y_i$, individually without summing it over mini-batches or
the full dataset.
Expanded equation form:
y = w$_1$x$_1$ + w$_2$x$_2$ + w$_3$x$_3$ + w$_4$x$_4$ + w$_5$x$_5$ + b
Vector form:
w = [w$_1$, w$_2$, w$_3$, w$_4$, w$_5$]$^T$, x = [x$_1$, x$_2$, x$_3$, x$_4$, x$_5$]$^T$, b
y = w$^T$x + b
Here, w are the weights and b is the offset.
Mean Squared Error:
$\mathcal{L}(w; \{(x^{(i)}, y^{(i)})\}) = \frac{1}{N} \sum_{i=1}^{N} (\hat{y}^{(i)} - y^{(i)})^2$,  i = 1,2,...,N
$= \frac{1}{N} \sum_{i=1}^{N} (w^Tx^{(i)} - y^{(i)})^2$,  i = 1,2,...,N
Here \{(x$^{(i)}$, y$^{(i)}$); i = 1,2,...,N\} are the labeled data samples available for the optimization.
$\hat{y}_i$ are the predictions from the linear regression model.
Submit the derivation of the gradients for the above 5-dimensional example, both in the
expanded equation form as well as the vector form.

$1. (a) (5 marks) Derive the gradients of linear regression with 5 variables using expanded equation form and vector form and write the gradient descent update rule for the weights using Mean Squared Error (MSE) as loss function. Typically, gradient descent is run on the entire dataset (full gradient descent) or mini-batches (batched gradient descent). This problem is simple enough that you should be able to get results using per example iterations and MSE for every yi, individually without summing it over mini-batches or the full dataset. Expanded equation form: y = w1x1 + w2x2 + w3x3 + w4x4 + w5x5 + b Vector form: w = [w1, w2, w3, w4, w5]^T, x = [x1, x2, x3, x4, x5]^T, b y = w^Tx + b Here, w are the weights and b is the offset. Mean Squared Error: ℒ(w; {(x^(i), y^(i))}) = (1)/(N)∑i=1^N (ŷ^(i) - y^(i))^2, i = 1,2,...,N = (1)/(N)∑i=1^N (w^Tx^(i) - y^(i))^2, i = 1,2,...,N Here {(x^(i), y^(i)); i = 1,2,...,N} are the labeled data samples available for the optimization. ŷi are the predictions from the linear regression model. Submit the derivation of the gradients for the above 5-dimensional example, both in the expanded equation form as well as the vector form.$

Added by Valerie J.

Question

Please give Ace some feedback