Problem 4: (20 pts) For linear regression, there exists an equation that directly
yields the coefficients that minimizes the sum square error. It is known as the
\"normal equation.\"
(a) For part (a) let
y = \Theta_0 + \Theta_1x
(6)
Given: a set of samples in a training data set ($x^{(i)}$, $y^{(i)}$) wherein $i \in [1, m]$ and
$x^{(i)}$ is the $i$th input and $y^{(i)}$ is the real valued output associated with $x^{(i)}$. $x \in \mathbb{R}$
and $y \in \mathbb{R}$.
Find the \"normal equation\" that yields ($\Theta_0$, $\Theta_1$) as a function of ($x^{(i)}$, $y^{(i)}$) such
that ($\Theta_0$, $\Theta_1$) minimizes the following cost function:
minimize $J(\Theta) = \sum_{i=1}^{m} \frac{1}{2}(h_{\Theta}(x^{(i)}) - y^{(i)})^2$
(7)
Hint: the minimum occurs where the derivative is zero.
(b) Find the \"normal equation\" where y is a function of two dimensions, i.e.,
y = \Theta_0 + \Theta_1x_1 + \Theta_2x_2
(8)
and $x \in \mathbb{R}^2$ and $y \in \mathbb{R}$.
