Question

Example: Ordinary Least Squares Estimation In a linear regression model the dependent variable (y) is a linear function of the independent variables ($x_1, x_2, \dots, x_k$) and an error term ($\varepsilon$). For observation $i$, $y_i = x_{i,1}\beta_1 + x_{i,2}\beta_2 + \dots + x_{i,k}\beta_k + \varepsilon_i$. Assuming $i = 1, \dots, n$ (that is, $n$ equations), we can use matrix notation to express the above as $y = X\beta + \varepsilon$, where $y$ is an $n \times 1$ column vector, $X$ is an $n \times k$ matrix, $\beta$ is a $k \times 1$ column vector, and $\varepsilon$ is an $n \times 1$ column vector. 22 The goal of the Ordinary Least Squares estimation is to find values of $\beta$ such that the squared distance between the \"fitted line\" and the actual data points, $\sum \varepsilon_i^2$, is minimized. In matrix notation, this means to minimize $\varepsilon'\varepsilon$: $\min_{\beta} (y - X\beta)'(y - X\beta)$ The first order condition implies that $-2X'(y - X\beta) = 0$ $X'X\beta = X'y$, where $X'X$ is a $k \times k$ square matrix. If $X'X$ is nonsingular, then the above yields $\beta = (X'X)^{-1}X'y$.

          Example: Ordinary Least Squares Estimation
In a linear regression model the dependent variable (y) is a linear function of the
independent variables ($x_1, x_2, \dots, x_k$) and an error term ($\varepsilon$). For observation $i$,
$y_i = x_{i,1}\beta_1 + x_{i,2}\beta_2 + \dots + x_{i,k}\beta_k + \varepsilon_i$.
Assuming $i = 1, \dots, n$ (that is, $n$ equations), we can use matrix notation to express
the above as
$y = X\beta + \varepsilon$,
where $y$ is an $n \times 1$ column vector, $X$ is an $n \times k$ matrix, $\beta$ is a $k \times 1$ column
vector, and $\varepsilon$ is an $n \times 1$ column vector.
22
The goal of the Ordinary Least Squares estimation is to find values of $\beta$ such that
the squared distance between the \"fitted line\" and the actual data points, $\sum \varepsilon_i^2$,
is minimized.
In matrix notation, this means to minimize $\varepsilon'\varepsilon$:
$\min_{\beta} (y - X\beta)'(y - X\beta)$
The first order condition implies that
$-2X'(y - X\beta) = 0$
$X'X\beta = X'y$,
where $X'X$ is a $k \times k$ square matrix. If $X'X$ is nonsingular, then the above yields
$\beta = (X'X)^{-1}X'y$.

Example: Ordinary Least Squares Estimation
In a linear regression model the dependent variable (y) is a linear function of the
independent variables (x1, x2, …, xk) and an error term (ε). For observation i,
yi = xi,1β1 + xi,2β2 + … + xi,k +.
Assuming i = 1, …, n (that is, n equations), we can use matrix notation to express
the above as
y = Xβ + ε,
where y is an n × 1 column vector, X is an n × k matrix, β is a k × 1 column
vector, and ε is an n × 1 column vector.
22
The goal of the Ordinary Least Squares estimation is to find values of β such that
the squared distance between the f̈itted lineänd the actual data points, ∑^2,
is minimized.
In matrix notation, this means to minimize ε'ε:
minβ (y - Xβ)'(y - Xβ)
The first order condition implies that
-2X'(y - Xβ) = 0
X'Xβ = X'y,
where X'X is a k × k square matrix. If X'X is nonsingular, then the above yields
β = (X'X)^-1X'y.

Added by Jennifer M.

Question

Please give Ace some feedback