a) Orthogonal Partitioned Regression Theorem
In the multiple linear least squares regression of y on two sets of variables X₁ and X₂, if the two sets of variables are orthogonal, then the separate coefficient vectors can be obtained by separate regressions of Y on X₁ alone and y on X₂ alone.
Using the necessary notations provide a convincing proof of this theorem. In your proof, you're required to explain the concept of orthogonality in relation to the two sets of variables.
b) Frisch-Waugh (1933)-Lovell (1963) Theorem
In the linear least squares regression of vector Y on two sets of variables, X₂ and X₁, the subvector $\hat{\beta}_2$ is the set of coefficients obtained when the residuals from a regression of Y on X₁ alone are regressed on the set of residuals obtained when each column of X₂ is regressed on X₁.
Using the necessary notations provide a convincing proof of this theorem
c) Assume a model (DSP) of the structure
$Y = X_1\beta_1 + X_2\beta_2 + u_i$
where $\hat{\beta} = (\hat{\beta}_1', \hat{\beta}_2') = (X'X)^{-1}X'Y$
Here, Y is the dependent variable, X₁ and X₂ are the matrices of the first and second set of regressors respectively, $\beta_1$ and $\beta_2$ are the coefficients to be estimated and $u_i$ is the error term.
1. Show that $\hat{\beta}_1 = (X_1'M_2X_1)^{-1}(X_1'M_2Y)$, where $M_2 = I_n - X_2(X_2'X_2)^{-1}X_2'$.
2. Explain the properties of the matrix $M_2$ and explain how this matrix compares to another similar matrix M. What does the matrix accomplish.
3. How does the matrix M come about? Derive this matrix (residual maker). In the process also derive the projection matrix and explain what the projection matrix accomplishes.
d) Suppose a correctly specified model is given by:
$Y = X_1\beta_1 + X_2\beta_2 + u_i$
If we by mistake of omission of the key variable we regress Y on X₁ without X₂ so that the misspecified model is:
$Y = X_1\beta_1 + v_i$
Show how the bias comes up in the estimation of $\beta_1$ and indicate the actual bias. Explain the special condition or circumstance in which the estimator for $\beta_1$ would be unbiased despite the omission of the key variable, X₂