Exercise 1 Consider the regression model yi = ?0 + ?1 xi + ?i. The Gauss-Markov conditions hold. Find cov(?i, ?j) using the following two methods: a. Express ei = sum_{l=1}^n q_l Y_l and ej = sum_{r=1}^n p_r Y_r and then use the results on handout #4, pages 2-3. b. Find cov(ei, ej) using cov(ei, ej) = cov(Yi - ?i, Yj - ?j) = cov(Yi - ? - ??1(xi - x?), Yj - ? - ??1(xj - x?)). Exercise 2 Consider the simple regression model Yi = ?0 + ?1 Xi + ?i, i = 1, ..., n. The Gauss-Markov conditions hold, i.e. E(?i) = 0, var(?i) = ?^2, and ?1, ..., ?n are independent. We have shown in class that the OLS estimators can be expressed as linear combinations of the Yi's. In particular, ??1 = sum_{i=1}^n ?_i yi and ??0 = sum_{i=1}^n l_i yi. Prove the Gauss-Markov theorem (that the OLS estimates have the smallest variance among all the linear unbiased estimators) for ??0. (We say that ??0 is BLUE.) Exercise 3 Consider the regression model yi = ?0 + ?1 xi + ?i. The Gauss-Markov conditions hold. Suppose b0 and b1 are the estimators (not the OLS estimators) of ?0 and ?1. Show that sum_{i=1}^n (yi - b0 - b1 xi)^2 ? sum_{i=1}^n e_i^2, where e_i = yi - ??0 - ??1 xi.
Added by Tricia C.
Close
Step 1
First, we need to find the covariance between ei and ej using the given methods. Method 1: Express ei = Yi - (Ī²0 + Ī²1Xi) and ej = Yj - (Ī²0 + Ī²1Xj), and use the results on handout #4, pages 2-3. cov(ei, ej) = cov(Yi - (Ī²0 + Ī²1Xi), Yj - (Ī²0 + Ī²1Xj)) = cov(Yi, Yj) Show moreā¦
Show all steps
Your feedback will help us improve your experience
Breanna Ollech and 56 other Intro Stats / AP Statistics educators are ready to help you.
Ask a new question
Labs
Want to see this concept in action?
Explore this concept interactively to see how it behaves as you change inputs.
Key Concepts
Recommended Videos
Consider the standard simple regression model $y=\beta_{0}+\beta_{1} x+u$ under the Gauss-Markov Assumptions SLR.1, SLR.2, SLR.3, SLR.4 and SLR.5. The usual OLS estimators $\hat{\beta}_{0}$ and $\widehat{\beta}_{1}$ are unbiased for their respective population parameters. Let $\tilde{\beta}_{1}$ be the estimator of $\beta_{1}$ obtained by assuming the intercept is zero (see Section $2-6$ ). i. Find $E\left(\tilde{\beta}_{1}\right)$ in terms of the $x_{i}, \beta_{0},$ and $\beta_{1}$. Verify that $\tilde{\beta}_{1}$ is unbiased for $\beta_{1}$ when the population intercept $\left(\beta_{0}\right)$ is zero. Are there other cases where $\tilde{\beta}_{1}$ is unbiased? ii. Find the variance of $\tilde{\beta}_{1}$. (Hint: The variance does not depend on $\beta_{0} .$ ) iii. Show that $\operatorname{Var}\left(\tilde{\beta}_{1}\right) \leq \operatorname{Var}\left(\widehat{\beta}_{1}\right)$. [Hint: For any sample of data, $\sum_{i=1}^{n} x_{i}^{2} \geq \sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2},$ with strict inequality unless $\bar{x}=0 .]$ iv. Comment on the tradeoff between bias and variance when choosing between $\widehat{\beta}_{1}$ and $\tilde{\beta}_{1}$.
Consider the simple linear regression model without an intercept: Y = Ī²X + Īµ . Here there is a single feature X ā R and a corresponding coefficient Ī² ā R. The above model presumes that the intercept is known a priori and hence does not need to be estimated. Suppose that we have collected data (x1, y1), ā¦, (xn, yn), where each xi is an observed scalar value of the feature X and each yi is a corresponding observed scalar value of the dependent variable Y. Please answer the following: a) (10 points) Following the principle of minimizing the RSS error (i.e., least squares), derive a closed form solution for Ī²Ģ, the estimator of Ī², in terms of the data that we have collected. Assume now that the collected data (x1, y1), ā¦, (xn, yn) satisfies: yi = Ī² xi + Īµi for all i = 1, ā¦, n , where Ī² ā R is the true coefficient value and Īµ1, ā¦, Īµn are i.i.d. random variables with mean zero and variance Ļ^2. Here we assume that the xi values (as well as the parameters Ī² and Ļ) are fixed and deterministic. Please answer the following: b) (2 points) Explain why Ī²Ģ is a random variable. c) (10 points) Derive formulas for the expected value and variance of Ī²Ģ. d) (3 points) Suppose we make an additional assumption that Īµ1, ā¦, Īµn are normally distributed. Argue that Ī²Ģ is also normally distributed then.
Sri K.
(Weighted Least Squares) Suppose that in the model $y_{i}=\beta_{0}+\beta_{1} x_{i}+e_{i},$ the errors have mean zero and are independent, but $\operatorname{Var}\left(e_{i}\right)=\rho_{i}^{2} \sigma^{2},$ where the $\rho_{i}$ are known constants, so the errors do not have equal variance. This situation arises when the $y_{i}$ are averages of several observations at $x_{i}$; in this case, if $y_{i}$ is an average of $n_{i}$ independent observations, $\rho_{i}^{2}=1 / n_{i}$ (why?). Because the variances are not equal, the theory developed in this chapter does not apply; intuitively, it seems that the observations with large variability should influence the estimates of $\beta_{0}$ and $\beta_{1}$ less than the observations with small variability. The problem may be transformed as follows: $$ \rho_{i}^{-1} y_{i}=\rho_{i}^{-1} \beta_{0}+\rho_{i}^{-1} \beta_{1} x_{i}+\rho_{i}^{-1} e_{i} $$ or $$ z_{i}=u_{i} \beta_{0}+v_{i} \beta_{1}+\delta_{i} $$ where $$ u_{i}=\rho_{i}^{-1} \quad v_{i}=\rho_{i}^{-1} x_{i} \quad \delta_{i}=\rho_{i}^{-1} e_{i} $$ a. Show that the new model satisfies the assumptions of the standard statistical model. b. Find the least squares estimates of $\beta_{0}$ and $\beta_{1}$ c. Show that performing a least squares analysis on the new model, as was done in part (b), is equivalent to minimizing $$ \sum_{i=1}^{n}\left(y_{i}-\beta_{0}-\beta_{1} x_{i}\right)^{2} \rho_{i}^{-2} $$ This is a weighted least squares criterion; the observations with large variances are weighted less. d. Find the variances of the estimates of part (b).
Recommended Textbooks
Elementary Statistics a Step by Step Approach
The Practice of Statistics for AP
Introductory Statistics
Transcript
Watch the video solution with this free unlock.
EMAIL
PASSWORD