In this problem, you will find out how to calculate PRESS residuals from the results of a single least square fit using all observations. There is another proof in Appendix C.7 of the textbook using the Sherman-Morrison-Woodbury formula. The arguments here are more intuitive and geometric. Suppose we have data (X1, Y1), (Xn, Yn). If we fit the regression model Vi = xi8 + ei for 1 < i < n and use B to denote the least square estimate, then the usual i-th residual is defined as e = Yi - Ji = yi - X@.
Fit the regression model using all observations except for the i-th observation (Ti, yi). Denote the least square estimate of the regression coefficients by Bd. The predicted value for Yi using this fitted model is denoted by y() 18(), and the i-th PRESS residual is defined as e(i) = Yi - Y(i). Suppose we have the following n observations: (X1, Y1), (xi-1, Yi-1), (xi, Y()), (xi+1, Yi+1), (xn, Yn). Note that the only difference from the original data is that the i-th observation (Xi, yi) is replaced by (X;, Uc)). Show that if we fit the regression model using the n observations in (1), then the least square estimate is the same as B().