Part IIC RDS/Lent 2015 STATISTICAL MODELLING Example Sheet 2 (of 4) In all the questions that follow, X is an n by p design matrix with full column rank and P is the orthogonal projection on to the column space of X. We will assume that n - p ? 2. The vector Y E R" will be a vector of responses and we will define 3 := (XTX)-1xTY, õ2 := | (I - P)Y |2 / (n - p) and @ : = Y - XB. 1. Consider a linear model Y = XB + c. Now suppose we reparametrise by letting 0 = AB where A E RPXP is invertible, so now we have Y = XA-10 + E (with XA-1 the new design matrix). Show that the fitted values and predictions based on applying OLS in the reparametrised model will be identical to those in the original model. 2. Show that the AIC in a normal linear model is n{1+log(2Tô2)}+2(p+1). 3. Let f and g be two densities on R with S := {x : g(x) > 0} = {x : f(x) > 0}. Show that the Kullback-Leibler divergence, K(g, f) := [log{g(x)} - log{ f(x) }g(x)dx, 00 is non-negative. Hint: Use Jensen's inequality. 4. Suppose the design matrix X consists of just a single variable and a column of 1's rep- resenting an intercept term (as the first column). Show that the leverage, pi, of the ith observation satisfies 1 (Xi2- X2)2 Pi = ; + n k=1(Xk2- X2)2' where X2 := 1 2k=1 Xk2. Hint: Why can we assume that the ith component of the second column is Xi2 -X2 rather than Xi2? 5. Return to the brain sizes data studied in practical 3. > file_path <- + "http://www.statslab.cam.ac.uk/~rds37/teaching/statistical_modelling/" > BrainSize <- read.csv(paste0(file_path, "BrainSize.csv")) > attach(BrainSize) > BrainSizeLM2 <- lm(PIQ ~ MRI_Count + Height) In this question we will plot a confidence ellipse for the coefficients for brain size and height. To do this, first install the ellipsepackage using > install.packages("ellipse") and select a mirror of your choice. Next load the package with library(ellipse). Look at ?ellipse.lmand plot a 95% confidence ellipse for the coefficients with > plot(ellipse(BrainSizeLM2, c(2, 3)), type = "l") 1
Using ablineadd to the plot the end points of 95% confidence intervals for each of the coefficients in red, and also add in blue the sides of the confidence rectangle in question 8 of Example sheet 1. If you are using Rstudio, you can output a pdf of your plot by clicking on "Export" above the plot window. Now look at the correlation between the estimates of these coefficients using > summary(BrainSizeLM2, correlation = TRUE)$correlation and compare this to the correlation between the corresponding variables > cor(Height, MRI_Count) What do you notice? Explain. 6. One of the data sets in the Modern Applied Statistics in S-Plus (MASS) library is hills. You can find out about the data with > library(MASS) > ?hills > pairs(hills) The data contain one known error in