Generalised Linear Modelling STAT3015/STAT4030/STAT7030 Chapter 1: Linear Models Lecturer: Professor Andrew Wood, RSFAS, ANU Semester 2, 2020 1.1 Preliminaries 1.1.1 Opening remarks This chapter aims to achieve two principal goals: (i) to review the standard linear model (which you should have covered to some extent, perhaps using the termi- nology multiple regression); and (ii) to study two particular types of linear model in greater depth, namely Analysis of Variance (ANOVA) models and Analysis of Co- variance (ANCOVA) models. For many of you the topics in (ii) will be new. ANOVA and ANCOVA models are particular cases of the linear model; the reason we are studying these classes of linear model separately is that they are sufficiently impor- tant and interesting to warrant separate study. In the final section of this chapter we briefly study different types of linear model which have additional sources of vari- ation which lead to dependencies between the response variables. The models studied here are known as random effects models and mixed effects models. This course will make heavy use of the statistical programming language R. Useful R functions for fitting linear models and analysis of variance models are lm() and aov () . It is assumed that you have studied multiple regression in a previous course. The present course will draw on knowledge of the prerequisite STAT2008 or a 1
course which covers similar material (e.g. STAT2014 or STAT6038 or STAT6014 or STAT7001). 1.1.2 Some comments on notation . IID is shorthand for "Independently and Identically Distributed". This indicates a random sample of independent random variables where each random vari- able has the same distribution. . In this course, bold notation will always be used to indicate vectors and ma- trices while scalars will be represented in non-bold font. . In line with most of the literature on Generalised Linear Models we will NOT adhere to the convenition that upper case is used for a random variable and lower case is used for its observed value. Please be alert to this; it will always be clear from the context which of these two interpretations is correct. . If X is a random variable we write X ~ F as shorthand for "X has distribution F"; e.g. X ~ N(0, 1) means that X has a standard normal distribution. 1.1.3 Signal-plus-noise decomposition The most fundamental decomposition in statistical modelling is: y =u+E, (1) or, in words, observation = signal + noise, (2) where y is an observed random quantity that we actually measure or observe, u is the deterministic (or systematic) component of the model (the "signal"), and € is the random (or stochastic) component of the model (the "noise"). 2 ---
Usually the decomposition is arranged so that e has zero expectation, i.e. E[e] = 0, where E[ .. ] denotes expectation. The vector form of this equation, written in bold, is also relevant: y =p+€, (3) where y = (y1, . . . , yn)2, IL = (pl1, . . . , An)> and