In this problem, you’ll find MLEs for the Gaussian simple linear regression model. Let
Y i
indep.
⇠N 0 + 1 x i , 2 for i = 1, . . . , n, where x i are fixed ‘covariates’. Since the Y i ’s
are independent and the density of Y i is
f y i ; 0 , 1 , 2 = 1
p2⇡ 2 exp
⇢
1
2 2 (y i 0 1 x i ) 2
the likelihood function is
L 0 , 1 , 2 ; y i
=
nY
i=1
f y i ; 0 , 1 , 2 = 2⇡ 2 n
2 exp
(
1
2 2
nX
i=1
(y i 0 1 x i ) 2
)
(a) Write the log-likelihood ` 0 , 1 , 2 ; y i
= log L 0 , 1 , 2 ; y i
.
(b) Find @
@ 0 ` 0 , 1 , 2 ; y i
.
(c) Find @
@ 1 ` 0 , 1 , 2 ; y i
.
(d) Find @
@ 2 ` 0 , 1 , 2 ; y i
.
(e) Solve the system of equations
8
><
>:
@
@ 0 ` 0 , 1 , 2 ; y i
= 0
@
@ 1 ` 0 , 1 , 2 ; y i
= 0
@
@ 2 ` 0 , 1 , 2 ; y i
= 0
for 0 , 1 , 2 to obtain
0 = ̄y 1 ̄x
1 =
P n
i=1 x i y i n ̄y ̄x
P n
i=1 (x i ̄x) 2
2 = 1
n
nX
i=1
(y i 0 1 x i ) 2
Comment: to verify that these solutions jointly specify a maximum, one must show that
the matrix of second-order partial derivatives (known as the Hessian) is negative definite at
the solutions. This is the multivariate analogue of the second derivative test. You are not
expected to know how to do this.
8