Question 2: Suppose we have n sample pairs {x_(i),y_(i)}_(1)^(n), with hat(f)hat(f)=argmin_(f)sum_(i=1)^n (y_(i)-f(x_(i)))^(2)+lambda int_a^b (f^('')(t))^(2)dt.x_(1),dots,x_(n)tilde(f)a,bfx_(1),dots,x_(n)f(x_(i))=tilde(f)(x_(i)),i=1,dots,nh(x)=tilde(f)(x)-f(x).int_a^b f^('')(x)h^('')(x)dx=0.int_a^b f^('')(x)^(2)dx<=int_a^b tilde(f)^('')(x)^(2)dxh^('')(x)=0xin[a,b]h^('')(x)=0hh(x_(i))=0x_(i)h(x)=0a. The smoothing cubic spline estimate hat(f) is defined as
hat(f)=argmin_(f)sum_(i=1)^n (y_(i)-f(x_(i)))^(2)+lambda int_a^b (f^('')(t))^(2)dt.
In the class, we mentioned that it happens that the minimizer of the above problem is unique and is a natural cubic spline with knots at the input point x_(1),dots,x_(n). Here we will prove it. First, we assume that tilde(f) is any twice differentiable function on a,b.
Show that there exists a natural cubic spline f with knots at x_(1),dots,x_(n) (in the form of linear combination of those basis functions) such that f(x_(i))=tilde(f)(x_(i)),i=1,dots,n.
Define
h(x)=tilde(f)(x)-f(x).
Prove the following claim
int_a^b f^('')(x)h^('')(x)dx=0.
Hint: you may need to use integration by parts.
3. Now, we can show
int_a^b f^('')(x)^(2)dx<=int_a^b tilde(f)^('')(x)^(2)dx
with equality if and only if h^('')(x)=0 for all xin[a,b]. Note that h^('')(x)=0 implies that h must be linear, and since we already know that h(x_(i))=0 for all x_(i), this is equivalent to h(x)=0. And we finish the proof.