Chapter Questions
Find the least squares solution to the pair of equations $3 x=1,2 x=-1$.
Find the minimum value of the function $f(x, y, z)=x^2+2 x y+3 y^2+2 y z+z^2-2 x+3 z+2$. How do you know that your answer is really the global minimum?
Find the closest point in the plane spanned by $(1,2,-1)^T,(0,-1,3)^T$ to the point $(1,1,1)^T$. What is the distance between the point and the plane?
Find the least squares solution to the linear system $A \mathbf{x}=\mathbf{b}$ when(a) $A=\left(\begin{array}{l}1 \\ 2 \\ 1\end{array}\right), \mathbf{b}=\left(\begin{array}{l}1 \\ 1 \\ 0\end{array}\right)$,(b) $A=\left(\begin{array}{rr}1 & 0 \\ 2 & -1 \\ 3 & 5\end{array}\right), \mathbf{b}=\left(\begin{array}{l}1 \\ 3 \\ 7\end{array}\right)$,(c) $A=\left(\begin{array}{rrr}2 & 1 & -1 \\ 1 & -2 & 0 \\ 3 & -1 & 1\end{array}\right), \mathbf{b}=\left(\begin{array}{l}1 \\ 0 \\ 1\end{array}\right)$.
Find the straight line $y=\alpha+\beta t$ that best fits the following data in the least squaressense: (a)\begin{tabular}{c|cccc}$t_i$ & -2 & 0 & 1 & 3 \\\hline$y_i$ & 0 & 1 & 2 & 5\end{tabular}(b)\begin{tabular}{c|ccccc}$t_i$ & 1 & 2 & 3 & 4 & 5 \\\hline$y_i$ & 1 & 0 & -2 & -3 & -3\end{tabular}(c)\begin{tabular}{c|ccccc}$t_i$ & -2 & -1 & 0 & 1 & 2 \\\hline$y_i$ & -5 & -3 & -2 & 0 & 3\end{tabular}
Find (i) the discrete Fourier coefficients, and (ii) the low-frequency trigonometric interpolant, for the following functions using the indicated number of sample points: (a) $\sin x$,$$n=4 \text {, }$$$$|x-\pi|, n=6,$$(c)$$f(x)=\left\{\begin{array}{ll}1, & x \leq 2, \\0, & x>2,\end{array} \quad n=6,\right.$$$$\operatorname{sign}(x-\pi), n=8 \text {. }$$
Find the minimizer of the function $f(x, y)=(3 x-2 y+1)^2+(2 x+y+2)^2$.
For the potential energy function in (5.1), where is the equilibrium position of the ball?
Redo Exercise 5.3.1 using(a) the weighted inner product $\langle\mathbf{v}, \mathbf{w}\rangle=2 v_1 w_1+4 v_2 w_2+3 v_3 w_3 ;$ (b) the inner product $\langle\mathbf{v}, \mathbf{w}\rangle=\mathbf{v}^T C \mathbf{w}$ based on the positive definite matrix $C=\left(\begin{array}{rrr}2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2\end{array}\right)$.
Find the least squares solutions to the following linear systems:(a) $x+2 y=1,3 x-y=0,-x+2 y=3$,(b) $4 x-2 y=1,2 x+3 y=-4, x-2 y=-1,2 x+2 y=2$,(c) $2 u+v-2 w=1,3 u-2 w=0, u-v+3 w=2$,(d) $x-z=-1,2 x-y+3 z=1, y-3 z=0,-5 x+2 y+z=3$,(e) $x_1+x_2=2, x_2+x_4=1, x_1+x_3=0, x_3-x_4=1, x_1-x_4=2$.
The proprietor of an internet travel company compiled the following data relating the annual profit of the firm to its annual advertising expenditure (both measured in thousands of dollars):\begin{tabular}{c|cccccc}$\begin{array}{c}\text { Annual advertising } \\\text { expenditure }\end{array}$ & 12 & 14 & 17 & 21 & 26 & 30 \\\hline Annual profit & 60 & 70 & 90 & 100 & 100 & 120\end{tabular}(a) Determine the equation of the least squares line. (b) Plot the data and the least squares line. (c) Estimate the profit when the annual advertising budget is $\$ 50,000$.(d) What about a $\$ 100,000$ budget?
Find (i) the sample values, and (ii) the trigonometric interpolant corresponding to the following discrete Fourier coefficients:(a) $c_{-1}=c_1=1, c_0=0$,(b) $c_{-2}=c_0=c_2=1, c_{-1}=c_1=-1$,(d) $c_0=c_2=c_4=1, c_1=c_3=c_5=-1$.(c) $c_{-2}=c_0=c_1=2, c_{-1}=c_2=0$,
Find the closest point or points to $\mathbf{b}=(-1,2)^T$ that lie on (a) the $x$-axis, (b) the $y$-axis, $(c)$ the line $y=x,(d)$ the line $x+y=0,(e)$ the line $2 x+y=0$.
For each of the following quadratic functions, determine whether there is a minimum. If so, find the minimizer and the minimum value for the function.(a) $x^2-2 x y+4 y^2+x-1$(b) $3 x^2+3 x y+3 y^2-2 x-2 y+4$(c) $x^2+5 x y+3 y^2+2 x-y$(d) $x^2+y^2+y z+z^2+x+y-z$, (e) $x^2+x y-y^2-y z+z^2-3$,(f) $x^2+5 x z+y^2-2 y z+z^2+2 x-z-3$,(g) $x^2+x y+y^2+y z+z^2+z w+w^2-2 x-w$
Find the point in the plane $x+2 y-z=0$ that is closest to $(0,0,1)^T$.
Let $A=\left(\begin{array}{rrr}3 & -3 & 1 \\ 2 & 4 & 1 \\ 1 & 2 & 1\end{array}\right), \mathbf{b}=\left(\begin{array}{l}6 \\ 5 \\ 4\end{array}\right)$. Prove, using Gaussian Elimination, that the linear system $A \mathbf{x}=\mathbf{b}$ has a unique solution. Show that the least squares solution (5.34) is the same. Explain why this is necessarily the case.
The median price (in thousands of dollars) of existing homes in a certain metropolitan area from 1989 to 1999 was:\begin{tabular}{l|lllllllllll} year & 1989 & 1990 & 1991 & 1992 & 1993 & 1994 & 1995 & 1996 & 1997 & 1998 & 1999 \\\hline price & 86.4 & 89.8 & 92.8 & 96.0 & 99.6 & 103.1 & 106.3 & 109.5 & 113.3 & 120.0 & 129.5\end{tabular}(a) Find an equation of the least squares line for these data. (b) Estimate the median price of a house in the year 2005, and the year 2010, assuming that the trend continues.
Let $f(x)=x$. Compute its discrete Fourier coefficients based on $n=4,8$ and 16 interpolation points. Then, plot $f(x)$ along with the resulting (real) trigonometric interpolants and discuss their accuracy.
Solve Exercise 5.1.3 when distance is measured in (i) the $\infty$ norm, (ii) the 1 norm.
(a) For which numbers $b$ (allowing both positive and negative numbers) is the matrix $A=\left(\begin{array}{ll}1 & b \\ b & 4\end{array}\right)$ positive definite? (b) Find the factorization $A=L D L^T$ when $b$ is in the range for positive definiteness. (c) Find the minimum value (depending on $b$; it might be finite or it might be $-\infty)$ of the function $p(x, y)=x^2+2 b x y+4 y^2-2 y$.
Let $\mathbf{b}=(3,1,2,1)^T$. Find the closest point and the distance from $\mathbf{b}$ to the following subspaces: (a) the line in the direction $(1,1,1,1)^T ;(b)$ the plane spanned by $(1,1,0,0)^T$ and $(0,0,1,1)^T ;(c)$ the hyperplane spanned by $(1,0,0,0)^T,(0,1,0,0)^T,(0,0,1,0)^T$; (d) the hyperplane defined by the equation $x+y+z+w=0$.
Find the least squares solution to the linear system $A \mathbf{x}=\mathbf{b}$ when(a) $A=\left(\begin{array}{rr}2 & 3 \\ 4 & -2 \\ 1 & 5 \\ 2 & 0\end{array}\right), \quad \mathbf{b}=\left(\begin{array}{r}2 \\ -1 \\ 1 \\ 3\end{array}\right)$(b) $A=\left(\begin{array}{rrr}2 & 1 & 4 \\ 1 & -2 & 1 \\ 1 & 0 & -3 \\ 5 & 2 & -2\end{array}\right), \quad \mathbf{b}=\left(\begin{array}{l}0 \\ 0 \\ 1 \\ 0\end{array}\right)$.
A 20-pound turkey that is at the room temperature of $72^{\circ}$ is placed in the oven at 1:00 $\mathrm{pm}$. The temperature of the turkey is observed in 20 minute intervals to be $79^{\circ}, 88^{\circ}$, and $96^{\circ}$. A turkey is cooked when its temperature reaches $165^{\circ}$. How much longer do you need to wait until the turkey is done?
Answer Exercise 5.6.3 for the functions(a) $x^2$,(b) $(x-\pi)^2$,(c) $\sin x$,(d) $\cos \frac{1}{2} x$,(e) $\begin{cases}1, & \frac{1}{2} \pi \leq x \leq \frac{3}{2} \pi, \\ 0, & \text { otherwise, }\end{cases}$(f) $\begin{cases}x, & 0 \leq x \leq \pi, \\ 2 \pi-x, & \pi \leq x \leq 2 \pi .\end{cases}$
Given $\mathbf{b} \in \mathbb{R}^2$, is the closest point on a line $L$ unique when distance is measured in (a) the Euclidean norm? (b) the 1 norm? (c) the $\infty$ norm?
For each matrix $K$, vector $\mathbf{f}$, and scalar $c$, write out the quadratic function $p(\mathbf{x})$ given by (5.10). Then either find the minimizer $\mathbf{x}^{\star}$ and minimum value $p\left(\mathbf{x}^{\star}\right)$, or explain why there is none. (a) $K=\left(\begin{array}{rr}4 & -12 \\ -12 & 45\end{array}\right), \mathbf{f}=\left(\begin{array}{c}-\frac{1}{2} \\ 2\end{array}\right), c=3$;(b) $K=\left(\begin{array}{ll}3 & 2 \\ 2 & 1\end{array}\right), \mathbf{f}=\left(\begin{array}{l}4 \\ 1\end{array}\right)$,$$\begin{aligned}& c=0 ; \quad(c) K=\left(\begin{array}{rrr}3 & -1 & 1 \\-1 & 2 & -1 \\1 & -1 & 3\end{array}\right), \mathbf{f}=\left(\begin{array}{r}1 \\0 \\-2\end{array}\right), c=-3 ;(d) K=\left(\begin{array}{rrr}1 & 1 & 1 \\1 & 2 & -1 \\1 & -1 & 1\end{array}\right), \\& \mathbf{f}=\left(\begin{array}{r}-3 \\-1 \\2\end{array}\right), c=1 ;(e) K=\left(\begin{array}{llll}1 & 1 & 0 & 0 \\1 & 2 & 1 & 0 \\0 & 1 & 3 & 1 \\0 & 0 & 1 & 4\end{array}\right), \mathbf{f}=\left(\begin{array}{r}-1 \\2 \\-3 \\4\end{array}\right), c=0 .\end{aligned}$$(c)
Find the closest point and the distance from $\mathbf{b}=(1,1,2,-2)^T$ to the subspace spanned by $(1,2,-1,0)^T,(0,1,-2,-1)^T,(1,0,3,2)^T$.
Given $A=\left(\begin{array}{rrr}1 & 2 & -1 \\ 0 & -2 & 3 \\ 1 & 5 & -1 \\ -3 & 1 & 1\end{array}\right)$ and $\mathbf{b}=\left(\begin{array}{l}0 \\ 5 \\ 6 \\ 8\end{array}\right)$, find the least squares solution to the system $A \mathbf{x}=\mathbf{b}$. What is the error? Interpret your result.
The amount of waste (in millions of tons a day) generated in a certain city from 1960 to 1995 was\begin{tabular}{c|cccccccc} year & 1960 & 1965 & 1970 & 1975 & 1980 & 1985 & 1990 & 1995 \\\hline amount & 86 & 99.8 & 115.8 & 125 & 132.6 & 143.1 & 156.3 & 169.5\end{tabular}(a) Find the equation for the least squares line that best fits these data.(b) Use the result to estimate the amount of waste in the year 2000, and in the year 2005.(c) Redo your calculations using an exponential growth model $y=c e^{\alpha t}$.(d) Which model do you think most accurately reflects the data? Why?
(a) Draw a picture of the complex plane with the complex solutions to $z^6=1$ marked. (b) What is the exact formula (no trigonometric functions allowed) for the primitive sixth root of unity $\zeta_6$ ? (c) Verify explicitly that $1+\zeta_6+\zeta_6^2+\zeta_6^3+\zeta_6^4+\zeta_6^5=0$.(d) Give a geometrical explanation of this identity.
Let $L \subset \mathbb{R}^2$ be a line through the origin, and let $\mathbf{b} \in \mathbb{R}^2$ be any point.(a) Find a geometrical construction of the closest point $\mathbf{v} \in L$ to $\mathbf{b}$ when distance is measured in the standard Euclidean norm.(b) Use your construction to prove that there is one and only one closest point.(c) Show that if $\mathbf{0} \neq \mathbf{a} \in L$, then the distance equals $\frac{\sqrt{\|\mathbf{a}\|^2\|\mathbf{b}\|^2-(\mathbf{a} \cdot \mathbf{b})^2}}{\|\mathbf{a}\|}=\frac{|\mathbf{a} \times \mathbf{b}|}{\|\mathbf{a}\|}$, using the two-dimensional cross product (3.22).
Find the minimum value of the quadratic function$$p\left(x_1, \ldots, x_n\right)=4 \sum_{i=1}^n x_i^2-2 \sum_{i=1}^{n-1} x_i x_{i+1}+\sum_{i=1}^n x_i \quad \text { for } \quad n=2,3,4 .$$
Redo Exercises 5.3.4 and 5.3.5 using(i) the weighted inner product $\langle\mathbf{v}, \mathbf{w}\rangle=\frac{1}{2} v_1 w_1+v_2 w_2+\frac{1}{2} v_3 w_3+v_4 w_4$;(ii) the inner product based on the positive definite matrix $C=\left(\begin{array}{rrrr}4 & -1 & 1 & 0 \\ -1 & 4 & -1 & 1 \\ 1 & -1 & 4 & -1 \\ 0 & 1 & -1 & 4\end{array}\right)$.
Find the least squares solution to the linear systems in Exercise 5.4.1 under the weighted norm $\|\mathbf{x}\|^2=x_1^2+2 x_2^2+3 x_3^2$.
The amount of radium-224 in a sample was measured at the indicated times.\begin{tabular}{c|cccccccc} time in days & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\\hline $\mathrm{mg}$ & 100 & 82.7 & 68.3 & 56.5 & 46.7 & 38.6 & 31.9 & 26.4\end{tabular}(a) Estimate how much radium will be left after 10 days.(b) If the sample is considered to be safe when the amount of radium is less than .01 mg, estimate how long the sample needs to be stored before it can be safely disposed of.
(a) Explain in detail why the $n^{\text {th }}$ roots of 1 lie on the vertices of a regular $n$-gon. What is the angle between two consecutive sides?(b) Explain why this is also true for the $n^{\text {th }}$ roots of every non-zero complex number $z \neq 0$. Sketch a picture of the hexagon corresponding to $\sqrt[6]{z}$ for a given $z \neq 0$.
Suppose $\mathbf{a}$ and $\mathbf{b}$ are unit vectors in $\mathbb{R}^2$. Show that the distance from a to the line through $\mathbf{b}$ is the same as the distance from $\mathbf{b}$ to the line through $\mathbf{a}$. Use a picture to explain why this holds. How is the distance related to the angle between the two vectors?
Find the maximum value of the quadratic functions(a) $-x^2+3 x y-5 y^2-x+1$,(b) $-2 x^2+6 x y-3 y^2+4 x-3 y$.
Find the vector $\mathbf{w}^{\star} \in \operatorname{span}\{(0,0,1,1),(2,1,1,1)\}$ that minimizes $\|\mathbf{w}-(0,3,1,2)\|$.
Let $A$ be an $m \times n$ matrix with $\operatorname{ker} A=\{0\}$. Suppose that we use the Gram-Schmidt algorithm to factor $A=Q R$ as in Exercise 4.3.32. Prove that the least squares solution to the linear system $A \mathbf{x}=\mathbf{b}$ is found by solving the triangular system $R \mathbf{x}=Q^T \mathbf{b}$ by Back Substitution.
The following table gives the population of the United States for the years 1900-2000.\begin{tabular}{c|cccccc} year & 1900 & 1920 & 1940 & 1960 & 1980 & 2000 \\\hline population - in millions & 76 & 106 & 132 & 181 & 227 & 282\end{tabular}(a) Use an exponential growth model of the form $y=c e^{a t}$ to predict the population in 2020,2050 , and 3000 . (b) The actual population for the year 2020 has recently been estimated to be 334 million. How does this affect your predictions for 2050 and 3000 ?
In general, an $n^{\text {th }}$ root of unity $\zeta$ is called primitive if all the $n^{\text {th }}$ roots of unity are obtained by raising it to successive powers: $1, \zeta, \zeta^2, \zeta^3, \ldots$. (a) Find all primitive (i) fourth, (ii) fifth, (iii) ninth roots of unity. (b) Can you characterize all the primitive $n^{\text {th }}$ roots of unity?
(a) Prove that the distance from the point $\left(x_0, y_0\right)^T$ to the line $a x+b y=0$ is $\frac{\left|a x_0+b y_0\right|}{\sqrt{a^2+b^2}}$.(b) What is the minimum distance to the line $a x+b y+c=0$ ?
Suppose $K_1$ and $K_2$ are positive definite $n \times n$ matrices. Suppose that, for $i=1,2$, the minimizer of $p_i(\mathbf{x})=\mathbf{x}^T K_i \mathbf{x}-2 \mathbf{x}^T \mathbf{f}_i+c_i$, is $\mathbf{x}_i^{\star}$. Is the minimizer of $p(\mathbf{x})=p_1(\mathbf{x})+p_2(\mathbf{x})$ given by $\mathbf{x}^{\star}=\mathbf{x}_1^{\star}+\mathbf{x}_2^{\star}$ ? Prove or give a counterexample.
(a) Find the distance from the point $\mathbf{b}=(1,2,-1)^T$ to the plane $x-2 y+z=0$.(b) Find the distance to the plane $x-2 y+z=3$.
Apply the method in Exercise 5.4.7 to find the least squares solutions to the systems in Exercise 5.4.2.
Find the best linear least squares fit of the following data using the indicated weights:(a)\begin{tabular}{c|cccc}$t_i$ & 1 & 2 & 3 & 4 \\\hline$y_i$ & .2 & .4 & .7 & 1.2 \\\hline$c_i$ & 1 & 2 & 3 & 4\end{tabular}(c)\begin{tabular}{c|ccccc}$t_i$ & -2 & -1 & 0 & 1 & 2 \\\hline$y_i$ & -5 & -3 & -2 & 0 & 3 \\\hline$c_i$ & 2 & 1 & .5 & 1 & 2\end{tabular}\begin{tabular}{c|cccccc}$x$ & 1 & 1 & 2 & 2 & 3 & 3 \\\hline$y$ & 1 & 2 & 1 & 2 & 2 & 4 \\\hline$z$ & 3 & 6 & 11 & -2 & 0 & 3\end{tabular}(b)\begin{tabular}{c|cccc}$t_i$ & 0 & 1 & 3 & 6 \\\hline$y_i$ & 2 & 3 & 7 & 12 \\\hline$c_i$ & 4 & 3 & 2 & 1\end{tabular}(d)\begin{tabular}{c|ccccc}$t_i$ & 1 & 2 & 3 & 4 & 5 \\\hline$y_i$ & 2 & 1.3 & 1.1 & .8 & .2 \\\hline$c_i$ & 5 & 4 & 3 & 2 & 1\end{tabular}
(a) In Example 5.27, the $n=4$ discrete Fourier coefficients of the function $f(x)=2 \pi x-x^2$ were found to be real. Is this true when $n=16$ ? For general $n$ ?(b) What property of a function $f(x)$ will guarantee that its Fourier coefficients are real?
(a) Generalize Exercise 5.1.8 to find the distance between a point $\left(x_0, y_0, z_0\right)^T$ and the plane $a x+b y+c z+d=0$ in $\mathbb{R}^3$. (b) Use your formula to compute the distance between $(1,1,1)^T$ and the plane $3 x-2 y+z=1$.
Let $K>0$. Prove that a quadratic function $p(\mathbf{x})=\mathbf{x}^T K \mathbf{x}-2 \mathbf{x}^T \mathbf{f}$ without constant term has non-positive minimum value: $p\left(\mathbf{x}^{\star}\right) \leq 0$. When is the minimum value zero?
(a) Given a configuration of $n$ points $\mathbf{a}_1, \ldots, \mathbf{a}_n$ in the plane, explain how to find the point $\mathrm{x} \in \mathbb{R}^2$ that minimizes the total squared distance $\sum_{i=1}^n\left\|\mathbf{x}-\mathbf{a}_i\right\|^2$.(b) Apply your method when (i) $\mathbf{a}_1=(1,3), \mathbf{a}_2=(-2,5) ;($ ii $) \mathbf{a}_1=(0,0), \mathbf{a}_2=(0,1), \mathbf{a}_3=(1,0)$; (iii) $\mathbf{a}_1=(0,0), \mathbf{a}_2=(0,2), \mathbf{a}_3=(1,2), \mathbf{a}_4=(-2,-1)$.
(a) Find a formula for the least squares error (5.35) in terms of an orthonormal basis of the subspace. (b) Generalize your formula to the case of an orthogonal basis.
For the data points(a) determine the best plane $z=a+b x+c y$ that best fits the data in the least squares sense; (b) how would you answer the question in part (a) if the plane were constrained to go through the point $x=2, y=2, z=0$ ?
Let $\mathbf{c}=\left(c_0, c_1, \ldots, c_{n-1}\right)^T \in \mathbb{C}^n$ be the vector of discrete Fourier coefficients corresponding to the sample vector $\mathbf{f}=\left(f_0, f_1, \ldots, f_{n-1}\right)^T$. (a) Explain why the sampled signal $\mathbf{f}=F_n \mathbf{c}$ can be reconstructed by multiplying its Fourier coefficient vector by an $n \times n$ matrix $F_n$. Write down $F_2, F_3, F_4$, and $F_8$. What is the general formula for the entries of $F_n$ ? (b) Prove that, in general, $F_n^{-1}=\frac{1}{n} F_n^{\dagger}=\frac{1}{n}{\overline{F_n}}^T$, where ${ }^{\dagger}$ denotes the Hermitian transpose defined in Exercise 4.3.25. (c) Prove that $U_n=\frac{1}{\sqrt{n}} F_n$ is a unitary matrix, i.e., $U_n^{-1}=U_n^{\dagger}$.
(a) Explain in detail why the minimizer of $\|\mathbf{v}-\mathbf{b}\|$ coincides with the minimizer of $\|\mathbf{v}-\mathbf{b}\|^2$. (b) Find all scalar functions $F(x)$ for which the minimizer of $F(\|\mathbf{v}-\mathbf{b}\|)$ is the same as the minimizer of $\|\mathbf{v}-\mathbf{b}\|$.
Let $q(\mathbf{x})=\mathbf{x}^T A \mathbf{x}$ be a quadratic form. Prove that the minimum value of $q(\mathbf{x})$ is either 0 or $-\infty$.
Answer Exercise 5.3.9 when distance is measured in (a) the weighted norm $\|\mathbf{x}\|=\sqrt{2 x_1^2+3 x_2^2} ;$(b) the norm based on the positive definite matrix $\left(\begin{array}{rr}3 & -1 \\ -1 & 2\end{array}\right)$.
Find the least squares solutions to the following linear systems. (a) $\left(\begin{array}{rr}1 & -1 \\ 2 & 2 \\ 3 & -1\end{array}\right)\left(\begin{array}{l}x \\ y\end{array}\right)=\left(\begin{array}{r}1 \\ 0 \\ -1\end{array}\right)$,(b) $\left(\begin{array}{rr}3 & -1 \\ 0 & 2 \\ -2 & 1 \\ 1 & 5\end{array}\right)\left(\begin{array}{l}x \\ y\end{array}\right)=\left(\begin{array}{r}2 \\ 1 \\ -1 \\ 1\end{array}\right)$,$(c)\left(\begin{array}{rrr}1 & -1 & -1 \\ 1 & 3 & 2 \\ -2 & 1 & 0 \\ 1 & 0 & -1 \\ 0 & 7 & -1\end{array}\right)$$\left(\begin{array}{l}x \\ y \\ z\end{array}\right)=\left(\begin{array}{r}-1 \\ 0 \\ 1 \\ -1 \\ 0\end{array}\right)$.
For the data points in Exercise 5.5.9, determine the plane $z=\alpha+\beta x+\gamma y$ that fits the data in the least squares sense when the errors are weighted according to the reciprocal of the distance of the point $\left(x_i, y_i, z_i\right)$ from the origin.
Construct the discrete Fourier coefficients for $f(x)= \begin{cases}-x, & 0 \leq x \leq \frac{1}{3} \pi \\ x-\frac{2}{3} \pi, & \frac{1}{3} \pi \leq x \leq \frac{4}{3} \pi \\ -x+2 \pi, & \frac{4}{3} \pi \leq x \leq 2 \pi\end{cases}$ based on $n=128$ sample points. Then graph the reconstructed function when using the data compression algorithm that retains only the 11 and 21 lowest-frequency modes. Discuss what you observe.
(a) Explain why the problem of maximizing the distance from a point to a subspace does not have a solution. (b) Can you formulate a situation in which maximizing distance to a point leads to a problem with a solution?
Under what conditions does the affine function $p(\mathbf{x})=\mathbf{x}^T \mathbf{f}+c$ have a minimum?
Explain why the quantity inside the square root in (5.25) is always non-negative.
Suppose we are interested in solving a linear system $A \mathbf{x}=\mathbf{b}$ by the method of least squares when the coefficient matrix $A$ has linearly dependent columns. Let $K \mathbf{x}=\mathbf{f}$, where $K=A^T C A, \mathbf{f}=A^T C \mathbf{b}$, be the corresponding normal equations. (a) Prove that $\mathbf{f} \in \operatorname{img} K$, and so the normal equations have a solution. Hint: Use Exercise 3.4.32.(b) Prove that every solution to the normal equations minimizes the least squares error, and hence qualifies as a least squares solution to the original system. (c) Explain why the least squares solution is not unique.
Show, by constructing explicit examples, that $\overline{t^2} \neq(\bar{t})^2$ and $\overline{t y} \neq \bar{t} \bar{y}$. Can you find any data for which either equality is valid?
Answer Exercise 5.6.10 when $f(x)=(a) x ;(b) x^2(2 \pi-x)^2 ;(c) \begin{cases}\sin x, & 0 \leq x \leq \pi, \\ 0, & \pi \leq x \leq 2 \pi .\end{cases}$
Under what conditions does a quadratic function $p(\mathbf{x})=\mathbf{x}^T K \mathbf{x}-2 \mathbf{x}^T \mathbf{f}+c$ have a finite global maximum? Explain how to find the maximizer and maximum value.
Find the closest point to the vector $\mathbf{b}=(1,0,2)^T$ belonging the two-dimensional subspace spanned by the orthogonal vectors $\mathbf{v}_1=(1,-1,1)^T, \mathbf{v}_2=(-1,1,2)^T$.
Which is the more efficient algorithm: direct least squares based on solving the normal equations by Gaussian Elimination, or using Gram-Schmidt orthonormalization and then solving the resulting triangular system by Back Substitution as in Exercise 5.4.7? Justify your answer.
Given points $t_1, \ldots, t_m$, prove $\overline{t^2}-(\bar{t})^2=\frac{1}{m} \sum_{i=1}^m\left(t_i-\bar{t}\right)^2$, thereby justifying (5.45).
Let $q_l(x)$ denote the trigonometric polynomial (5.119) obtained by summing the first $2 l+1$ discrete Fourier modes. Suppose the criterion for compression of a signal $f(x)$ is that $\left\|f-q_l\right\|_{\infty}=\max \left\{\left|f(x)-q_l(x)\right| \mid 0 \leq x \leq 2 \pi\right\}<\varepsilon$. For the particular function in Exercise 5.6.10, how large do you need to choose $k$ when $\varepsilon=.1$ ? $\varepsilon=.01$ ? $\varepsilon=.001$ ?
True or false: The minimal-norm solution to $A \mathbf{x}=\mathbf{b}$ is obtained by setting all the free variables to zero.
Let $\mathbf{b}=(0,3,1,2)^T$. Find the vector $\mathbf{w}^{\star} \in \operatorname{span}\left\{(0,0,1,1)^T,(2,1,1,-1)^T\right\}$ such that $\left\|\mathbf{w}^{\star}-\mathbf{b}\right\|$ is minimized.
A group of students knows that the least squares solution to $A \mathbf{x}=\mathbf{b}$ can be identified with the closest point on the subspace img $A$ spanned by the columns of the coefficient matrix. Therefore, they try to find the solution by first orthonormalizing the columns using Gram-Schmidt, and then finding the least squares coefficients by the orthonormal basis formula (4.41). To their surprise, they does not get the same solution! Can you explain the source of their difficulty? How can you use their solution to obtain the proper least squares solution $\mathbf{x}$ ? Check your algorithm with the system that we treated in Example 5.12.
Find and graph the polynomial of minimal degree that passes through the following points:(a) $(3,-1),(6,5)$;(b) $(-2,4),(0,6),(1,10)$;(c) $(-2,3),(0,-1),(1,-3)$;(d) $(-1,2),(0,-1),(1,0),(2,-1)$;(e) $(-2,17),(-1,-3),(0,-3),(1,-1),(2,9)$.
Let $f(x)=x(2 \pi-x)$ be sampled on $n=128$ equally spaced points between 0 and $2 \pi$. Use a random number generator with $-1 \leq r_j \leq 1$ to add noise by replacing each sample value $f_j=f\left(x_j\right)$ by $g_j=f_j+\varepsilon r_j$. Investigate, for different values of $\varepsilon$, how many discrete Fourier modes are required to reconstruct a reasonable denoised approximation to the original signal.
Prove that if $K$ is a positive semi-definite matrix, and $\mathbf{f} \notin \operatorname{img} K$, then the quadratic function $p(\mathbf{x})=\mathbf{x}^T K \mathbf{x}-2 \mathbf{x}^T \mathbf{f}+c$ has no minimum value.
Find the closest point to $\mathbf{b}=(1,2,-1,3)^T$ in the subspace $W=\operatorname{span}\left\{(1,0,2,1)^T\right.$, $\left.(1,1,0,-1)^T,(2,0,1,-1)^T\right\}$ by first constructing an orthogonal basis of $W$ and then applying the orthogonal projection formula (4.42).
For the following data values, construct the interpolating polynomial in Lagrange form:(a)\begin{tabular}{c|cc}$t_i$ & -3 & 2 \\\hline$y_i$ & 1 & 5\end{tabular}(d)(b)\begin{tabular}{c|ccc}$t_i$ & 0 & 1 & 3 \\\hline$y_i$ & 1 & .5 & .25\end{tabular}(c)\begin{tabular}{c|ccc}$t_i$ & -1 & 0 & 1 \\\hline$y_i$ & 1 & 2 & -1\end{tabular}(e)\begin{tabular}{r|rrrrr}$t_i$ & -2 & -1 & 0 & 1 & 2 \\\hline$y_i$ & -1 & -2 & 2 & 1 & 3\end{tabular}(a) find the straight line $y=\alpha+\beta t$ that best fits the
The signal in Figure 5.20 was obtained from the explicit formula$$f(x)=-\frac{1}{5}\left(\frac{x(2 \pi-x)}{10}\right)^5(x+1.5)(x+2.5)(x-4)+1.7 .$$Noise was added by using a random number generator. Experiment with different intensities of noise and different numbers of sample points and discuss what you observe.
Why can't you minimize a complex-valued quadratic function?
Repeat Exercise 5.3.14 using the weighted norm $\|\mathbf{v}\|=v_1^2+2 v_2^2+v_3^2+3 v_4^2$.
Given\begin{tabular}{l|rrr}$t_i$ & 1 & 2 & 3 \\\hline$y_i$ & 3 & 6 & 11\end{tabular}(a) find the straight line $y=\alpha+\beta t$ that best fits thedata in the least squares sense; (b) find the parabola $y=\alpha+\beta t+\gamma t^2$ that best fits the data. Interpret the error.
If we use the original form (5.100) of the discrete Fourier representation, we might be tempted to denoise/compress the signal by retaining only the first $0 \leq k \leq l$ terms in the sum. Test this method on the signal in Exercise 5.6.10 and discuss what you observe.
Justify the formulas in (5.29).
Re-solve Exercise 5.5.15 using the respective weights $2,1, .5$ at the three data points.
True or false: If $f(x)$ is real, the compressed/denoised signal (5.119) is a real trigonometric polynomial.
The table \begin{tabular}{c|cccc} time in sec & 0 & 10 & 20 & 30 \\ \cline { 2 - 6 } meters & 4500 & 4300 & 3930 & 3000 \end{tabular} measures the altitude of a falling parachutist before her chute has opened. Predict how many seconds she can wait before reaching the minimum altitude of 1500 meters.
Use the Fast Fourier Transform to find the discrete Fourier coefficients for the the following functions using the indicated number of sample points. Carefully indicate each step in your analysis.(a) $\frac{x}{\pi}, n=4$(b) $\sin x, n=8$;(c) $|x-\pi|, n=8$;(d) $\operatorname{sign}(x-\pi), n=16$.
A missile is launched in your direction. Using a range finder, you measure its altitude at the times:\begin{tabular}{c|cccccc} time in sec & 0 & 10 & 20 & 30 & 40 & 50 \\\hline altitude in meters & 200 & 650 & 970 & 1200 & 1375 & 1130\end{tabular}How long until you have to run?
5.6.18. discrete Fourier coefficients. Carefully indicate each step in your analysis.(a) $c_0=c_2=1, c_1=c_3=-1$,(b) $c_0=c_1=c_4=2, c_2=c_6=0, c_3=c_5=c_7=-1$.
A student runs an experiment six times in an attempt to obtain an equation relating two physical quantities $x$ and $y$. For $x=1,2,4,6,8,10$ units, the experiments result in corresponding $y$ values of $3,3,4,6,7,8$ units. Find and graph the following: (a) the least squares line; (b) the least squares quadratic polynomial; $(c)$ the interpolating polynomial. (d) Which do you think is the most likely theoretical model for this data?
Use the Inverse Fast Fourier Transform to reassemble the sampled function data corresponding to the following discrete Fourier coefficients. Carefully indicate each step in your analysis.(a) $c_0=c_2=1, c_1=c_3=-1$,(b) $c_0=c_1=c_4=2, c_2=c_6=0, c_3=c_5=c_7=-1$.
(a) Write down the Taylor polynomials of degrees 2 and 4 at $t=0$ for the function $f(t)=e^t$. (b) Compare their accuracy with the interpolating and least squares polynomials in Examples 5.18 and 5.22 .
Given the values of $\sin t$ at $t=0^{\circ}, 30^{\circ}, 45^{\circ}, 60^{\circ}$, find the following approximations:(a) the least squares linear polynomial; (b) the least squares quadratic polynomial; (c) the quadratic Taylor polynomial at $t=0 ;(d)$ the interpolating polynomial; (e) the cubic Taylor polynomial at $t=0 ;(f)$ Graph each approximation and discuss its accuracy.
Find the quartic (degree 4) polynomial that exactly interpolates the function tan $t$ at the five data points $t_0=0, t_1=.25, t_2=.5, t_3=.75, t_4=1$. Compare the graphs of the two functions over $0 \leq t \leq \frac{1}{2} \pi$.
(a) Find the least squares linear polynomial approximating $\sqrt{t}$ on $[0,1]$, choosing six different exact data values. (b) How much more accurate is the least squares quadratic polynomial based on the same data?
A table of logarithms contains the following entries:\begin{tabular}{c|cccc}$t$ & 1.0 & 2.0 & 3.0 & 4.0 \\\hline $\log _{10} t$ & 0 & .3010 & .4771 & .6021\end{tabular}Approximate $\log _{10} e$ by constructing an interpolating polynomial of (a) degree two using the entries at $x=1.0,2.0$, and $3.0,(b)$ degree three using all the entries.
Let $q(t)$ denote the quadratic interpolating polynomial that goes through the data points $\left(t_0, y_0\right),\left(t_1, y_1\right),\left(t_2, y_2\right)$. (a) Under what conditions does $q(t)$ have a minimum? A maximum? (b) Show that the minimizing/maximizing value is at $t^{\star}=\frac{m_0 s_1-m_1 s_0}{s_1-s_0}$, where $s_0=\frac{y_1-y_0}{t_1-t_0}, s_1=\frac{y_2-y_1}{t_2-t_1}, m_0=\frac{t_0+t_1}{2}, m_1=\frac{t_1+t_2}{2}$.(c) What is $q\left(t^{\star}\right)$ ?
Use the orthogonal sample vectors (5.63) to find the best polynomial least squares fits of degree 1,2 and 3 for the following sets of data:(a)\begin{tabular}{c|cccccccc}$t_i$ & -2 & -1 & 0 & 1 & 2 & & & \\\hline$y_i$ & 7 & 11 & 13 & 18 & 21 & & & \\$t_i$ & -3 & -2 & -1 & 0 & 1 & 2 & 3 \\\hline$y_i$ & -2.7 & -2.1 & -.5 & .5 & 1.2 & 2.4 & 3.2 \\$t_i$ & -3 & -2 & -1 & 0 & 1 & 2 & 3 \\\hline$y_i$ & 60 & 80 & 90 & 100 & 120 & 120 & 130\end{tabular}
(a) Verify the orthogonality of the sample polynomial vectors in (5.63). (b) Construct the next orthogonal sample polynomial $q_4(t)$ and the norm of its sample vector.(c) Use your result to compute the quartic least squares approximation for the data in Example 5.23.
Use the result of Exercise 5.5.27 to find the best approximating polynomial of degree 4 to the data in Exercise 5.5.26.
Justify the fact that the orthogonal sample vector $\mathbf{q}_k$ in (5.62) is a linear combination of only the first $k$ monomial sample vectors.
The formulas (5.63) apply only when the sample times are symmetric around 0 . When the sample points $t_1, \ldots, t_n$ are equally spaced, so $t_{i+1}-t_i=h$ for all $i=1, \ldots, n-1$, then there is a simple trick to convert the least squares problem into a symmetric form.(a) Show that the translated sample points $s_i=t_i-\bar{t}$, where $\bar{t}=\frac{1}{n} \sum_{i=1}^n t_i$ is the average, are symmetric around 0 . (b) Suppose $q(s)$ is the least squares polynomial for the data points $\left(s_i, y_i\right)$. Prove that $p(t)=q(t-\bar{t})$ is the least squares polynomial for the original data $\left(t_i, y_i\right)$. (c) Apply this method to find the least squares polynomials of degrees 1 and 2 for the following data:\begin{tabular}{c|cccccc}$t_i$ & 1 & 2 & 3 & 4 & 5 & 6 \\\hline$y_i$ & -8 & -6 & -4 & -1 & 1 & 3\end{tabular}
Construct the first three orthogonal basis elements for sample points $t_1, \ldots, t_m$ that are in general position.
Use $n+1$ equally spaced data points to interpolate $f(t)=1 /\left(1+t^2\right)$ on an interval $-a \leq t \leq a$ for $a=1,1.5,2,2.5,3$ and $n=2,4,10,20$. Do all intervals exhibit the pathology illustrated in Figure 5.9? If not, how large can $a$ be before the interpolants have poor approximation properties? What happens when the number of interpolation points is taken to be $n=50$ ?
Repeat Exercise 5.5.32 for the hyperbolic secant function $f(t)=\operatorname{sech} t=1 / \cosh t$.
Given $A$ as in (5.50) with $m<n+1$, how would you characterize those polynomials $p(t)$ whose coefficient vector $\mathbf{x}$ lies in ker $A$ ?
(a) Give an example of an interpolating polynomial through $n+1$ points that has degree $<n$. (b) Can you explain, without referring to the explicit formulas, why all the Lagrange interpolating polynomials based on $n+1$ points must have degree equal to $n$ ?
Let $x_1, \ldots, x_n$ be distinct real numbers. Prove that the $n \times n$ matrix $K$ with entries$$k_{i j}=\frac{1-\left(x_i x_j\right)^n}{1-x_i x_j} \text { is positive definite. }$$
Prove the determinant formula $\operatorname{det} A=\Pi_{1 \leq i<j \leq n+1}\left(t_i-t_j\right)$ for the $(n+1) \times(n+1)$ Vandermonde matrix defined in (5.50).
(a) Prove that a polynomial $p(x)=a_0+a_1 x+a_2 x^2+\cdots+a_n x^n$ of degree $\leq n$ vanishes at $n+1$ distinct points, so $p\left(x_1\right)=p\left(x_2\right)=\cdots=p\left(x_{n+1}\right)=0$, if and only if $p(x) \equiv 0$ is the zero polynomial. (b) Prove that the monomials $1, x, x^2, \ldots, x^n$ are linearly independent.(c) Explain why $p(x) \equiv 0$ if and only if all its coefficients $a_0=a_1=\cdots=a_n=0$.
Numerical differentiation: The most common numerical methods for approximating the derivatives of a function are based on interpolation. To approximate the $k^{\text {th }}$ derivative $f^{(k)}\left(x_0\right)$ at a point $x_0$, one replaces the function $f(x)$ by an interpolating polynomial $p_n(x)$ of degree $n \geq k$ based on the nearby points $x_0, \ldots, x_n$ (the point $x_0$ is almost always included as an interpolation point), leading to the approximation $f^{(k)}\left(x_0\right) \approx p_n^{(k)}\left(x_0\right)$. Use this method to construct numerical approximations to (a) $f^{\prime}(x)$ using a quadratic interpolating polynomial based on $x-h, x, x+h$. (b) $f^{\prime \prime}(x)$ with the same quadratic polynomial. (c) $f^{\prime}(x)$ using a quadratic interpolating polynomial based on $x, x+h, x+2 h$. (d) $f^{\prime}(x), f^{\prime \prime}(x), f^{\prime \prime \prime}(x)$ and $f^{(i v)}(x)$ using a quartic interpolating polynomial based on $x-2 h, x-h, x, x+h, x+2 h$. (e) Test your methods by approximating the derivatives of $e^x$ and $\tan x$ at $x=0$ with step sizes $h=\frac{1}{10}, \frac{1}{100}, \frac{1}{1000}, \frac{1}{10000}$. Discuss the accuracies you observe. Can the step size be arbitrarily small? (f) Why do you need $n \geq k$ ?
Numerical integration: Most numerical methods for evaluating a definite integral $\int_a^b f(x) d x$ are based on interpolation. One chooses $n+1$ interpolation points $a \leq x_0<$ $x_1<\cdots<x_n \leq b$ and replaces the integrand by its interpolating polynomial $p_n(x)$ of degree $n$, leading to the approximation $\int_a^b f(x) d x \approx \int_a^b p_n(x) d x$, where the polynomial integral can be done explicitly. Write down the following popular integration rules:(a) Trapezoid Rule: $x_0=a, x_1=b$. (b) Simpson's Rule: $x_0=a, x_1=\frac{1}{2}(a+b), x_2=b$.(c) Simpson's $\frac{3}{8}$ Rule: $x_0=a, x_1=\frac{1}{3}(a+b), x_2=\frac{2}{3}(a+b), x_3=b$.(d) Midpoint Rule: $x_0=\frac{1}{2}(a+b)$. (e) Open Rule: $x_0=\frac{1}{3}(a+b), x_1=\frac{2}{3}(a+b)$.(f) Test your methods for accuracy on the following integrals:(i) $\int_0^1 e^x d x$(ii) $\int_0^\pi \sin x d x$,(iii) $\int_1^e \log x d x$(iv) $\int_0^{\pi / 2} \sqrt{x^3+1} d x$.Note: For more details on numerical differentiation and integration, you are encouraged to consult a basic numerical analysis text, e.g., [8].
Given the values\begin{tabular}{c|ccc}$t_i$ & 0 & .5 & 1 \\\hline$y_i$ & 1 & .5 & .25\end{tabular}construct the trigonometric function of the form $g(t)=a \cos \pi t+b \sin \pi t$ that best approximates the data in the least squares sense.
Find the hyperbolic function $g(t)=a \cosh t+b \sinh t$ that best approximates the data in Exercise 5.5.41.
(a) Find the exponential function of the form $g(t)=a e^t+b e^{2 t}$ that best approximates $t^2$ in the least squares sense based on the sample points $0,1,2,3,4$. (b) What is the least squares error? (c) Compare the graphs on the interval $[0,4]$ - where is the approximation the worst? (d) How much better can you do by including a constant term in $g(t)=a e^t+b e^{2 t}+c ?$
(a) Find the best trigonometric approximation of the form $g(t)=r \cos (t+\delta)$ to $t^2$ using 5 and 9 equally spaced sample points on $[0, \pi]$.(b) Can you answer the question for $g(t)=r_1 \cos \left(t+\delta_1\right)+r_2 \cos \left(2 t+\delta_2\right)$ ?
A trigonometric polynomial of degree $n$ is a function of the form$$p(t)=a_0+a_1 \cos t+b_1 \sin t+a_2 \cos 2 t+b_2 \sin 2 t+\cdots+a_n \cos n t+b_n \sin n t,$$where $a_0, a_1, b_1, \ldots, a_n, b_n$ are the coefficients. Find the trigonometric polynomial of degree $n$ that is the least squares approximation to the function $f(t)=1 /\left(1+t^2\right)$ on the interval $[-\pi, \pi]$ based on the $k$ equally spaced data points $t_j=-\pi+\frac{2 \pi j}{k}, j=0, \ldots, k-1$, (omitting the right-hand endpoint), when (a) $n=1, k=4$, (b) $n=2, k=8$, (c) $n=2, k=16,(d) n=3, k=16$. Compare the graphs of the trigonometric approximant and the function, and discuss. (e) Why do we not include the right-hand endpoint $t_k=\pi$ ?
The sinc functions are defined as $S_0(x)=\frac{\sin (\pi x / h)}{\pi x / h}$, while $S_j(x)=S_0(x-j h)$ whenever $h>0$ and $j$ is an integer. We will interpolate a function $f(x)$ at the mesh points $x_j=j h, j=0, \ldots, n$, by a linear combination of sinc functions: $S(x)=$ $c_0 S_0(x)+\cdots+c_n S_n(x)$. What are the coefficients $c_j$ ? Graph and discuss the accuracy of the sinc interpolant for the functions $x^2$ and $\frac{1}{2}-\left|x-\frac{1}{2}\right|$ on the interval $[0,1]$ using $h=.25, .1$, and .025 .
Approximate the function $f(t)=\sqrt[3]{t}$ using the least squares method based on the $\mathrm{L}^2$ norm on the interval $[0,1]$ by (a) a straight line; (b) a parabola; (c) a cubic polynomial.
Approximate the function $f(t)=\frac{1}{8}(2 t-1)^3+\frac{1}{4}$ by a quadratic polynomial on the interval $[-1,1]$ using the least squares method based on the $\mathrm{L}^2$ norm. Compare the graphs. Where is the error the largest?
For the function $f(t)=\sin t$ determine the approximating linear and quadratic polynomials that minimize the least squares error based on the $L^2$ norm on $\left[0, \frac{1}{2} \pi\right]$.
Find the quartic (degree 4) polynomial that best approximates the function $e^t$ on the interval $[0,1]$ by minimizing the $\mathrm{L}^2$ error $(5.72)$.
(a) Find the quadratic interpolant to $f(x)=x^5$ on the interval $[0,1]$ based on equally spaced data points. (b) Find the quadratic least squares approximation based on the data points $0, .25, .5, .75,1$. (c) Find the quadratic least squares approximation with respect to the $\mathrm{L}^2$ norm. (d) Discuss the strengths and weaknesses of each approximation.
Let $f(x)=x$. Find the trigonometric function of the form $g(x)=a+b \cos x+c \sin x$ that minimizes the $\mathrm{L}^2$ error $\|g-f\|=\sqrt{\int_{-\pi}^\pi[g(x)-f(x)]^2 d x}$.
Let $g_1(t), \ldots, g_n(t)$ be prescribed, linearly independent functions. Explain how to best approximate a function $f(t)$ by a linear combination $c_1 g_1(t)+\cdots+c_n g_n(t)$ when the least squares error is measured in a weighted $\mathrm{L}^2$ norm $\|f\|_w^2=\int_a^b f(t)^2 w(t) d t$ with weight function $w(t)>0$.
(a) Find the quadratic least squares approximation to $f(t)=t^5$ on the interval $[0,1]$ with weights $(i) w(t)=1$, (ii) $w(t)=t$, (iii) $w(t)=e^{-t}$. (b) Compare the errors - which gives the best result over the entire interval?
Let $f_a(x)=\sqrt{\frac{a}{1+a^4 x^2}}$. Prove that (a) $\left\|f_a\right\|_2=\sqrt{\frac{\pi}{a}}$, where $\|\cdot\|_2$ denotes the $\mathrm{L}^2$ norm on $(-\infty, \infty)$. (b) $\left\|f_a\right\|_{\infty}=\sqrt{a}$, where $\|\cdot\|_{\infty}$ denotes the $\mathrm{L}^{\infty}$ norm on $(-\infty, \infty)$. (c) Use this example to explain why having a small least squares error does not necessarily mean that the functions are everywhere close.
Find the plane $z=\alpha+\beta x+\gamma y$ that best approximates the following functions on the square $S=\{0 \leq x \leq 1,0 \leq y \leq 1\}$ using the $\mathrm{L}^2$ norm $\|f\|^2=\iint_S|f(x, y)|^2 d x d y$ to measure the least squares error:(a) $x^2+y^2$,(b) $x^3-y^3$,(c) $\sin \pi x \sin \pi y$.
Find the radial polynomial $p(x, y)=a+b r+c r^2$, where $r^2=x^2+y^2$, that best approximates the function $f(x, y)=x$ using the $\mathrm{L}^2$ norm on the unit disk $D=\{r \leq 1\}$ to measure the least squares error.
Use the Legendre polynomials to find the best (a) quadratic, and (b) cubic approximation to $t^4$, based on the $\mathrm{L}^2$ norm on $[-1,1]$.
Repeat Exercise 5.5 .58 using the $\mathrm{L}^2$ norm on $[0,1]$.
Find the best cubic approximation to $f(t)=e^t$ based on the $\mathrm{L}^2$ norm on $[0,1]$.
Find the (a) linear, (b) quadratic, and (c) cubic polynomials $q(t)$ that minimize the following integral: $\int_0^1\left[q(t)-t^3\right]^2 d t$. What is the minimum value in each case?
Find the best quadratic and cubic approximations for $\sin t$ for the $\mathrm{L}^2$ norm on $[0, \pi]$ by using an orthogonal basis. Graph your results and estimate the maximal error.
Answer Exercise 5.5.60 when $f(t)=\sin t$. Use a computer to numerically evaluate the integrals.
Find the degree 6 least squares polynomial approximation to $e^t$ on the interval $[-1,1]$ under the $\mathrm{L}^2$ norm.
(a) Use the polynomials and weighted norm from Exercise 4.5.12 to find the quadratic least squares approximation to $f(t)=1 / t$. In what sense is your quadratic approximation "best"? (b) Now find the best approximating cubic polynomial. (c) Compare the graphs of the quadratic and cubic approximants with the original function and discuss what you observe.
Use the Laguerre polynomials (4.68) to find the quadratic and cubic polynomial least squares approximation to $f(t)=\tan ^{-1} t$ relative to the weighted inner product (4.66). Use a computer to evaluate the coefficients. Graph your result and discuss what you observe.
Find and graph the natural cubic spline interpolant for the following data:(a)\begin{tabular}{c|ccc}$x$ & -1 & 0 & 1 \\\hline$y$ & -2 & 1 & -1\end{tabular}(b)\begin{tabular}{l|llll}$x$ & 0 & 1 & 2 & 3 \\\hline$y$ & 1 & 2 & 0 & 1\end{tabular}(c)\begin{tabular}{l|lll}$x$ & 1 & 2 & 4 \\\hline$y$ & 3 & 0 & 2\end{tabular}(d)\begin{tabular}{c|ccccc}$x$ & -2 & -1 & 0 & 1 & 2 \\\hline$y$ & 5 & 2 & 3 & -1 & 1\end{tabular}
Repeat Exercise 5.5.67 when the spline has homogeneous clamped boundary conditions.
Find and graph the periodic cubic spline that interpolates the following data:(a)\begin{tabular}{l|llll}$x$ & 0 & 1 & 2 & 3 \\\hline$y$ & 1 & 0 & 0 & 1\end{tabular}(b)(c)\begin{tabular}{l|lllll}$x$ & 0 & 1 & 2 & 3 & 4 \\\hline$y$ & 1 & 0 & 0 & 0 & 1\end{tabular}\begin{tabular}{l|llll}$x$ & 0 & 1 & 2 & 3 \\\hline$y$ & 1 & 2 & 0 & 1\end{tabular}(d)\begin{tabular}{c|ccccc}$x$ & -2 & -1 & 0 & 1 & 2 \\\hline$y$ & 1 & 2 & -2 & -1 & 1\end{tabular}
(a) Given the known values of $\sin x$ at $x=0^{\circ}, 30^{\circ}, 45^{\circ}, 60^{\circ}$, construct the natural cubic spline interpolant. (b) Compare the accuracy of the spline with the least squares and interpolating polynomials you found in Exercise 5.5.21.
(a) Using the exact values for $\sqrt{x}$ at $x=0, \frac{1}{4}, \frac{9}{16}, 1$, construct the natural cubic spline interpolant. (b) What is the maximal error of the spline on the interval $[0,1]$ ? (c) Compare the error with that of the interpolating cubic polynomial you found in Exercise 5.5.23. Which is the better approximation? (d) Answer part (d) using the cubic least squares approximant based on the $\mathrm{L}^2$ norm on $[0,1]$.
According to Figure 5.9, the interpolating polynomials for the function $1 /\left(1+x^2\right)$ on the interval $[-3,3]$ based on equally spaced mesh points are very inaccurate near the ends of the interval. Does the natural spline interpolant based on the same 3,5 , and 11 data points exhibit the same inaccuracy?
(a) Draw outlines of the block capital letters I, C, S, and Y on a sheet of graph paper. Fix several points on the graphs and measure their $x$ and $y$ coordinates. (b) Use periodic cubic splines $x=u(t), y=v(t)$ to interpolate the coordinates of the data points using equally spaced nodes for the parameter values $t_k$. Graph the resulting spline letters, and discuss how the method could be used in font design. To get nicer results, you may wish to experiment with different numbers and locations for the points.
Repeat Exercise 5.5.73, using the Lagrange interpolating polynomials instead of splines to parameterize the curves. Compare the two methods and discuss advantages and disadvantages.
Let $x_0<x_1<\cdots<x_n$. For each $j=0, \ldots, n$, the $j^{\text {th }}$ cardinal spline $C_j(x)$ is defined to be the natural cubic spline interpolating the Lagrange data$$y_0=0, \quad y_1=0, \quad \ldots \quad y_{j-1}=0, \quad y_j=1, \quad y_{j+1}=0, \quad \ldots \quad y_n=0 .$$(a) Construct and graph the natural cardinal splines corresponding to the nodes $x_0=0$, $x_1=1, x_2=2$, and $x_3=3$. (b) Prove that the natural spline that interpolates the data $y_0, \ldots, y_n$ can be uniquely written as a linear combination $u(x)=y_0 C_0(x)+y_1 C_1(x)+$ $\cdots+y_n C_n(x)$ of the cardinal splines. (c) Explain why the space of natural splines on $n+1$ nodes is a vector space of dimension $n+1$. (d) Discuss briefly what modifications are required to adapt this method to periodic and to clamped splines.
A bell-shaped or $B$-spline $u=\beta(x)$ interpolates the data$$\beta(-2)=0, \quad \beta(-1)=1, \quad \beta(0)=4, \quad \beta(1)=1, \quad \beta(2)=0 .$$(a) Find the explicit formula for the natural $B$-spline and plot its graph. (b) Show that $\beta(x)$ also satisfies the homogeneous clamped boundary conditions $u^{\prime}(-2)=u^{\prime}(2)=0$.(c) Show that $\beta(x)$ also satisfies the periodic boundary conditions. Thus, for this particular interpolation problem, the natural, clamped, and periodic splines happen to coincide.(d) Show that $\beta^{\star}(x)=\left\{\begin{array}{ll}\beta(x), & -2 \leq x \leq 2 \\ 0, & \text { otherwise, }\end{array}\right.$ defines a $\mathrm{C}^2$ spline on every interval $[-k, k]$.
Let $\beta(x)$ denote the $B$-spline function of Exercise 5.5.76. Assuming $n \geq 4$, let $P_n$ denote the vector space of periodic cubic splines based on the integer nodes $x_j=j$ for $j=0, \ldots, n$. (a) Prove that the $B$-splines $B_j(x)=\beta((x-j-m) \bmod n+m)$,