• Home
  • Textbooks
  • Linear Algebra in Action
  • Inner product spaces and orthogonality

Linear Algebra in Action

Harry Dym

Chapter 8

Inner product spaces and orthogonality - all with Video Answers

Educators

Chapter Questions

11:55

Problem 1

Let $\mathcal{U}$ be an inner product space over $F$ and let $\mathbf{u} \in \mathcal{U}$. Show that
$$
\langle\mathbf{u}, \mathbf{v}\rangle=0 \text { for every } \mathbf{v} \in \mathcal{U} \Longleftrightarrow \mathbf{u}=\mathbf{0}
$$
and (consequently)
$$
\left\langle\mathbf{u}_1, \mathbf{v}\right\rangle=\left\langle\mathbf{u}_2, \mathbf{v}\right\rangle \text { for every } \mathbf{v} \in \mathcal{U} \Longleftrightarrow \mathbf{u}_1=\mathbf{u}_2 \text {. }
$$

The symbol $\langle\mathbf{x}, \mathbf{y}\rangle_{s t}$, which is defined for $\mathbf{x}, \mathbf{y} \in \mathbb{F}^n$ by the formula
$$
\langle\mathbf{x}, \mathbf{y}\rangle_{s t}=\mathbf{y}^H \mathbf{x}=\sum_{i=1}^n \overline{y_i} x_i
$$
will be used on occasion to denote the standard inner product on $\mathbb{F}^n$. The conjugation in this formula can be dropped if $\mathbf{x}, \mathbf{y} \in \mathbb{R}^n$. It is important to bear in mind that there are many other inner products that can be imposed on $\mathbb{F}^n$ :

Chris Trentman
Chris Trentman
Numerade Educator
13:34

Problem 2

Show that if $B \in \mathbb{C}^{n \times n}$ is invertible, then the formula
$$
\langle\mathbf{x}, \mathbf{y}\rangle=(B \mathbf{y})^H B \mathbf{x}
$$
defines an inner product on $\mathbb{C}^n$.

Anthony Ramos
Anthony Ramos
Numerade Educator

Problem 3

Let $\mathcal{U}$ denote the set of continuous complex valued functions $f(t)$ on the finite closed interval $[a, b]$.
(a) Show that $\mathcal{U}$ is a vector space over $\mathbb{C}$ with respect to the natural rules of addition and multiplication by constants. Identify the zero element.
(b) Show that $\mathcal{U}$ is a normed linear space with respect to the norm $\|f\|=$ $\left\{\int_a^b|f(t)|^2 d t\right\}^{1 / 2}$.
(c) Show that $\mathcal{U}$ is an inner product space with respect to the inner product $\langle f, g\rangle=\int_a^b f(t) \overline{g(t)} d t$.

Check back soon!
03:57

Problem 4

Show that if $f(t)$ and $g(t)$ are continuous complex valued functions $f(t)$ on the finite closed interval $[a, b]$, then
$$
\left|\int_a^b f(t) \overline{g(t)} d t\right|^2 \leq \int_a^b|f(t)|^2 d t \int_a^b|g(t)|^2 d t
$$
with equality if and only if there exists a pair of constants $\alpha, \beta \in \mathbb{C}$ such that $\alpha f(t)+\beta g(t)=0$ for every point $t \in[a, b]$.

Mengchun Cai
Mengchun Cai
Numerade Educator
06:33

Problem 5

Show that the space $\mathcal{U}=\mathbb{C}^{p \times q}$ endowed with the inner product $\langle A, B\rangle=$ trace $\left\{B^H A\right\}$ is a $p q$-dimensional inner product space.

Chris Trentman
Chris Trentman
Numerade Educator
01:24

Problem 6

Verify formula (8.4) in the setting of Lemma 8.2.

Manik Pulyani
Manik Pulyani
Numerade Educator

Problem 7

Verify formula (8.5) in the setting of Lemma 8.2 .

Check back soon!
01:14

Problem 8

Let $\mathcal{U}$ be a normed linear space, and for $\mathbf{u}, \mathbf{v} \in \mathcal{U}$, let $f(\mathbf{u})=$ $\|\mathbf{u}+\mathbf{v}\|$. Show that $\left|f\left(\mathbf{u}_2\right)-f\left(\mathbf{u}_1\right)\right| \leq\left\|\mathbf{u}_2-\mathbf{u}_1\right\|$ for any two elements $\mathbf{u}_1, \mathbf{u}_2 \in \mathcal{U}$

Carson Merrill
Carson Merrill
Numerade Educator
01:41

Problem 9

Let $\mathcal{U}=\mathbb{C}^n$ endowed with the standard inner product. Show that if $n \geq 2$ and if $\mathbf{u}=\mathbf{e}_1$ and $\mathbf{v}=\mathbf{e}_2$, where $\mathbf{e}_j$ denotes the $j^{\prime}$ th column of $I_n$, and if $1 \leq s \leq \infty$, then
$$
\|\mathbf{u}+\mathbf{v}\|_s^2+\|\mathbf{u}-\mathbf{v}\|_s^2=2\|\mathbf{u}\|_s^2+2\|\mathbf{v}\|_s^2 \Longleftrightarrow s=2 .
$$

Rukhmani Jain
Rukhmani Jain
Numerade Educator
02:35

Problem 10

Show that every orthogonal sum decomposition is a direct sum decomposition and give an example of a direct sum decomposition that is not an orthogonal decomposition.

Urvashi Arora
Urvashi Arora
Numerade Educator

Problem 11

Show that if $\left\{\mathbf{u}_1, \ldots, \mathbf{u}_k\right\}$ is an orthogonal family of nonzero vectors in an inner product space $\mathcal{U}$ over $\mathbb{F}$, then $\mathbf{u}_1, \ldots, \mathbf{u}_k$ are linearly independent.

Check back soon!

Problem 12

Verify formula (8.7).

Check back soon!

Problem 13

Verify Corollary 8.6.

Check back soon!
05:54

Problem 14

Verify the identification (8.9) in the preceding example by checking that for all rank one matrices $A=\mathbf{x y}^H$, with $\mathbf{x} \in \mathbb{C}^p$ and $\mathbf{y} \in \mathbb{C}^q$,
$$
\langle S A, \mathbf{v}\rangle_{\mathcal{V}}=\mathbf{y}^H \mathbf{u} \mathbf{v}^H \mathbf{x} \text { and }\left\langle A, S^* \mathbf{v}\right\rangle_{\mathcal{U}}=\mathbf{y}^H\left(S^* \mathbf{v}\right)^H \mathbf{x} .
$$

Chris Trentman
Chris Trentman
Numerade Educator
View

Problem 15

Let $\mathcal{U}=\mathbb{C}^n$ equipped with the inner product $\langle\mathbf{u}, \mathbf{v}\rangle_{\mathcal{U}}=$ $\sum_{j=1}^n j \overline{v_j} u_j$ for vectors $\mathbf{u}, \mathbf{v} \in \mathbb{C}^n$ with components $u_1, \ldots, u_n$ and $v_1, \ldots, v_n$, respectively. Find the adjoint $A^*$ of a matrix $A \in \mathbb{C}^{n \times n}$ with respect to this inner product.

Michelle Z.
Michelle Z.
Numerade Educator

Problem 16

Complete the proof of Lemma 8.8.

Check back soon!
02:37

Problem 17

Let $T$ be a linear transformation from a finite dimensional inner product space $\mathcal{U}$ into itself, and let $\mathcal{V}$ be a subspace of $\mathcal{U}$. Show that
$$
T \mathcal{V} \subseteq \mathcal{V} \Longleftrightarrow T^* \mathcal{V}^{\perp} \subseteq \mathcal{V}^{\perp}
$$

ET
Ed Tam
Numerade Educator

Problem 18

Let $\mathcal{U}$ be an inner product space over $\mathbb{F}$, let $\mathbf{y} \in \mathcal{U}$ and let $f(\mathbf{x})=\langle\mathbf{x}, \mathbf{y}\rangle_{\mathcal{U}} \quad$ for every $\quad \mathbf{x} \in \mathcal{U}$.

Show that $f$ is a linear functional on $\mathcal{U}$ and that
$\|\mathbf{y}\|_{\mathcal{U}}=\max \left\{|f(\mathbf{x})|: \mathbf{x} \in \mathcal{U}\right.$ and $\left.\|\mathbf{x}\|_{\mathcal{U}}=1\right\}$

Check back soon!

Problem 19

Let $\mathcal{U}$ be a finite dimensional inner product space over $F$, with basis $\left\{\mathbf{u}_1, \ldots, \mathbf{u}_n\right\}$ and Gram matrix $G$, and let $f$ be a linear functional on $\mathcal{U}$. Show that
$$
f(\mathbf{x})=\langle\mathbf{x}, \mathbf{y}\rangle\rangle_{\mathcal{U}} \text { for every } \mathbf{x} \in \mathcal{U},
$$
where $\mathbf{y}=\sum_{i=1}^n d_i \mathbf{u}_i$ and $d_i$ is the $i^{\prime}$ th component of the vector
$$
\mathbf{d}=G^{-1}\left[\begin{array}{lll}
f\left(\mathbf{u}_1\right) & \cdots & f\left(\mathbf{u}_n\right)
\end{array}\right]^H .
$$

Check back soon!
03:06

Problem 20

Show that $\left|\lambda_j\right|=1$ for each eigenvalue $\lambda_j$ of the unitary transformation $T$ considered in Theorem 8.12.

Nick Johnson
Nick Johnson
Numerade Educator
02:19

Problem 21

Let $\mathbf{u}_1$ and $\mathbf{u}_2$ be a pair of orthonormal vectors in an inner product space $\mathcal{U}$ over $\mathbb{F}$ and let $\alpha \in \mathbb{F}$. Show that the transformation $P$ that is defined by the formula $P \mathbf{u}=\left\langle\mathbf{u}, \mathbf{u}_1+\alpha \mathbf{u}_2\right\rangle \mathcal{U} \mathbf{u}_1$ is a projection but is not an orthogonal projection unless $\alpha=0$.

Nick Johnson
Nick Johnson
Numerade Educator

Problem 22

Verify assertions (4) and (5) in Lemma 8.14.

Check back soon!
02:07

Problem 23

Let $\{\mathbf{v}, \mathbf{w}\}$ be a basis for a vector space $\mathcal{U}$ over $\mathbb{F}$. Find the projection $P \mathcal{V} \mathbf{u}$ of the vector $\mathbf{u}=2 \mathbf{v}+3 \mathbf{w}$ onto the space $\mathcal{V}$ with respect to each of the following direct sum decompositions: $\mathcal{U}=\mathcal{V}+\mathcal{W}$ and $\mathcal{U}=\mathcal{V}+\mathcal{W}_1$, when $\mathcal{V}=\operatorname{span}\{\mathbf{v}\}, \mathcal{W}=\operatorname{span}\{\mathbf{w}\}$ and $\mathcal{W}_1=\operatorname{span}\{\mathbf{w}+\mathbf{v}\}$

Amy Jiang
Amy Jiang
Numerade Educator

Problem 24

Show that in the setting of Lemma 8.15,
$$
\left\|\mathbf{u}-\sum_{j=1}^k c_j \mathbf{v}_j\right\|^2 \geq\|\mathbf{u}\|^2-\mathbf{b}^H G^{-1} \mathbf{b}
$$
with equality if and only if $c_j=\left(G^{-1} \mathbf{b}\right)_j$ for $j=1, \ldots, k$.

Check back soon!
04:22

Problem 25

Verify directly that the transformation $P \mathcal{V}$ defined by formula (8.23) for $\mathcal{U}=\mathbb{C}^n$ endowed with the standard inner product meets the following conditions:
(1) $\left(P_{\mathcal{V}}\right)^2=P_{\mathcal{V}}$.
(2) $\left(P_{\mathcal{V}}\right)^H=P_{\mathcal{V}}$.
(3) $P_{\mathcal{\nu}} \mathbf{v}_j=\mathbf{v}_j$ for $j=1, \ldots, k$.
(4) $P_{\mathcal{V}} \mathbf{u}=0$ if $\mathbf{u} \in \mathcal{V}^{\perp}$, computed with respect to the standard inner product.

WM
William Mead
Numerade Educator

Problem 26

Calculate the norm of the projection $P$ that is defined in Exercise 8.21 .

Check back soon!
01:46

Problem 27

Show that if $P$ is a nonzero projection matrix, then:
(a) $\|P\|=1$ if $P$ is an orthogonal projection matrix.
(b) $\|P\|$ can be very large if $P$ is not an orthogonal projection matrix.

Victor Salazar
Victor Salazar
Numerade Educator
05:31

Problem 28

Let
$$
\begin{aligned}
& \qquad\left[\begin{array}{lllllll}
\mathbf{u}_1 & \mathbf{u}_2 & \mathbf{u}_3 & \mathbf{u}_4 & \mathbf{u}_5 & \mathbf{u}_6
\end{array}\right]=\left[\begin{array}{rrrrrr}
1 & 1 & 1 & 2 & 3 & 4 \\
2 & 0 & 4 & 1 & 5 & 0 \\
1 & 1 & 1 & 0 & 1 & 0 \\
0 & 1 & -1 & 0 & -1 & 1
\end{array}\right] \text {, } \\
& \text { and let } \mathcal{U}=\operatorname{span}\left\{\mathbf{u}_1, \mathbf{u}_2, \mathbf{u}_3, \mathbf{u}_4\right\}, \mathcal{V}=\operatorname{span}\left\{\mathbf{u}_1, \mathbf{u}_2, \mathbf{u}_3\right\}, \mathcal{W}_1=\operatorname{span}\left\{\mathbf{u}_4\right\} \\
& \text { and } \mathcal{W}_2=\operatorname{span}\left\{\mathbf{u}_5\right\}
\end{aligned}
$$
(a) Find a basis for the vector space $\mathcal{V}$.
(b) Show that $\mathcal{U}=\mathcal{V}+\mathcal{W}_1$ and $\mathcal{U}=\mathcal{V}+\mathcal{W}_2$.
(c) Find the projection of the vector $\mathbf{u}_6$ onto the space $\mathcal{V}$ with respect to the first direct sum decomposition.
(d) Find the projection of the vector $\mathbf{u}_6$ onto the space $\mathcal{V}$ with respect to the second direct sum decomposition.
(e) Find the orthogonal projection of the vector $\mathbf{u}_6$ onto the space $\mathcal{V}$.

Anthony Ramos
Anthony Ramos
Numerade Educator

Problem 29

Show that no matter how large you choose $k$, the family $\varphi_j(t)=e^{j 2 \pi i t}, j=1, \ldots, k$, is not a basis for the space $\mathcal{U}$ considered just above.

Check back soon!
01:41

Problem 30

Show that in the setting of Lemma 8.19
$$
\sum_{j=1}^n\left\|S \mathbf{u}_j\right\|_{\mathcal{U}}^2=\sum_{j=1}^n\left\|S \mathbf{w}_j\right\|_{\mathcal{U}}^2=\sum_{j=1}^n\left\|S^* \mathbf{u}_j\right\|_{\mathcal{U}}^2
$$

Rukhmani Jain
Rukhmani Jain
Numerade Educator
07:23

Problem 31

Show that if $\left\{\mathbf{u}_1, \ldots, \mathbf{u}_k\right\}$ is a set of $k$ linearly independent vectors in $\mathrm{F}^n$, then there exists an invertible upper triangular matrix $B \in$ $\mathbb{F}^{k \times k}$ such that the matrix $V=\left[\begin{array}{lll}\mathbf{u}_1 & \cdots & \mathbf{u}_k\end{array}\right] B$ has orthonormal columns.

Anthony Ramos
Anthony Ramos
Numerade Educator
01:42

Problem 32

Show that if $A \in \mathbb{C}^{n \times n}$ is invertible, then there exists an invertible upper triangular matrix $B$ such that $A B$ is unitary and an invertible lower triangular matrix $C$ such that $C A$ is unitary. [HINT: Exploit Exercise 8.31.]

Nick Johnson
Nick Johnson
Numerade Educator
01:42

Problem 33

Show that if $A \in \mathbb{R}^{n \times n}$ is invertible, then there exist an invertible upper triangular matrix $B$ such that $A B$ is an orthogonal matrix and an invertible lower triangular matrix $C$ such that $C A$ is an orthogonal matrix.

Nick Johnson
Nick Johnson
Numerade Educator
04:21

Problem 34

Find a set of three polynomials $p_0(t)=a, p_1(t)=b+c t$, and $p_3(t)=d+e t+f t^2$ with real coefficients $a, b, c, d, e, f$ so that they form an orthonormal set with respect to the real inner product $\langle f, g\rangle=\int_0^2 f(t) g(t) d t$.

Chris Trentman
Chris Trentman
Numerade Educator

Problem 35

Show that $A \in \mathbb{C}^{n \times n}$ is a Toeplitz matrix if and only if $Z_n A$ is a Hankel matrix.

Check back soon!

Problem 36

Show that if $A \in \mathbb{C}^{n \times n}$ is a Hankel matrix with $a_{i j}=$ $\beta_{i+j-1}$ for $i, j=1, \ldots, n$, then, in terms of the matrices $Z=Z_n$ and $N=N_n$ defined in formula (8.30),
$$
A=\sum_{j=1}^n \beta_j Z\left(N^T\right)^{n-j}+\sum_{j=1}^{n-1} \beta_{n+j} Z N^j .
$$

Check back soon!
09:21

Problem 37

Show that if $A \in \mathbb{C}^{n \times n}$ is a Toeplitz matrix with $a_{i j}=\alpha_{i-j}$, then, in terms of the matrices $Z=Z_n$ and $N=N_n$ defined in formula (8.30),
$$
A=\sum_{i=0}^{n-1} \alpha_{-i} N^i+\sum_{i=1}^{n-1} \alpha_i\left(N^T\right)^i
$$

Suman Saurav Thakur
Suman Saurav Thakur
Numerade Educator

Problem 38

The $n \times n$ Hankel matrix $H_n$ with entries $h_{i j}=1 /(i+j+1)$ for $i, j=0, \ldots, n-1$ is known as the Hilbert matrix. Show that the Hilbert matrix is invertible. [HINT: $\int_0^1 x^i x^j d x=1 /(i+j+1)$ ]

Check back soon!
06:33

Problem 39

Show that if $H_n$ denotes the $n \times n$ Hankel matrix introduced in Exercise 8.38 and if $\mathbf{a} \in \mathbb{C}^n$ and $\mathbf{b} \in \mathbb{C}^n$ are vectors with components $a_0, \ldots, a_{n-1}$ and $b_0, \ldots, b_{n-1}$, respectively, then
$$
\left\langle H_n \mathbf{a}, \mathbf{b}\right\rangle_{s t}=\frac{1}{2 \pi} \int_0^{2 \pi}\left(\sum_{k=0}^{n-1} \overline{b_k} e^{-i k t}\right)\left(i e^{-i t}(\pi-t)\right)\left(\sum_{j=0}^{n-1} a_j e^{-i j t}\right) d t .
$$

Chris Trentman
Chris Trentman
Numerade Educator

Problem 40

Show that if $H_n$ denotes the $n \times n$ Hankel matrix introduced in Exercise 8.38, then $\left\|H_n\right\|_{2,2}<\pi$.

Check back soon!

Problem 41

Justify formula (8.34). [HINT: First check that the integral is equal to $\left\langle\pi_n, S_n \pi_n\right\rangle_{\mathcal{U}}$.]

Check back soon!

Problem 42

Show that in the setting of this section,
$$
\mathfrak{P}_n x^{n+1}=\sum_{j=0}^n b_j x^j, \quad \text { where }\left[\begin{array}{c}
b_0 \\
\vdots \\
b_n
\end{array}\right]=H_n^{-1}\left[\begin{array}{c}
h_{n+1} \\
\vdots \\
h_{2 n+1}
\end{array}\right],
$$
$h_j$ and the $(n+1) \times(n+1)$ Hankel matrix $H_n$ are defined in (8.35).

Check back soon!

Problem 43

Show that in the setting of this section,
$$
S_n \sum_{j=0}^n a_j x^j=\sum_{j=0}^n b_j x^j \Longleftrightarrow\left[\begin{array}{c}
b_0 \\
\vdots \\
b_n
\end{array}\right]=H_n^{-1} K_n\left[\begin{array}{c}
a_0 \\
\vdots \\
a_n
\end{array}\right],
$$
where the $(n+1) \times(n+1)$ Hankel matrices $H_n$ and $K_n$ are defined in (8.35).

Check back soon!
10:06

Problem 44

Show that if $\mathbf{a}$ and $\mathbf{b}$ are vectors with components $a_0, \ldots, a_n$ and $b_0, \ldots, b_n$, respectively, then
$$
\left\langle\sum_{j=0}^n a_j x^j, \sum_{k=0}^n b_k x^k\right\rangle_{\mathcal{U}}=\mathbf{b}^H H_n \mathbf{a} \text { and }\left\langle S_n \sum_{j=0}^n a_j x^j, \sum_{k=0}^n b_k x^k\right\rangle_{\mathcal{U}}=\mathbf{b}^H K_n \mathbf{a} \text {, }
$$
where the $(n+1) \times(n+1)$ Hankel matrices $H_n$ and $K_n$ are defined in (8.35).

Chris Trentman
Chris Trentman
Numerade Educator