Matrix Algebra (Econometric Exercises)

Karim M. Abadir, Jan R. Magnus

Chapter 12 Matrix inequalities - all with Video Answers

Educators

Chapter Questions

Problem 1

(Cauchy-Schwarz inequality, once more)
Let $\boldsymbol{a}$ and $\boldsymbol{b}$ be two $n \times 1$ vectors. Then, $$ \left(\boldsymbol{a}^{\prime} \boldsymbol{b}\right)^2 \leq\left(\boldsymbol{a}^{\prime} \boldsymbol{a}\right)\left(\boldsymbol{b}^{\prime} \boldsymbol{b}\right), $$
with equality if and only if $\boldsymbol{a}$ and $\boldsymbol{b}$ are linearly dependent. Prove this result by considering:
(a) the matrix $\boldsymbol{A}=\boldsymbol{a} \boldsymbol{b}^{\prime}-\boldsymbol{b} \boldsymbol{a}^{\prime}$;
(b) the matrix $\boldsymbol{A}=\boldsymbol{I}_n-\left(1 / \boldsymbol{b}^{\prime} \boldsymbol{b}\right) \boldsymbol{b} \boldsymbol{b}^{\prime}$ for $\boldsymbol{b} \neq \mathbf{0}$;
(c) the $2 \times 2$ matrices $\boldsymbol{A}_i:=\boldsymbol{c}_i \boldsymbol{c}_i^{\prime}$, where $\boldsymbol{c}_i^{\prime}:=\left(a_i: b_i\right)$ denotes the $i$-th row of $\boldsymbol{C}:=(\boldsymbol{a}: \boldsymbol{b})$.

Check back soon!

Problem 2

(Bound for $\left.a_{i j}\right) \quad$ Let $\boldsymbol{A}:=\left(a_{i j}\right)$ be a positive semidefinite $n \times n$ matrix.
(a) Show that $\left(\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{y}\right)^2 \leq\left(\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}\right)\left(\boldsymbol{y}^{\prime} \boldsymbol{A} \boldsymbol{y}\right)$, with equality if and only if $\boldsymbol{A} \boldsymbol{x}$ and $\boldsymbol{A} \boldsymbol{y}$ are collinear.
(b) Hence, show that $\left|a_{i j}\right| \leq \max \left\{a_{11}, \ldots, a_{n n}\right\}$ for all $i$ and $j$. (Compare Exercise 8.7.)

Check back soon!

Problem 3

(Bergstrom's inequality) Let $\boldsymbol{A}$ be positive definite.
(a) Show that $\left(\boldsymbol{x}^{\prime} \boldsymbol{y}\right)^2 \leq\left(\boldsymbol{x}^{\prime} \boldsymbol{A}^{-1} \boldsymbol{x}\right)\left(\boldsymbol{y}^{\prime} \boldsymbol{A} \boldsymbol{y}\right)$, with equality if and only if $\boldsymbol{A}^{-1} \boldsymbol{x}$ and $\boldsymbol{y}$ are collinear.
(b) For given $\boldsymbol{x} \neq \mathbf{0}$, define $\psi(\boldsymbol{A}):=\left(\boldsymbol{x}^{\prime} \boldsymbol{A}^{-1} \boldsymbol{x}\right)^{-1}$. Show that
$$ \psi(\boldsymbol{A})=\min _{\boldsymbol{y}} \frac{\boldsymbol{y}^{\prime} \boldsymbol{A} \boldsymbol{y}}{\left(\boldsymbol{y}^{\prime} \boldsymbol{x}\right)^2} $$
(c) Use (b) to prove that $$
\boldsymbol{x}^{\prime}(\boldsymbol{A}+\boldsymbol{B})^{-1} \boldsymbol{x} \leq \frac{\left(\boldsymbol{x}^{\prime} \boldsymbol{A}^{-1} \boldsymbol{x}\right)\left(\boldsymbol{x}^{\prime} \boldsymbol{B}^{-1} \boldsymbol{x}\right)}{\boldsymbol{x}^{\prime}\left(\boldsymbol{A}^{-1}+\boldsymbol{B}^{-1}\right) \boldsymbol{x}}
$$
for any two positive definite matrices $\boldsymbol{A}$ and $\boldsymbol{B}$ and $\boldsymbol{x} \neq \mathbf{0}$ (Bergstrom).

Check back soon!

Problem 4

(Cauchy's inequality)
(a) Show that
$$
\left(\sum_{i=1}^n x_i\right)^2 \leq n \sum_{i=1}^n x_i^2
$$
with equality if and only if $x_1=x_2=\cdots=x_n$ (Cauchy).
(b) If all eigenvalues of $\boldsymbol{A}$ are real, show that $|(1 / n) \operatorname{tr} \boldsymbol{A}| \leq\left((1 / n) \operatorname{tr} \boldsymbol{A}^2\right)^{1 / 2}$, with equality if and only if the eigenvalues of the $n \times n$ matrix $\boldsymbol{A}$ are all equal.
(c) If $\boldsymbol{A}$ is symmetric and $\boldsymbol{A} \neq \mathrm{O}$, show that
$$
\operatorname{rk}(\boldsymbol{A}) \geq \frac{(\operatorname{tr} \boldsymbol{A})^2}{\operatorname{tr} \boldsymbol{A}^2} .
$$

Check back soon!

Problem 5

(Cauchy-Schwarz, trace version) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be two matrices of the same order. Show that:
(a) $\left(\operatorname{tr} \boldsymbol{A}^{\prime} \boldsymbol{B}\right)^2 \leq\left(\operatorname{tr} \boldsymbol{A}^{\prime} \boldsymbol{A}\right)\left(\operatorname{tr} \boldsymbol{B}^{\prime} \boldsymbol{B}\right)$ with equality if and only if one of the matrices $\boldsymbol{A}$ and $B$ is a multiple of the other;
(b) $\operatorname{tr}\left(\boldsymbol{A}^{\prime} \boldsymbol{B}\right)^2 \leq \operatorname{tr}\left(\boldsymbol{A}^{\prime} \boldsymbol{A} \boldsymbol{B}^{\prime} \boldsymbol{B}\right)$ with equality if and only if $\boldsymbol{A} \boldsymbol{B}^{\prime}$ is symmetric;
(c) $\operatorname{tr}\left(\boldsymbol{A}^{\prime} \boldsymbol{B}\right)^2 \leq \operatorname{tr}\left(\boldsymbol{A} \boldsymbol{A}^{\prime} \boldsymbol{B} \boldsymbol{B}^{\prime}\right)$ with equality if and only if $\boldsymbol{A}^{\prime} \boldsymbol{B}$ is symmetric.

Check back soon!

Problem 6

(Schur's inequality) Let $A$ be a square real matrix. Prove Schur's inequality, $\operatorname{tr} \boldsymbol{A}^2 \leq \operatorname{tr} \boldsymbol{A}^{\prime} \boldsymbol{A}$ with equality if and only if $\boldsymbol{A}$ is symmetric.

Check back soon!

Problem 7

(The fundamental determinantal inequality)
(a) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be positive semidefinite matrices. Show that $|\boldsymbol{A}+\boldsymbol{B}| \geq|\boldsymbol{A}|+|\boldsymbol{B}|$.
(b) When does equality occur?

Check back soon!

Problem 8

(Determinantal inequality, special case)
(a) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be positive semidefinite matrices and let $\boldsymbol{A} \geq \boldsymbol{B}$. Show that $|\boldsymbol{A}| \geq|\boldsymbol{B}|$.
(b) Show that equality occurs if and only if $\boldsymbol{A}$ and $\boldsymbol{B}$ are nonsingular and $\boldsymbol{A}=\boldsymbol{B}$, or (trivially) if $\boldsymbol{A}$ and $\boldsymbol{B}$ are both singular.

Check back soon!

01:30

Problem 9

(Condition for $\boldsymbol{A}=\boldsymbol{I}$ ) Let $\mathrm{O} \leq \boldsymbol{A} \leq \boldsymbol{I}$. (This means that both $\boldsymbol{A}$ and $\boldsymbol{I}-\boldsymbol{A}$ are positive semidefinite.) Show that $\boldsymbol{A}=\boldsymbol{I}$ if and only if $|\boldsymbol{A}|=1$.

Vishnu P

Numerade Educator

Problem 10

(Lines in the plane) Let $A$ be a set of $n \geq 3$ elements, and let $A_1, \ldots, A_m$ be proper subsets of $A$, such that every pair of elements of $A$ is contained in precisely one set $A_j$.
(a) Give two examples for $n=4$, and represent these examples graphically.
(b) Show that $m \geq n$.
(c) Prove that it is not possible to arrange $n$ points in the plane in such a way that every line through two points also passes through a third, unless they all lie on the same line (Sylvester).

Check back soon!

Problem 11

(Arithmetic-geometric mean inequality)
(a) Show that $\left(\prod_i \lambda_i\right)^{1 / n} \leq(1 / n) \sum_i \lambda_i$ for any set of nonnegative numbers $\lambda_1, \ldots, \lambda_n$, with equality if and only if all $\lambda_i$ are the same (Euclid).
(b) Use (a) to show that $|\boldsymbol{A}|^{1 / n} \leq(1 / n) \operatorname{tr} \boldsymbol{A}$ for any positive semidefinite $n \times n$ matrix $\boldsymbol{A}$.
(c) Show that equality in (b) occurs if and only if $\boldsymbol{A}=\alpha \boldsymbol{I}_n$ for some $\alpha \geq 0$.

Check back soon!

Problem 12

(Quasilinear representation of $|\boldsymbol{A}|^{1 / n}$ ) Let $\boldsymbol{A}$ be a positive semidefinite $n \times n$ matrix.
(a) Show that (1/n) $\operatorname{tr} \boldsymbol{A} \boldsymbol{X} \geq|\boldsymbol{A}|^{1 / n}$ for every $n \times n$ matrix $\boldsymbol{X}>\mathbf{O}$ satisfying $|\boldsymbol{X}|=1$.
(b) Show that equality occurs if and only if $\boldsymbol{X}=|\boldsymbol{A}|^{1 / n} \boldsymbol{A}^{-1}$ or $\boldsymbol{A}=\mathbf{O}$.

Check back soon!

Problem 13

Minkowski's inequality)
(a) Use the quasilinear representation of Exercise 12.12 to show that
$$
|\boldsymbol{A}+\boldsymbol{B}|^{1 / n} \geq|\boldsymbol{A}|^{1 / n}+|\boldsymbol{B}|^{1 / n}
$$
for every two positive semidefinite $n \times n$ matrices $\boldsymbol{A} \neq \mathbf{O}$ and $\boldsymbol{B} \neq \mathbf{O}$.
(b) When does equality occur?
*

Check back soon!

Problem 14

(Trace inequality, 1) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be positive semidefinite $n \times n$ matrices.
(a) Show that $0 \leq \operatorname{tr} \boldsymbol{A B} \leq(\operatorname{tr} \boldsymbol{A})(\operatorname{tr} \boldsymbol{B})$.
(b) Show that $\sqrt{\operatorname{tr} \boldsymbol{A B}} \leq(\operatorname{tr} \boldsymbol{A}+\operatorname{tr} \boldsymbol{B}) / 2$.
(c) When does equality in (b) occur?

Check back soon!

Problem 15

(Cauchy-Schwarz, determinantal version) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be two matrices of the same order. Show that $\left|\boldsymbol{A}^{\prime} \boldsymbol{B}\right|^2 \leq\left|\boldsymbol{A}^{\prime} \boldsymbol{A} \| \boldsymbol{B}^{\prime} \boldsymbol{B}\right|$ with equality if and only if $\boldsymbol{A}^{\prime} \boldsymbol{A}$ or $\boldsymbol{B}^{\prime} \boldsymbol{B}$ is singular, or if $\boldsymbol{B}=\boldsymbol{A} \boldsymbol{Q}$ for some nonsingular matrix $\boldsymbol{Q}$.

Check back soon!

01:06

Problem 16

(Inequality for the inverse) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be positive definite. Show that $\boldsymbol{A}>\boldsymbol{B}$ if and only if $\boldsymbol{B}^{-1}>\boldsymbol{A}^{-1}$.

Linh Vu

Numerade Educator

Problem 17

(Kantorovich's inequality)
(a) Show that $\lambda^2-(a+b) \lambda+a b \leq 0$ for all $\lambda \in[a, b]$.
(b) Let $\boldsymbol{A}$ be a positive definite $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n>0$.

Use (a) to show that the matrix
$$
\left(\lambda_1+\lambda_n\right) \boldsymbol{I}_n-\boldsymbol{A}-\left(\lambda_1 \lambda_n\right) \boldsymbol{A}^{-1}
$$
is positive semidefinite of rank $\leq n-2$.
(c) Use Schur's inequality (Exercise 12.6) and (b) to show that
$$
1 \leq\left(\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}\right)\left(\boldsymbol{x}^{\prime} \boldsymbol{A}^{-1} \boldsymbol{x}\right) \leq \frac{\left(\lambda_1+\lambda_n\right)^2}{4 \lambda_1 \lambda_n}
$$
for every positive definite $n \times n$ matrix $\boldsymbol{A}$ and every $n \times 1$ vector $\boldsymbol{x}$ satisfying $\boldsymbol{x}^{\prime} \boldsymbol{x}=1$ (Kantorovich).

Check back soon!

Problem 18

(Inequality when $\boldsymbol{A}^{\prime} \boldsymbol{B}=\boldsymbol{I}$ ) For any two matrices $\boldsymbol{A}$ and $\boldsymbol{B}$ satisfying $\boldsymbol{A}^{\prime} \boldsymbol{B}=\boldsymbol{I}$, show that
$$
\boldsymbol{A}^{\prime} \boldsymbol{A} \geq\left(\boldsymbol{B}^{\prime} \boldsymbol{B}\right)^{-1} \quad \text { and } \quad \boldsymbol{B}^{\prime} \boldsymbol{B} \geq\left(\boldsymbol{A}^{\prime} \boldsymbol{A}\right)^{-1} .
$$

Check back soon!

Problem 19

(Unequal powers) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be positive semidefinite matrices and let $\boldsymbol{A} \geq \boldsymbol{B}$.
(a) Show that $\boldsymbol{A}^{1 / 2} \geq \boldsymbol{B}^{1 / 2}$.
(b) Show that it is not true, in general, that $\boldsymbol{A}^2 \geq \boldsymbol{B}^2$.
(c) However, if $\boldsymbol{A}$ and $\boldsymbol{B}$ commute (that is, $\boldsymbol{A} \boldsymbol{B}=\boldsymbol{B} \boldsymbol{A}$ ), then show that $\boldsymbol{A}^k \geq \boldsymbol{B}^k$ for $k=2,3, \ldots$

Check back soon!

Problem 20

(Bound for $\log |\boldsymbol{A}|$ ) Let $\boldsymbol{A}$ be a positive definite $n \times n$ matrix. Show that
$$ \log |\boldsymbol{A}| \leq \operatorname{tr} \boldsymbol{A}-n, $$
with equality if and only if $\boldsymbol{A}=\boldsymbol{I}_n$.

Check back soon!

Problem 21

(Concavity of $\log |\boldsymbol{A}|$ )
(a) For every $\lambda>0$ and $0<\alpha<1$, show that $\lambda^\alpha \leq \alpha \lambda+1-\alpha$, with equality if and only if $\lambda=1$.
(b) For any $\boldsymbol{A} \geq \mathbf{O}$ and $B \geq \mathbf{O}$ of the same order, show that
$$
|\boldsymbol{A}|^\alpha|\boldsymbol{B}|^{1-\alpha} \leq|\alpha \boldsymbol{A}+(1-\alpha) \boldsymbol{B}|
$$
for every $0<\alpha<1$.
(c) When does equality occur?

Check back soon!

Problem 22

(Implication of concavity)
(a) Let $\alpha_i>0, \sum_i \alpha_i=1$ and $\boldsymbol{A}_i>\mathbf{O}$ for $i=1, \ldots, k$. Show that
$$
\left|\boldsymbol{A}_1\right|^{\alpha_1}\left|\boldsymbol{A}_2\right|^{\alpha_2} \ldots\left|\boldsymbol{A}_k\right|^{\alpha_k} \leq\left|\alpha_1 \boldsymbol{A}_1+\alpha_2 \boldsymbol{A}_2+\cdots+\alpha_k \boldsymbol{A}_k\right| .
$$
(b) When does equality occur?

Check back soon!

Problem 23

(Positive definiteness of bordered matrix) Let $\boldsymbol{A}$ be a positive definite $n \times n$ matrix, and let $\boldsymbol{B}$ be the $(n+1) \times(n+1)$ bordered matrix
$$
\boldsymbol{B}:=\left(\begin{array}{ll}
\boldsymbol{A} & \boldsymbol{b} \\
\boldsymbol{b}^{\prime} & \alpha
\end{array}\right) .
$$

Show that:
(a) $|\boldsymbol{B}| \leq \alpha|\boldsymbol{A}|$ with equality if and only if $\boldsymbol{b}=\mathbf{0}$;
(b) $\boldsymbol{B}$ is positive definite if and only if $|\boldsymbol{B}|>0$.

Check back soon!

Problem 24

(Positive semidefiniteness of bordered matrix) Let $\boldsymbol{A}$ be a positive semidefinite $n \times n$ matrix, and let $\boldsymbol{B}$ be the $(n+1) \times(n+1)$ bordered matrix
$$
\boldsymbol{B}:=\left(\begin{array}{ll}
\boldsymbol{A} & \boldsymbol{b} \\
\boldsymbol{b}^{\prime} & \alpha
\end{array}\right)
$$

Check back soon!

Problem 25

(Bordered matrix, special case) Consider the matrix
$$
\boldsymbol{B}:=\left(\begin{array}{cc}
\boldsymbol{A} & \boldsymbol{b} \\
\boldsymbol{b}^{\prime} & \boldsymbol{b}^{\prime} \boldsymbol{A}^{-1} \boldsymbol{b}
\end{array}\right),
$$
where $\boldsymbol{A}$ is positive definite.
(a) Show that $\boldsymbol{B}$ is positive semidefinite and singular.
(b) Find the eigenvector associated with the zero eigenvalue.

Check back soon!

Problem 26

(Hadamard's inequality)
(a) Let $\boldsymbol{A}:=\left(a_{i j}\right)$ be a positive definite $n \times n$ matrix. Show that $|\boldsymbol{A}| \leq \prod_{i=1}^n a_{i i}$ with equality if and only if $\boldsymbol{A}$ is diagonal.
(b) Use (a) to show that
$$ |\boldsymbol{A}|^2 \leq \prod_{i=1}^n\left(\sum_{j=1}^n a_{i j}^2\right) . $$
for any $n \times n$ matrix $\boldsymbol{A}$ (Hadamard).
(c) When does Hadamard's inequality become an equality?

Check back soon!

Problem 27

(When is a symmetric matrix diagonal?)
(a) We know from Exercise 12.26(a) that, if $\boldsymbol{A}$ is a positive definite $n \times n$ matrix, then, $|\boldsymbol{A}|=\prod_{i=1}^n a_{i i}$ if and only if $\boldsymbol{A}$ is diagonal. If $\boldsymbol{A}$ is merely symmetric, show that this result is still true for $n=2$, but not in general for $n \geq 3$.
(b) Now show that a symmetric matrix is diagonal if and only if its eigenvalues and its diagonal elements coincide.

Check back soon!

Problem 28

Trace inequality, 2)
(a) If $\boldsymbol{A}$ is positive definite, show that $\boldsymbol{A}+\boldsymbol{A}^{-1}-2 \boldsymbol{I}$ is positive semidefinite.
(b) When is the matrix positive definite?
(c) Let $\boldsymbol{A}>\mathbf{O}$ and $\boldsymbol{B}>\mathbf{O}$, both of order $n$. Use (a) to show that
$$ \operatorname{tr}\left(\left(\boldsymbol{A}^{-1}-\boldsymbol{B}^{-1}\right)(\boldsymbol{A}-\boldsymbol{B})\right) \leq 0 . $$

Check back soon!

Problem 29

(OLS and GLS) Let $\boldsymbol{V}$ be a positive definite $n \times n$ matrix, and let $\boldsymbol{X}$ be an $n \times k$ matrix with $\operatorname{rk}(\boldsymbol{X})=k$. Show that
$$
\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime} \boldsymbol{V} \boldsymbol{X}\left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1} \geq\left(\boldsymbol{X}^{\prime} \boldsymbol{V}^{-1} \boldsymbol{X}\right)^{-1},
$$
(a) using Exercise 12.18 ;
(b) by considering the matrix $\boldsymbol{M}:=\boldsymbol{V}^{-1 / 2} \boldsymbol{X}\left(\boldsymbol{X}^{\prime} \boldsymbol{V}^{-1} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime} \boldsymbol{V}^{-1 / 2}$.
(c) Can you explain the title "OLS and GLS" of this exercise? (Conditions for equality are provided in Exercise 8.69.)

Check back soon!

Problem 30

(Bound for $\log |\boldsymbol{A}|$, revisited) Show that
$$
\frac{|\boldsymbol{A}+\boldsymbol{B}|}{|\boldsymbol{A}|} \leq \exp \left(\operatorname{tr}\left(\boldsymbol{A}^{-1} \boldsymbol{B}\right)\right),
$$
where $\boldsymbol{A}$ and $\boldsymbol{A}+\boldsymbol{B}$ are positive definite, with equality if and only if $\boldsymbol{B}=\mathbf{O}$.

Check back soon!

Problem 31

(Olkin's inequality)
(a) Let $\boldsymbol{A}$ be positive definite and $\boldsymbol{B}$ symmetric such that $|\boldsymbol{A}+\boldsymbol{B}| \neq 0$. Show that
$$
(\boldsymbol{A}+\boldsymbol{B})^{-1} \boldsymbol{B}(\boldsymbol{A}+\boldsymbol{B})^{-1} \leq \boldsymbol{A}^{-1}-(\boldsymbol{A}+\boldsymbol{B})^{-1} .
$$
(b) Show that the inequality is strict if and only if $\boldsymbol{B}$ is nonsingular.

Check back soon!

Problem 32

(Positive definiteness of Hadamard product) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be square $n \times n$ matrices. The Hadamard product $\boldsymbol{A} \odot \boldsymbol{B}$ is defined as the $n \times n$ matrix whose $i j$-th element is $a_{i j} b_{i j}$. This is the element-by-element matrix product.
(a) Let $\boldsymbol{\lambda}:=\left(\lambda_1, \ldots, \lambda_n\right)^{\prime}$ and $\boldsymbol{\Lambda}:=\operatorname{diag}\left(\lambda_1, \ldots, \lambda_n\right)$. Show that
$$
\boldsymbol{\lambda}^{\prime}(\boldsymbol{A} \odot \boldsymbol{B}) \boldsymbol{\lambda}=\operatorname{tr} \boldsymbol{A} \boldsymbol{\Lambda} \boldsymbol{B}^{\prime} \boldsymbol{\Lambda} .
$$
(b) If $\boldsymbol{A}>\mathbf{O}$ and $\boldsymbol{B}>\mathbf{O}$, show that $\boldsymbol{A} \odot \boldsymbol{B}>\mathbf{O}$.

Check back soon!

Problem 33

(Schur complement: basic inequality) Let $\boldsymbol{A}$ be a positive definite $n \times n$ matrix and let $\boldsymbol{B}$ be an $n \times m$ matrix. Show that, for any symmetric $m \times m$ matrix $\boldsymbol{X}$,
$$
\boldsymbol{Z}:=\left(\begin{array}{ll}
\boldsymbol{A} & \boldsymbol{B} \\
\boldsymbol{B}^{\prime} & \boldsymbol{X}
\end{array}\right) \geq \mathbf{O} \Longleftrightarrow \boldsymbol{X} \geq \boldsymbol{B}^{\prime} \boldsymbol{A}^{-1} \boldsymbol{B},
$$
and that, if one inequality is strict, the other is strict as well.

Check back soon!

Problem 34

(Fischer's inequality, again) As in Exercise 8.46, consider
$$
\boldsymbol{Z}:=\left(\begin{array}{ll}
\boldsymbol{A} & \boldsymbol{B} \\
\boldsymbol{B}^{\prime} & \boldsymbol{D}
\end{array}\right)>\mathbf{O} .
$$
(a) Show that $|\boldsymbol{Z}| \leq|\boldsymbol{A}||\boldsymbol{D}|$ with equality if and only if $\boldsymbol{B}=\mathbf{O}$.
(b) If $\boldsymbol{B}$ is a square matrix, show that $|\boldsymbol{B}|^2<|\boldsymbol{A}||\boldsymbol{D}|$.

Check back soon!

Problem 35

(A positive semidefinite matrix) Show that
$$
\left(\begin{array}{cc}
I & A \\
A^{\prime} & A^{\prime} A
\end{array}\right) \geq \mathbf{O}
$$
for any matrix $\boldsymbol{A}$.

Check back soon!

Problem 36

(OLS and GLS, continued) Let $\boldsymbol{V}$ be a positive definite $n \times n$ matrix, and let $\boldsymbol{X}$ be an $n \times k$ matrix with $\operatorname{rk}(\boldsymbol{X})=k$.
(a) Show that $\boldsymbol{X}\left(\boldsymbol{X}^{\prime} \boldsymbol{V}^{-1} \boldsymbol{X}\right)^{-1} \boldsymbol{X}^{\prime} \leq \boldsymbol{V}$.
(b) Conclude that $\left(\boldsymbol{X}^{\prime} \boldsymbol{V}^{-1} \boldsymbol{X}\right)^{-1} \leq \boldsymbol{X}^{\prime} \boldsymbol{V} \boldsymbol{X}$ for any $\boldsymbol{X}$ satisfying $\boldsymbol{X}^{\prime} \boldsymbol{X}=\boldsymbol{I}_k$.

Check back soon!

Problem 37

(Another positive semidefinite matrix) Let $\boldsymbol{A}$ and $B$ be matrices of order $m \times n$.
(a) Show that
$$
\left(\begin{array}{cc}
\boldsymbol{I}_m+\boldsymbol{A} \boldsymbol{A}^{\prime} & \boldsymbol{A}+\boldsymbol{B} \\
(\boldsymbol{A}+\boldsymbol{B})^{\prime} & \boldsymbol{I}_n+\boldsymbol{B}^{\prime} \boldsymbol{B}
\end{array}\right) \geq \mathbf{O} .
$$
(b) Use the Schur complement to show that
$$
\boldsymbol{I}_n+\boldsymbol{B}^{\prime} \boldsymbol{B} \geq(\boldsymbol{A}+\boldsymbol{B})^{\prime}\left(\boldsymbol{I}_m+\boldsymbol{A} \boldsymbol{A}^{\prime}\right)^{-1}(\boldsymbol{A}+\boldsymbol{B}) .
$$

Check back soon!

Problem 38

(An inequality equivalence) Let $\boldsymbol{A}>\mathrm{O}$ and $B>0$. Show that
$$
\left(\begin{array}{cc}
\boldsymbol{A}+\boldsymbol{B} & \boldsymbol{A} \\
\boldsymbol{A} & \boldsymbol{A}+\boldsymbol{X}
\end{array}\right) \geq \mathbf{O} \Longleftrightarrow \boldsymbol{X} \geq-\left(\boldsymbol{A}^{-1}+\boldsymbol{B}^{-1}\right)^{-1} .
$$

Check back soon!

Problem 39

(Bounds of Rayleigh quotient, continued) Let $\boldsymbol{A}$ be a symmetric $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$.
(a) Show that the Rayleigh quotient $\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x} / \boldsymbol{x}^{\prime} \boldsymbol{x}$ is bounded by
$$
\lambda_n \leq \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}} \leq \lambda_1
$$
(b) Can the bounds of $\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x} / \boldsymbol{x}^{\prime} \boldsymbol{x}$ be achieved?
(c) (Quasilinear representation) Show that we may express $\lambda_1$ and $\lambda_n$ as
$$
\lambda_1=\max _{\boldsymbol{x}} \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}} \quad \text { and } \quad \lambda_n=\min _{\boldsymbol{x}} \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}} .
$$
What potential use is this quasilinear representation?

Check back soon!

Problem 40

(Applications of the quasilinear representation) Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be symmetric $n \times n$ matrices.
(a) Use the quasilinear representations in Exercise 12.39 to show that
$$
\lambda_1(\boldsymbol{A}+\boldsymbol{B}) \leq \lambda_1(\boldsymbol{A})+\lambda_1(\boldsymbol{B}) \text { and } \quad \lambda_n(\boldsymbol{A}+\boldsymbol{B}) \geq \lambda_n(\boldsymbol{A})+\lambda_n(\boldsymbol{B}) .
$$
(b) If, in addition, $\boldsymbol{B}$ is positive semidefinite, show that
$$
\lambda_1(\boldsymbol{A}+\boldsymbol{B}) \geq \lambda_1(\boldsymbol{A}) \quad \text { and } \quad \lambda_n(\boldsymbol{A}+\boldsymbol{B}) \geq \lambda_n(\boldsymbol{A}) .
$$

Check back soon!

Problem 41

Convexity of $\lambda_1$, concavity of $\lambda_n$ ) For any two symmetric matrices $\boldsymbol{A}$ and $\boldsymbol{B}$ of order $n \times n$ and $0 \leq \alpha \leq 1$, show that
$$
\begin{aligned}
& \lambda_1(\alpha \boldsymbol{A}+(1-\alpha) \boldsymbol{B}) \leq \alpha \lambda_1(\boldsymbol{A})+(1-\alpha) \lambda_1(\boldsymbol{B}), \\
& \lambda_n(\alpha \boldsymbol{A}+(1-\alpha) \boldsymbol{B}) \geq \alpha \lambda_n(\boldsymbol{A})+(1-\alpha) \lambda_n(\boldsymbol{B}) .
\end{aligned}
$$
Hence, $\lambda_1$ is convex and $\lambda_n$ is concave on the space of symmetric matrices.

Check back soon!

Problem 42

(Variational description of eigenvalues) Let $\boldsymbol{A}$ be a symmetric $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$. Let $\boldsymbol{S}:=\left(s_1, s_2, \ldots, s_n\right)$ be an orthogonal $n \times n$ matrix that diagonalizes $\boldsymbol{A}$, so that $\boldsymbol{S}^{\prime} \boldsymbol{A} \boldsymbol{S}=\operatorname{diag}\left(\lambda_1, \lambda_2, \ldots, \lambda_n\right)$. Show that
$$
\lambda_k=\max _{\boldsymbol{R}_{k-1}^{\prime} \boldsymbol{x}=\mathbf{0}} \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}}=\min _{\boldsymbol{T}_{k+1}^{\prime} \boldsymbol{x}=0} \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}} \quad(k=1, \ldots, n),
$$
where
$$
\boldsymbol{R}_k:=\left(\boldsymbol{s}_1, \boldsymbol{s}_2, \ldots, \boldsymbol{s}_k\right) \text { and } \boldsymbol{T}_k:=\left(\boldsymbol{s}_k, \boldsymbol{s}_{k+1}, \ldots, \boldsymbol{s}_n\right),
$$
and we agree to interpret $\boldsymbol{R}_0$ and $\boldsymbol{T}_{n+1}$ as "empty" in the sense that the restrictions $\boldsymbol{R}_0^{\prime} \boldsymbol{x}=$ $\mathbf{0}$ and $\boldsymbol{T}_{n+1}^{\prime} \boldsymbol{x}=\mathbf{0}$ do not impose a restriction on $\boldsymbol{x}$.

Check back soon!

Problem 43

(Variational description, generalized) Let $\boldsymbol{A}$ be a symmetric $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$. Show that, for every $n \times(k-1)$ matrix $\boldsymbol{B}$ and $n \times(n-k)$ matrix $C$,
$$
\min _{\boldsymbol{C}^{\prime} \boldsymbol{x}=0} \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}} \leq \lambda_k \leq \max _{\boldsymbol{B}^{\prime} \boldsymbol{x}=0} \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}} .
$$

Check back soon!

Problem 44

(Fischer's min-max theorem) Let $\boldsymbol{A}$ be a symmetric $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$. Obtain a quasilinear representation of the eigenvalues, that is, show that $\lambda_k$ can be expressed as
$$
\lambda_k=\min _{B^{\prime} \boldsymbol{B}=I_{k-1}} \max _{B^{\prime} \boldsymbol{x}=\mathbf{0}} \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}},
$$
and equivalently as
$$
\lambda_k=\max _{C^{\prime} C=I_{n-k}} \min _{C^{\prime} \boldsymbol{x}=\mathbf{0}} \frac{\boldsymbol{x}^{\prime} \boldsymbol{A} \boldsymbol{x}}{\boldsymbol{x}^{\prime} \boldsymbol{x}},
$$
where, as the notation indicates, $\boldsymbol{B}$ is an $n \times(k-1)$ matrix and $\boldsymbol{C}$ is an $n \times(n-k)$ matrix.

Check back soon!

01:25

Problem 45

(Monotonicity of eigenvalue function) Generalize Exercise 12.40(b) by showing that, for any symmetric matrix $\boldsymbol{A}$ and positive semidefinite matrix $\boldsymbol{B}$,
$$
\lambda_k(\boldsymbol{A}+\boldsymbol{B}) \geq \lambda_k(\boldsymbol{A}) \quad(k=1,2, \ldots, n) .
$$
If $\boldsymbol{B}$ is positive definite, show that the inequality is strict.

Jack Chen

Numerade Educator

Problem 46

Poincaré's separation theorem) Let $\boldsymbol{A}$ be a symmetric $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$, and let $G$ be a semi-orthogonal $n \times r$ matrix $(1 \leq r \leq n)$, so that $\boldsymbol{G}^{\prime} \boldsymbol{G}=\boldsymbol{I}_r$. Show that the eigenvalues $\mu_1 \geq \mu_2 \geq \cdots \geq \mu_r$ of $\boldsymbol{G}^{\prime} \boldsymbol{A} \boldsymbol{G}$ satisfy
$$
\lambda_{n-r+k} \leq \mu_k \leq \lambda_k \quad(k=1,2, \ldots, r) .
$$

Check back soon!

Problem 47

(Poincaré applied, 1) Let $\boldsymbol{A}$ be a symmetric $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$, and let $\boldsymbol{M}$ be an idempotent symmetric $n \times n$ matrix of rank $r$ $(1 \leq r \leq n$ ). Denote the eigenvalues of the $n \times n$ matrix $\boldsymbol{M} \boldsymbol{A M}$, apart from $n-r$ zeros, by $\mu_1 \geq \mu_2 \geq \cdots \geq \mu_r$. Use Poincaré's separation theorem to show that
$$
\lambda_{n-r+k} \leq \mu_k \leq \lambda_k \quad(k=1,2, \ldots, r) .
$$

Check back soon!

Problem 48

(Poincaré applied, 2) Let $\boldsymbol{A}$ be a symmetric $n \times n$ matrix with eigenvalues $\lambda_1(\boldsymbol{A}) \geq \lambda_2(\boldsymbol{A}) \geq \cdots \geq \lambda_n(\boldsymbol{A})$, and let $\boldsymbol{A}_{(r)}$ be an $r \times r$ principal submatrix of $\boldsymbol{A}$ (not necessarily a leading principal submatrix) with eigenvalues $\lambda_1\left(\boldsymbol{A}_{(r)}\right) \geq \lambda_2\left(\boldsymbol{A}_{(r)}\right) \geq$ $\cdots \geq \lambda_r\left(\boldsymbol{A}_{(r)}\right)$
(a) Use Poincare's separation theorem to show that
$$
\lambda_{n-r+k}(\boldsymbol{A}) \leq \lambda_k\left(\boldsymbol{A}_{(r)}\right) \leq \lambda_k(\boldsymbol{A}) \quad(k=1,2, \ldots, r) .
$$
(b) In particular, show that
$$
\begin{aligned}
\lambda_n\left(\boldsymbol{A}_{(n)}\right) & \leq \lambda_{n-1}\left(\boldsymbol{A}_{(n-1)}\right) \leq \lambda_{n-1}\left(\boldsymbol{A}_{(n)}\right) \leq \lambda_{n-2}\left(\boldsymbol{A}_{(n-1)}\right) \\
& \leq \cdots \leq \lambda_1\left(\boldsymbol{A}_{(n-1)}\right) \leq \lambda_1\left(\boldsymbol{A}_{(n)}\right) .
\end{aligned}
$$

Check back soon!

Problem 49

(Bounds for $\operatorname{tr} \boldsymbol{A}_{(r)}$ ) Let $\boldsymbol{A}$ be a symmetric $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$
(a) Show that
$$
\max _{\boldsymbol{X}^{\prime} \boldsymbol{X}=\boldsymbol{I}_r} \operatorname{tr} \boldsymbol{X}^{\prime} \boldsymbol{A} \boldsymbol{X}=\sum_{k=1}^r \lambda_k \quad \text { and } \min _{\boldsymbol{X}^{\prime} \boldsymbol{X}=\boldsymbol{I}_r} \operatorname{tr} \boldsymbol{X}^{\prime} \boldsymbol{A} \boldsymbol{X}=\sum_{k=1}^r \lambda_{n-r+k} .
$$

Check back soon!

Problem 50

(Bounds for $\left|\boldsymbol{A}_{(r)}\right|$ ) Let $\boldsymbol{A}$ be a positive definite $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$.
(a) Show that
$$
\max _{\boldsymbol{X}^{\prime} \boldsymbol{X}=\boldsymbol{I}_r}\left|\boldsymbol{X}^{\prime} \boldsymbol{A} \boldsymbol{X}\right|=\prod_{k=1}^r \lambda_k \quad \text { and } \quad \min _{\boldsymbol{X}^{\prime} \boldsymbol{X}=\boldsymbol{I}_r}\left|\boldsymbol{X}^{\prime} \boldsymbol{A} \boldsymbol{X}\right|=\prod_{k=1}^r \lambda_{n-r+k} .
$$
(b) In particular, letting $\boldsymbol{A}_{(r)}$ be as defined in Exercise 12.49, show that
$$
\prod_{k=1}^r \lambda_{n-r+k} \leq\left|\boldsymbol{A}_{(r)}\right| \leq \prod_{k=1}^r \lambda_k .
$$

Check back soon!

Problem 51

(A consequence of Hadamard's inequality) Let $\boldsymbol{A}=\left(a_{i j}\right)$ be a positive definite $n \times n$ matrix with eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$. Show that
$$
\prod_{k=1}^r \lambda_{n-r+k} \leq \prod_{k=1}^r a_{k k} \quad(r=1, \ldots, n) .
$$

Check back soon!