Outer-product matrices have all nonnegative eigenvalues (a)...

Outer-product matrices have all nonnegative eigenvalues (a) Use the method described in Exercise 4.1 to verify that for any $N \times 1$ vector $\mathbf{x}$, the $N \times N$ outer-product matrix $\mathbf{x x}^T$ has all nonnegative eigenvalues. (b) Similarly show that for any set of $P$ vectors $\mathrm{x}_1, \mathrm{x}_2, \ldots, \mathrm{x}_P$ of length $N$ that the sum of outer-product matrices $\sum_{p=1}^p \delta_p x_p x_p^T$ has all nonnegative eigenvalues if each $\delta_p \geq 0$ (c) Show that the matrix $\sum_{p=1}^P \delta_p \mathbf{x}_p \mathbf{x}_p^T+\lambda \mathbf{I}_{N \times N}$ where each $\delta_p \geq 0$ and $\lambda>0$ has all positive eigenvalues.

Outer-product matrices have all nonnegative eigenvalues
(a) Use the method described in Exercise 4.1 to verify that for any $N \times 1$ vector $\mathbf{x}$, the $N \times N$ outer-product matrix $\mathbf{x x}^T$ has all nonnegative eigenvalues.
(b) Similarly show that for any set of $P$ vectors $\mathrm{x}_1, \mathrm{x}_2, \ldots, \mathrm{x}_P$ of length $N$ that the sum of outer-product matrices $\sum_{p=1}^p \delta_p x_p x_p^T$ has all nonnegative eigenvalues if each $\delta_p \geq 0$
(c) Show that the matrix $\sum_{p=1}^P \delta_p \mathbf{x}_p \mathbf{x}_p^T+\lambda \mathbf{I}_{N \times N}$ where each $\delta_p \geq 0$ and $\lambda>0$ has all positive eigenvalues.

Key Concepts

Outer-Product Matrix

An outer-product matrix is formed by multiplying a vector by its transpose. This construction produces a symmetric matrix with rank one (assuming the vector is non-zero) whose nonzero eigenvalue is equal to the squared norm of the vector. This characteristic makes outer-product matrices useful in understanding the spectral properties of more complex matrices constructed from rank-one components.

Eigenvalues and Positive Semidefiniteness

Eigenvalues of a symmetric matrix are real, and a matrix is considered positive semidefinite if all its eigenvalues are nonnegative. For an outer-product matrix, the fact that its nonzero eigenvalue corresponds to a squared norm (always nonnegative) ensures it meets the criteria for positive semidefiniteness.

Matrix Summation of Weighted Outer-Products

When summing multiple outer-product matrices, especially when each is multiplied by a nonnegative scalar, the resulting matrix is still symmetric. The sum inherits the property of positive semidefiniteness because it acts as a weighted sum of nonnegative eigenvalues, thereby ensuring that the overall matrix does not introduce any negative eigenvalues.

Diagonal Perturbation and Positive Definiteness

Adding a positive scalar multiple of the identity matrix to a positive semidefinite matrix shifts all its eigenvalues by a positive amount. This diagonal perturbation transforms the matrix into a positive definite one by guaranteeing that every eigenvalue becomes strictly positive, which is critical in many optimization and stability analyses.

Transcript

00:01 Alright, so we have a matrix a and we're going to find the eigenvalues and eigenvectors of this matrix.

00:08 So to find eigenvectors, what we do is we find the characteristic polynomial by subtracting lambda i, where lambda is what we're solving for.

00:18 So we have a minus lambda i, that is 1 minus lambda, negative 1, 2, and 4 minus lambda.

00:29 So we're just subtracting lambda on the diagonals.

00:32 Then we need the determinant of this to equal zero.

00:38 To get the determinant of a 2 by 2 matrix is simple.

00:42 First we multiply the diagonals going this way, so lambda, 1 minus lambda times 4 minus lambda, and then we subtract the multiplication of the diagonals going this way, so it's minus negative 2, which is just plus 2.

01:00 And we want this to equal zero because we want this to have some sort of null space.

01:10 We don't want it to be full rank, so we want the determinant of this to be zero.

01:14 To solve, we need to simplify this equation.

01:18 So i'm going to multiply out by foiling, and then i'm going to combine like terms and order from highest exponent to lowest exponent.

01:35 So minus 4 lambda minus 1 is negative 5 lambda, and then we get 4 and plus 2, which is 6, equals zero.

01:42 And if we can factor, you should factor.

01:45 If you have to use the quadratic equation, that's fine, but we can factor this.

01:54 We just need lambda times lambda to get that lambda squared.

01:57 And then it's positive 6, so we know the signs have to be the same, but in the middle it's going to be both negative.

02:06 And then 3 and 2 is going to give us a sum of 5 but a product of 6.

02:12 And then i can solve each one for lambda.

02:14 This one works for lambda equals 3, and this one is lambda equals 2.

02:19 So our eigenvalues, sorry i wrote vectors, are 2 and 3.

02:25 Now there are two different ways to get the eigenvectors.

02:30 I'm going to do the easier way that does not involve the null space.

02:35 So our eigenvalues were 2 and 3, and that is we take a times gamma to equal lambda gamma and solve for when the euclidean distance of gamma is 1, because there's infinite eigenvectors because it's in the null space.

02:55 You could take any multiple of it.

02:58 So we'll start with for lambda equals 2.

03:02 We have 1, negative 1, 2, and 4 times gamma 1, gamma 2, and we want that to equal 2 gamma 1 and 2 gamma 2.

03:18 These are just placeholders.

03:20 So i'm going to multiply these out.

03:22 We get gamma 1 plus, minus, sorry, gamma 2 equals 2 gamma 1, and we get 2 gamma 1 plus 4 gamma 2 equals 2 gamma 2.

03:39 And we can solve this for one of them.

03:41 I will solve this one for gamma 2...