Suppose K1 and K2 are positive definite n ×n matrices. Suppose that, for i=1,2, the minimizer of

Suppose K1 and K2 are positive definite n ×n matrices....

Suppose $K_1$ and $K_2$ are positive definite $n \times n$ matrices. Suppose that, for $i=1,2$, the minimizer of $p_i(\mathbf{x})=\mathbf{x}^T K_i \mathbf{x}-2 \mathbf{x}^T \mathbf{f}_i+c_i$, is $\mathbf{x}_i^{\star}$. Is the minimizer of $p(\mathbf{x})=p_1(\mathbf{x})+p_2(\mathbf{x})$ given by $\mathbf{x}^{\star}=\mathbf{x}_1^{\star}+\mathbf{x}_2^{\star}$ ? Prove or give a counterexample.

Key Concepts

Positive Definiteness

A positive definite matrix in quadratic forms ensures that the quadratic function is strictly convex. This strict convexity guarantees that there is a unique global minimum for the optimization problem, which is crucial when determining and analyzing the minimizers of quadratic functions.

Quadratic Optimization

Quadratic optimization involves minimizing or maximizing a quadratic function, typically expressed in the form x^T K x - 2 x^T f + c. The problem is simplified by the structure of the function, and when K is positive definite, the problem is convex, allowing the use of analytical methods to find the unique minimizer.

Gradient and First-Order Necessary Condition

The minimizer of a differentiable quadratic function is found by setting its gradient to zero. This process leads to a linear system whose solution gives the optimal point. In the context of quadratic functions, this condition directly leads to the equation Kx = f, relating the matrix and the linear term of the function.

Combination of Quadratic Functions

When combining two quadratic functions by addition, the resulting function remains quadratic but its minimizer is generally not the simple sum of the individual minimizers. Instead, the minimizer is determined by the sum of the respective Hessians (or the underlying matrices) and the sum of the corresponding linear terms.

Counterexample in Minimizer Addition

A counterexample demonstrates that the minimizer of the sum of two quadratic functions is not necessarily the sum of their individual minimizers. This concept is pivotal in understanding that the superposition principle does not apply directly to optimization of quadratic forms unless specific conditions are met, highlighting the need for a careful derivation of the combined minimizer by considering weighted contributions of the individual functions.

Transcript

00:02 We first want to verify that the inverse of a inverse is a for 2 by 2 matrices.

00:17 And for 2 by 2 matrices, we have a formula for the inverse.

00:22 So let's say we have a matrix a, which equals a, b, c, d, and a, capital a is invertible.

00:30 Well, we have that the, we have a formula that says that the inverse is 1 over the determinant, of a multiplied by d minus b minus c a so the main diagonal entries are swapped and the other entries are negated so in order to take the inverse of a inverse let's first figure out its determinant because we're going to be using its determinant so to do that let's first write this in another way we have d over determinant a minus b over determinant a minus c over determinant a and a over determinant a.

01:27 So now we can see that the determinant of a inverse is equal to ad over determinant a squared minus b c over determinant a squared equals now ad minus bc is itself the determinant of a so we get the determinant of a over the determinant of a squared equals 1 over the determinant of a which is actually nice that the determinant of a inverse is the inverse of the determinant of a so now all we have to do is apply our formula for the inverse this time of a inverse.

02:52 So looking at this new way that we wrote a inverse, where we put the determinant in everywhere, we get by the formula that we swap the main diagonal entries, so we get a over determinant of a, d over determinant of a, and then we negate the other entries.

03:18 B over determinant of a, c over determinant of a.

03:26 And let's not forget, we have to divide by the determinant of a inverse...

Please give Ace some feedback