SVD and Image Compression Overview
Consider the m X n matrix A with entries represented by "one-byte" integers. One byte equals 8 bits of information, so each value in the matrix is represented by an integer in the range from 0 to 255. Since each matrix entry requires one byte of storage, the total matrix requires mn bytes of storage space.
The singular value decomposition (SVD) can be used to compute an orthogonal decomposition of a matrix. Given the m X n matrix A, the SVD can be used to decompose the matrix as A = USV^('), where U is an orthogonal m X m matrix, V is an orthogonal n X n matrix, and S is a m X n rectangular diagonal matrix whose diagonal contains the singular values of A. This representation is very similar to the eigenvalue diagonalization (called eigendecomposition) already studied in the course. However, a primary difference between SVD and eigendecomposition is that SVD can be used for rectangular matrices, while eigendecomposition can be used only on square matrices.
SVD can be used to construct a low-rank approximation of the original matrix. If only the first k singular values are used, then A can be approximated by the matrix A defined as:
A_k = U_kS_kV_k^(')
where U_k is a m X k matrix containing the first k columns of U, S_k is a k X k matrix containing the first k rows and k columns of S, and V_k is a n X k matrix containing the first k columns of V. Observe that this approximation essentially throws away all of the information stored in the dimensions k+1 and higher.
After computing the image approximation, we must be careful to make sure that the approximation has the correct data format. Each value in the matrix must be an unsigned 8-bit integer data type. Since the approximation now likely has real-valued quantities in it, we will round these entries to the closest integer and then cast the data type to an integer. This allows the image to be properly displayed in software programs such as MATLAB. So, after computing A_k, we'll typically execute a command similar to the following:
A_k = uint8(round(A_k))
The m X k matrix U_k has a total of mk entries that must be stored. Since S_k is a diagonal matrix, only its k diagonal elements need to be stored. The n X k matrix V_k has nk entries that must be stored. Thus, the total number of values to store for the low-rank matrix A_k is mk + k + nk = k(m + n + 1). Compared to the original storage requirement of mn, using only k singular values to approximate A with A_k achieves a compression ratio of
CR = (mn) / (k(m + n + 1))
To provide a specific example, if m = 600, n = 480, and k = 50, this would result in a compression ratio of
