Singular value decomposition
Undergraduate
Consider an $m \times n$ real matrix $A$. A singular value decomposition of $A$ is a factorization
\[A = U \Sigma V^{\top},\]where $U$ and $V$ are orthogonal matrices and $\Sigma$ is an $m \times n$ diagonal matrix whose diagonal entries $\sigma_1 \geq \sigma_2 \geq \dots \geq 0$ are the singular values of $A$.
Examples
Worked examples
-
The matrix \(\begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix}\)
: $\Sigma = \operatorname{diag}\bigl(\sqrt{15 + \sqrt{221}}, \sqrt{15 - \sqrt{221}}\bigr) \approx \operatorname{diag}(5.46, 0.37)$
Undergraduate
The singular values of \(A = \begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix}\) are the square roots of the eigenvalues $15 \pm \sqrt{221}$ of \(A^{\top} A = \begin{bmatrix} 20 & 14 \\ 14 & 10 \end{bmatrix}\). Their product is $\abs{\det A} = 2$.
-
The matrix \(\begin{bmatrix} 4 & 2 \\ 2 & 3 \end{bmatrix}\)
: $\Sigma = \Lambda$, $U = V = Q$, as in the eigendecomposition
Undergraduate
The matrix \(A = \begin{bmatrix} 4 & 2 \\ 2 & 3 \end{bmatrix}\) is symmetric positive definite, so its eigendecomposition $A = Q \Lambda Q^{\top}$ is also a singular value decomposition.
-
The matrix \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\)
: $\Sigma = \operatorname{diag}(\varphi, 1 / \varphi)$
Undergraduate
Here $\varphi = \frac{1 + \sqrt{5}}{2}$ is the golden ratio. The singular values of \(A = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) are the golden ratio $\varphi$ and $1 / \varphi$, the square roots of the eigenvalues $\frac{3 \pm \sqrt{5}}{2}$ of \(A^{\top} A = \begin{bmatrix} 1 & 1 \\ 1 & 2 \end{bmatrix}\). Both eigenvalues of $A$ are $1$, so the singular values differ from the eigenvalues. # TODO: Check the singular values.
- The matrix \(A = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\) : $U = A$, $\Sigma = I$, $V = I$ Undergraduate
Non-examples
Things that may be mistaken for examples.
-
An eigendecomposition $A = P D P^{-1}$, with $D$ diagonal
(definition)
Undergraduate
An eigendecomposition $A = P D P^{-1}$ is not a singular value decomposition, because the columns of $P$ need not be orthonormal, and the eigenvalues in $D$ can be negative or complex. For a symmetric positive semidefinite matrix, the two decompositions coincide.
Properties
- Every matrix has a singular value decomposition. The singular values are the square roots of the eigenvalues of $A^{\top} A$.
- The rank of $A$ is the number of nonzero singular values.
- Geometrically, every linear map is a rotation (or reflection), followed by stretching along perpendicular axes, followed by another rotation (or reflection).
Notes
For complex matrices, $V^{\top}$ is replaced by the conjugate transpose $V^{\ast}$, and $U$ and $V$ are unitary.
Distinctions involving this term
- Types of matrix decompositions: A matrix decomposition writes a matrix in terms of simpler matrices. Most are products: LU, PLU, and Cholesky decompositions use triangular factors, QR and singular value decompositions use orthogonal factors, and eigendecompositions, Schur decompositions, and Jordan normal forms are similarities $A = P M P^{-1}$ to a simpler matrix $M$ (diagonal, triangular, or made of Jordan blocks). In contrast, the Jordan–Chevalley decomposition is a sum of commuting diagonalizable and nilpotent parts, and the Toeplitz decomposition is a sum $H + K$ of a symmetric matrix $H$ and a skew-symmetric matrix $K$.