Definition · applies to matrices · also called SVD

Undergraduate

Consider an $m \times n$ real matrix $A$. A singular value decomposition of $A$ is a factorization $A = U \Sigma V^{\top}$, where $U$ and $V$ are orthogonal matrices and $\Sigma$ is an $m \times n$ diagonal matrix whose diagonal entries $\sigma_1 \geq \sigma_2 \geq \dots \geq 0$ are the singular values of $A$.

Examples

Worked examples

  • The matrix \(\begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix}\) : $\Sigma = \operatorname{diag}\bigl(\sqrt{15 + \sqrt{221}}, \sqrt{15 - \sqrt{221}}\bigr) \approx \operatorname{diag}(5.46, 0.37)$ Undergraduate
    The singular values of \(A = \begin{bmatrix} 2 & 1 \\ 4 & 3 \end{bmatrix}\) are the square roots of the eigenvalues $15 \pm \sqrt{221}$ of \(A^{\top} A = \begin{bmatrix} 20 & 14 \\ 14 & 10 \end{bmatrix}\). Their product is $\lvert \det A \rvert = 2$.
  • The matrix \(\begin{bmatrix} 4 & 2 \\ 2 & 3 \end{bmatrix}\) : $\Sigma = \Lambda$, $U = V = Q$, as in the eigendecomposition Undergraduate
    The matrix \(A = \begin{bmatrix} 4 & 2 \\ 2 & 3 \end{bmatrix}\) is symmetric positive definite, so its eigendecomposition $A = Q \Lambda Q^{\top}$ is also a singular value decomposition.
  • The matrix \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) : $\Sigma = \operatorname{diag}(\varphi, 1 / \varphi)$, where $\varphi = \frac{1 + \sqrt{5}}{2}$ Undergraduate
    The singular values of \(A = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) are the golden ratio $\varphi$ and $1 / \varphi$, the square roots of the eigenvalues $\frac{3 \pm \sqrt{5}}{2}$ of \(A^{\top} A = \begin{bmatrix} 1 & 1 \\ 1 & 2 \end{bmatrix}\). Both eigenvalues of $A$ are $1$, so the singular values differ from the eigenvalues. # TODO: Check the singular values.
  • The matrix \(A = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\) : $U = A$, $\Sigma = I$, $V = I$ Undergraduate
    $A$ has no LU decomposition without exchanging rows, because its top-left entry is $0$, even though $A$ is invertible, but it has the PLU decomposition $A = A \cdot I \cdot I$. $A$ is orthogonal, so its QR decomposition is $A \cdot I$, its singular value decomposition is $A \cdot I \cdot I$, and its Jordan–Chevalley decomposition has $N = 0$ because $A$ is diagonalizable.

Non-examples

Things that may be mistaken for examples.

  • An eigendecomposition $A = P D P^{-1}$, with $D$ diagonal (definition) Undergraduate
    An eigendecomposition $A = P D P^{-1}$ is not a singular value decomposition, because the columns of $P$ need not be orthonormal, and the eigenvalues in $D$ can be negative or complex. For a symmetric positive semidefinite matrix, the two decompositions coincide.

Properties

  • Every matrix has a singular value decomposition. The singular values are the square roots of the eigenvalues of $A^{\top} A$.
  • The rank of $A$ is the number of nonzero singular values.
  • Geometrically, every linear map is a rotation (or reflection), followed by stretching along perpendicular axes, followed by another rotation (or reflection).

Notes

For complex matrices, $V^{\top}$ is replaced by the conjugate transpose $V^{\ast}$, and $U$ and $V$ are unitary.

Distinctions involving this term

  • Types of matrix decompositions: A matrix decomposition writes a matrix in terms of simpler matrices. Most are products: LU, PLU, and Cholesky decompositions use triangular factors, QR and singular value decompositions use orthogonal factors, and eigendecompositions, Schur decompositions, and Jordan normal forms are similarities $A = P M P^{-1}$ to a simpler matrix $M$ (diagonal, triangular, or made of Jordan blocks). In contrast, the Jordan–Chevalley decomposition is a sum of commuting diagonalizable and nilpotent parts, and the Toeplitz decomposition is a sum $H + K$ of a symmetric matrix $H$ and a skew-symmetric matrix $K$.

All definitions · All distinctions · All examples