Linear Algebra · Siblings

Eigenvalues and singular values both describe how a matrix stretches vectors, but they agree only for special matrices. An eigenvalue of a square matrix $A$ is a factor $\lambda$ by which $A$ scales a particular nonzero vector $x$, called an eigenvector: $Ax = \lambda x$. A singular value of a matrix $A$, square or not, is a factor $\sigma$ by which $A$ stretches vectors along some direction, allowing them to turn. Singular values are real and nonnegative, but eigenvalues can be negative or complex.

Definitions

eigenvalue

High school

Consider a square matrix $A$, which stretches and turns vectors when it multiplies them. An eigenvalue of $A$ is a number $\lambda$ such that $A$ stretches some nonzero vector $v$ by the factor $\lambda$ without turning it off its line:

\[A v = \lambda v.\]
Caveat. An eigenvalue can be a complex number, with a complex eigenvector. Then the picture of stretching without turning does not apply in the real plane: for example, the matrix that rotates the plane by $90°$ turns every real vector off its line, but it has the complex eigenvalues $i$ and $-i$.
Undergraduate

Consider a square matrix $A$. An eigenvalue of $A$ is a number $\lambda$ such that

\[A v = \lambda v\]

for some nonzero vector $v$, called an eigenvector.

Examples
  • The matrix \(\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) : $i$ and $-i$
    The eigenvalues of \(A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) are $i$ and $-i$, because \(A \begin{bmatrix} 1 \\ -i \end{bmatrix} = \begin{bmatrix} i \\ 1 \end{bmatrix} = i \begin{bmatrix} 1 \\ -i \end{bmatrix}\) and \(A \begin{bmatrix} 1 \\ i \end{bmatrix} = \begin{bmatrix} -i \\ 1 \end{bmatrix} = -i \begin{bmatrix} 1 \\ i \end{bmatrix}\). $A$ rotates every nonzero real vector by $90°$, so it has no real eigenvalues.
  • The matrix \(\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) : None
    The matrix \(A = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) has no eigenvalues, because only square matrices have eigenvalues.
  • The matrix \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}\) : $2$ and $-3$
    The eigenvalues are $2$ and $-3$, because \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix} \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 2 \\ 0 \end{bmatrix} = 2 \begin{bmatrix} 1 \\ 0 \end{bmatrix}\) and \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix} \begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ -3 \end{bmatrix} = -3 \begin{bmatrix} 0 \\ 1 \end{bmatrix}\).
  • The matrix \(\begin{bmatrix} 3 & 1 \\ -1 & 1 \end{bmatrix}\) : $2$ (twice)
    The eigenvalue of \(A = \begin{bmatrix} 3 & 1 \\ -1 & 1 \end{bmatrix}\) is $2$, because \(A \begin{bmatrix} 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 2 \\ -2 \end{bmatrix} = 2 \begin{bmatrix} 1 \\ -1 \end{bmatrix}\).
  • The matrix \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) : $1$ (twice)
    The shear \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) has the eigenvalue $1$ twice, and it fixes the vector $(1, 0)$.
Properties
  • The eigenvalues of $A$ are the roots of its characteristic polynomial $\det(\lambda I - A)$.
  • The origin of $\dot{x} = A x$ is asymptotically stable if and only if every eigenvalue of $A$ has negative real part ($A$ is Hurwitz).

singular value

Undergraduate

Consider an $m \times n$ real matrix $A$. The singular values of $A$ are the square roots of the eigenvalues of the $n \times n$ matrix $A^{\top} A$, which are nonnegative.

Examples
  • The matrix \(\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) : $1$ and $1$
    The singular values of \(A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) are $1$ and $1$, because $A^{\top} A = I$, whose eigenvalues are $1$ and $1$. A rotation does not change the length of any vector.
  • The matrix \(\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) : $\sqrt{14}$ and $1$
    The singular values of \(A = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) are $\sqrt{14}$ and $1$, the square roots of the eigenvalues of \(A A^{\top} = \begin{bmatrix} 5 & 6 \\ 6 & 10 \end{bmatrix}\). That matrix has trace $15$ and determinant $50 - 36 = 14$, so its eigenvalues are the roots of $\lambda^2 - 15 \lambda + 14 = (\lambda - 14)(\lambda - 1)$. The $3 \times 3$ matrix $A^{\top} A$ has the same nonzero eigenvalues, plus $0$.
  • The matrix \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}\) : $3$ and $2$
    The singular values of \(A = \begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}\) are $3$ and $2$, the square roots of the eigenvalues $9$ and $4$ of \(A^{\top} A = \begin{bmatrix} 4 & 0 \\ 0 & 9 \end{bmatrix}\). $A$ is symmetric, so its singular values are the absolute values of its eigenvalues.
  • The matrix \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) : $\varphi$ and $1 / \varphi$
    Here $\varphi = \frac{1 + \sqrt{5}}{2}$ is the golden ratio.
Properties
  • Geometrically, $A$ maps the unit sphere to an ellipsoid, and the lengths of the ellipsoid’s semi-axes are the singular values of $A$.
  • The largest singular value of $A$ is its operator norm $\norm{A}_2$, the most that $A$ stretches a vector.
  • For a symmetric matrix, the singular values are the absolute values of the eigenvalues.

Examples

Each cell lists the eigenvalues or the singular values of the matrix in that row.

Example eigenvalue singular value Why
The matrix \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}\) $2$ and $-3$ $3$ and $2$
The matrix \(\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) $i$ and $-i$ $1$ and $1$
The matrix \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) $1$ (twice) $\varphi$ and $1 / \varphi$
The matrix \(\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) None $\sqrt{14}$ and $1$

Notes

  • For a normal matrix ($A^{\top} A = A A^{\top}$, such as a symmetric or orthogonal matrix), the singular values are the absolute values of the eigenvalues.
  • For a square matrix, the product of the singular values is $\abs{\det A}$, the absolute value of the product of the eigenvalues.

See also

All distinctions · All definitions