Linear Algebra · Siblings

An eigenvalue of a square matrix $A$ is a factor $\lambda$ by which $A$ scales a particular nonzero vector $x$, called a eigenvalue: $Ax = \lambda x$. A singular value of any matrix $A$ is a factor $\sigma$ by which $A$ stretches vectors along some direction, allowing it them turn. Singular values are real and nonnegative, but eigenvalues can be negative or complex.

Definitions

eigenvalue

High school

Consider a square matrix $A$, which stretches and turns vectors when it multiplies them. An eigenvalue of $A$ is a number $\lambda$ such that $A$ stretches some nonzero vector $v$ by the factor $\lambda$ without turning it off its line: $A v = \lambda v$.

Caveat. An eigenvalue can be a complex number, with a complex eigenvector. Then the picture of stretching without turning does not apply in the real plane: for example, the matrix that rotates the plane by $90°$ turns every real vector off its line, but it has the complex eigenvalues $i$ and $-i$.
Undergraduate

Consider a square matrix $A$. An eigenvalue of $A$ is a number $\lambda$ such that $A v = \lambda v$ for some nonzero vector $v$, called an eigenvector.

Examples
  • The matrix \(\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) : $i$ and $-i$
    The eigenvalues of \(A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) are $i$ and $-i$, because \(A \begin{bmatrix} 1 \\ -i \end{bmatrix} = \begin{bmatrix} i \\ 1 \end{bmatrix} = i \begin{bmatrix} 1 \\ -i \end{bmatrix}\) and \(A \begin{bmatrix} 1 \\ i \end{bmatrix} = \begin{bmatrix} -i \\ 1 \end{bmatrix} = -i \begin{bmatrix} 1 \\ i \end{bmatrix}\). $A$ rotates every nonzero real vector by $90°$, so it has no real eigenvalues.
  • The matrix \(\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) : None, since the matrix is not square
    The $2 \times 3$ matrix has no eigenvalues, because only square matrices have them. Its singular values are $\sqrt{14}$ and $1$, the square roots of the eigenvalues $14$ and $1$ of $A A^{\top}$ (which are the nonzero eigenvalues of $A^{\top} A$).
  • The matrix \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}\) : $2$ and $-3$
    The eigenvalues are $2$ and $-3$, because \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix} \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 2 \\ 0 \end{bmatrix} = 2 \begin{bmatrix} 1 \\ 0 \end{bmatrix}\) and \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix} \begin{bmatrix} 0 \\ 1 \end{bmatrix} = \begin{bmatrix} 0 \\ -3 \end{bmatrix} = -3 \begin{bmatrix} 0 \\ 1 \end{bmatrix}\).
  • The matrix \(\begin{bmatrix} 3 & 1 \\ -1 & 1 \end{bmatrix}\) : $2$ (twice)
    The eigenvalue of \(A = \begin{bmatrix} 3 & 1 \\ -1 & 1 \end{bmatrix}\) is $2$, because \(A \begin{bmatrix} 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 2 \\ -2 \end{bmatrix} = 2 \begin{bmatrix} 1 \\ -1 \end{bmatrix}\).
  • The matrix \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) : $1$ (twice)
    The shear \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) has the eigenvalue $1$ twice, and it fixes the vector $(1, 0)$. Its singular values are $\varphi \approx 1.618$ and $1 / \varphi \approx 0.618$, the square roots of the eigenvalues $\frac{3 \pm \sqrt{5}}{2}$ of $A^{\top} A$, because the shear stretches some directions and shrinks others.
Properties
  • The eigenvalues of $A$ are the roots of its characteristic polynomial $\det(\lambda I - A)$.
  • The origin of $\dot{x} = A x$ is asymptotically stable if and only if every eigenvalue of $A$ has negative real part ($A$ is Hurwitz).

singular value

Undergraduate

Consider an $m \times n$ real matrix $A$. The singular values of $A$ are the square roots of the eigenvalues of the $n \times n$ matrix $A^{\top} A$, which are nonnegative.

Examples
  • The matrix \(\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) : $1$ and $1$
    The singular values of \(A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) are $1$ and $1$, because $A^{\top} A = I$, whose eigenvalues are $1$ and $1$. A rotation does not change the length of any vector.
  • The matrix \(\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) : $\sqrt{14}$ and $1$
    The singular values of \(A = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) are $\sqrt{14}$ and $1$, the square roots of the eigenvalues of \(A A^{\top} = \begin{bmatrix} 5 & 6 \\ 6 & 10 \end{bmatrix}\). That matrix has trace $15$ and determinant $50 - 36 = 14$, so its eigenvalues are the roots of $\lambda^2 - 15 \lambda + 14 = (\lambda - 14)(\lambda - 1)$. The $3 \times 3$ matrix $A^{\top} A$ has the same nonzero eigenvalues, plus $0$.
  • The matrix \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}\) : $3$ and $2$
    The singular values of \(A = \begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}\) are $3$ and $2$, the square roots of the eigenvalues $9$ and $4$ of \(A^{\top} A = \begin{bmatrix} 4 & 0 \\ 0 & 9 \end{bmatrix}\). $A$ is symmetric, so its singular values are the absolute values of its eigenvalues.
  • The matrix \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) : $\varphi$ and $1 / \varphi$, where $\varphi = \frac{1 + \sqrt{5}}{2}$
    The shear \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) has the eigenvalue $1$ twice, and it fixes the vector $(1, 0)$. Its singular values are $\varphi \approx 1.618$ and $1 / \varphi \approx 0.618$, the square roots of the eigenvalues $\frac{3 \pm \sqrt{5}}{2}$ of $A^{\top} A$, because the shear stretches some directions and shrinks others.
Properties
  • Geometrically, $A$ maps the unit sphere to an ellipsoid, and the lengths of the ellipsoid’s semi-axes are the singular values of $A$.
  • The largest singular value of $A$ is its operator norm $\lVert A \rVert_2$, the most that $A$ stretches a vector.
  • For a symmetric matrix, the singular values are the absolute values of the eigenvalues.

Examples

Each cell lists the eigenvalues or the singular values of the matrix in that row.

Example eigenvalue singular value Why
The matrix \(\begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}\) $2$ and $-3$ $3$ and $2$ The diagonal matrix with entries $2$ and $-3$ has eigenvalues $2$ and $-3$, its diagonal entries. Its singular values are their absolute values, $3$ and $2$, as for every symmetric matrix.
The matrix \(\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) $i$ and $-i$ $1$ and $1$ The rotation by $90°$ has the eigenvalues $\pm i$, because it turns every nonzero real vector off its line. Its singular values are $1$ and $1$, because a rotation does not change lengths.
The matrix \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) $1$ (twice) $\varphi$ and $1 / \varphi$, where $\varphi = \frac{1 + \sqrt{5}}{2}$ The shear \(\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\) has the eigenvalue $1$ twice, and it fixes the vector $(1, 0)$. Its singular values are $\varphi \approx 1.618$ and $1 / \varphi \approx 0.618$, the square roots of the eigenvalues $\frac{3 \pm \sqrt{5}}{2}$ of $A^{\top} A$, because the shear stretches some directions and shrinks others.
The matrix \(\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}\) None, since the matrix is not square $\sqrt{14}$ and $1$ The $2 \times 3$ matrix has no eigenvalues, because only square matrices have them. Its singular values are $\sqrt{14}$ and $1$, the square roots of the eigenvalues $14$ and $1$ of $A A^{\top}$ (which are the nonzero eigenvalues of $A^{\top} A$).

Notes

  • For a normal matrix ($A^{\top} A = A A^{\top}$, such as a symmetric or orthogonal matrix), the singular values are the absolute values of the eigenvalues.
  • For a square matrix, the product of the singular values is $\lvert \det A \rvert$, the absolute value of the product of the eigenvalues.

See also

All distinctions · All definitions