Linear Algebra · Siblings

A norm measures the size of a vector, and $\reals^n$ has several common norms that measure size in different ways. The Euclidean norm is the straight-line length, the 1-norm adds the sizes of the entries, and the infinity norm takes the largest entry. Each is a norm, so each satisfies the triangle inequality, and for every vector, $\norm{x}_{\infty} \leq \norm{x}_2 \leq \norm{x}_1$.

Definitions

Euclidean norm

High school

Consider a vector $x = (x_1, \dots, x_n)$, drawn as an arrow from the origin. The Euclidean norm of $x$ is the length of the arrow:

\[\norm{x}_2 = \sqrt{x_1^2 + \dots + x_n^2}.\]
Examples
  • The vector $x = (1, 1) \in \reals^2$ : $\norm{x}_2 = \sqrt{1^2 + 1^2} = \sqrt{2}$
    The three norms of $(1, 1)$ are as far apart as possible for a vector in $\reals^2$: the 1-norm is $\sqrt{2}$ times the Euclidean norm, which is $\sqrt{2}$ times the infinity norm.
  • The vector $x = (3, -4) \in \reals^2$ : $\norm{x}_2 = \sqrt{3^2 + (-4)^2} = 5$
    The Euclidean norm of $(3, -4)$ is the length of the arrow from the origin, $5$.
  • The vector $x = (5, 0) \in \reals^2$ : $\norm{x}_2 = \sqrt{5^2 + 0^2} = 5$
    All three norms of $(5, 0)$ equal $5$, because the vector lies along an axis: the norms differ only for vectors that point between the axes.
Properties
  • The Euclidean norm comes from the dot product, as $\norm{x}_2 = \sqrt{x \cdot x}$, so it is the only one of the common norms on $\reals^n$ that measures angles as well as lengths.
  • Rotations and reflections do not change the Euclidean norm of a vector.

1-norm

High school

Consider a vector $x = (x_1, \dots, x_n)$. The 1-norm of $x$ is the sum of the absolute values of its entries:

\[\norm{x}_1 = \abs{x_1} + \dots + \abs{x_n}.\]
Examples
Properties
  • In the plane, $\norm{x - y}1$ is the distance from $y$ to $x$ when traveling only along horizontal and vertical streets, as in a city grid. The is why the 1-norm is sometimes called the _taxicab norm or Manhattan norm.
  • For every $x \in \reals^n$, $\norm{x}_{\infty} \leq \norm{x}_2 \leq \norm{x}_1$.

infinity norm

High school

Consider a vector $x = (x_1, \dots, x_n)$. The infinity norm of $x$ is the largest absolute value of its entries:

\[\norm{x}_{\infty} = \max_i \abs{x_i}.\]
Examples
Properties
  • The infinity norm is the limit of the $p$-norms $\norm{x}_p = (\abs{x_1}^p + \dots + \abs{x_n}^p)^{1/p}$ as $p \to \infty$, which explains its name.
  • For functions on an interval, the analogous norm is the sup norm $\sup_t \abs{f(t)}$, which measures the largest deviation of $f$ from $0$.

Examples

Each cell is the norm of the vector in that row.

Example Euclidean norm 1-norm infinity norm Why
The vector $x = (3, -4) \in \reals^2$ $\norm{x}_2 = \sqrt{3^2 + (-4)^2} = 5$ $\norm{x}_1 = \abs{3} + \abs{-4} = 7$ $\norm{x}_{\infty} = \max(\abs{3}, \abs{-4}) = 4$
The vector $x = (1, 1) \in \reals^2$ $\norm{x}_2 = \sqrt{1^2 + 1^2} = \sqrt{2}$ $\norm{x}_1 = \abs{1} + \abs{1} = 2$ $\norm{x}_{\infty} = \max(\abs{1}, \abs{1}) = 1$
The vector $x = (5, 0) \in \reals^2$ $\norm{x}_2 = \sqrt{5^2 + 0^2} = 5$ $\norm{x}_1 = \abs{5} + \abs{0} = 5$ $\norm{x}_{\infty} = \max(\abs{5}, \abs{0}) = 5$ All three norms of $(5, 0)$ equal $5$, because the vector lies along an axis: the norms differ only for vectors that point between the axes.

Notes

  • The unit “circles” $\setdef{x \in \reals^2 \suchthat \norm{x} = 1}$ of the three norms are a circle (Euclidean), a diamond (1-norm), and a square (infinity norm).
  • All three are special cases of the $p$-norm $\norm{x}_p = (\abs{x_1}^p + \dots + \abs{x_n}^p)^{1/p}$, with $p = 2$, $1$, and $\infty$.
  • On $\reals^n$, all norms are equivalent: each is within a constant factor of the others, so they agree on which sequences converge.
  • A norm is defined only on a vector space, such as $\reals^n$ or a space of functions, because the definition uses scaling, $\norm{c x} = \abs{c} \norm{x}$. On a set without vector operations, such as a sphere, use a metric instead.

See also

All distinctions · All definitions