Linear Algebra, Analysis · Siblings

Norms, metrics, and inner products measure size, distance, and angle, and each one gives the next in a natural way. A norm measures the lengths of vectors, a metric measures distances between points, and an inner product measures both lengths and angles. An inner product gives a norm, and a norm gives a metric, but not the reverse.

An inner product gives a norm, $\norm{x} = \sqrt{\ip{x}{x}}$, and a norm gives a metric, $d(x, y) = \norm{x - y}$.

Definitions

norm

Elementary

A vector is arrow between two points. Such a vector has a direction and a length. The norm of the vector is its length: how long the arrow is.

High school

Consider a vector $x = (x_1, \dots, x_n)$. The norm of $x$ is its length:

\[\norm{x} = \sqrt{x_1^2 + \dots + x_n^2}.\]
Caveat. This is the Euclidean norm, which is only one of many norms. Others include $\abs{x_1} + \dots + \abs{x_n}$ and $\max_i \abs{x_i}$.
Undergraduate

Consider a vector space $V$ and a function $\norm{\cdot} : V \to [0, \infty)$. The function is a norm on $V$ if

  1. $\norm{x} = 0$ only for $x = 0$
  2. $\norm{c x} = \abs{c} \norm{x}$, and
  3. $\norm{x + y} \leq \norm{x} + \norm{y}$ (the triangle inequality).
Examples
Non-examples
Properties
  • Every norm satisfies $\abs{\norm{x} - \norm{y}} \leq \norm{x - y}$, so it is a continuous function.
  • On a finite-dimensional space, all norms are equivalent: they give the same open sets and the same convergent sequences.

metric

High school

A metric is a way of generalizing the concept of distance. Consider a set of points, and a rule that gives a “distance” between every two of them. The rule is a metric if

  1. no distance is negative,
  2. the distance is zero only from a point to itself,
  3. the distance is the same in both directions, and
  4. no distance is longer than a detour through a third point (the triangle inequality).
Undergraduate

Consider a set $X$ and a function $d : X \times X \to [0, \infty)$. The function $d$ is a metric on $X$ if, for all $x, y, z \in X$,

  1. $d(x, y) = 0$ if and only if $x = y$,
  2. $d(x, y) = d(y, x)$ (symmetry), and
  3. $d(x, z) \leq d(x, y) + d(y, z)$ (the triangle inequality).

The last condition is the triangle inequality. The set $X$ need not be a vector space.

Examples

inner product

High school

An inner product is a way of multiplying two vectors to get a number, generalizing the dot product, that gives a way to measure lengths and angles.

Undergraduate (real vector spaces)

Consider a real vector space $V$ and a function $\ip{\cdot}{\cdot} : V \times V \to \reals$. The function is an inner product if

  1. it is linear in its first argument: $\ip{a x + y}{z} = a \ip{x}{z} + \ip{y}{z}$ for all $x, y, z \in V$ and $a \in \reals$,
  2. it is symmetric: $\ip{x}{y} = \ip{y}{x}$ for all $x, y \in V$, and
  3. it is positive definite: $\ip{x}{x} > 0$ for for all $x \neq 0 in $V$.
Caveat. For complex vector spaces, symmetry is replaced by conjugate symmetry, $\ip{x}{y} = \overline{\ip{y}{x}}$, so the inner product is linear in one argument and conjugate-linear in the other.
Graduate (real or complex vector spaces)

Consider a vector space $V$ over $\mathbb{F}$, where $\mathbb{F}$ is $\reals$ or $\mathbb{C}$, and a sesquilinear form $\ip{\cdot}{\cdot}$ on $V$: a function $V \times V \to \mathbb{F}$ that is linear in its first argument and conjugate-linear in its second. The form is an inner product if, for all $x, y \in V$,

  1. it is Hermitian: $\ip{x}{y} = \overline{\ip{y}{x}}$, and
  2. it is positive definite: $\ip{x}{x} > 0$ for $x \neq 0$.

Over $\reals$, a sesquilinear form is a bilinear form, and Hermitian means symmetric.

Examples
Non-examples

Examples

Example norm metric inner product Why
The Euclidean norm $x \mapsto \sqrt{x_1^2 + x_2^2}$ on $\reals^2$ ✓–– The Euclidean norm, which gives the length of a vector in the plane.
$x \mapsto \abs{x_1} + \abs{x_2}$ on $\reals^2$ ✓–✗
$d(x, y) = \sqrt{(x_1 - y_1)^2 + (x_2 - y_2)^2}$ on $\reals^2$ –✓– The Euclidean distance between points in the plane.
$d(x, y) = 1$ for $x \neq y$, and $d(x, x) = 0$ (definition) ✗✓–
Great-circle distance on the surface of the Earth ✗✓–
$(x, y) \mapsto x_1 y_1 + x_2 y_2$ on $\reals^2$ ––✓

Notes

  • A norm comes from an inner product exactly when it satisfies the parallelogram law,

    \[\norm{x + y}^2 + \norm{x - y}^2 = 2 \norm{x}^2 + 2 \norm{y}^2.\]
  • A metric comes from a norm only if it is translation invariant ($d(x + z, y + z) = d(x, y)$) and scales ($d(c x, c y) = \abs{c} d(x, y)$).
  • Each structure needs a different setting. An inner product and a norm are defined only on a (real or complex) vector space, because their definitions add vectors and multiply them by scalars. A metric needs only a set: the discrete metric works on any set, and the great-circle distance works on a sphere, which is not a vector space. So every inner product space is a normed space, and every normed space is a metric space, but a metric space need not have a norm or an inner product.

See also

All distinctions · All definitions