Types of multivariable derivatives
The directional derivative, the Jacobian, the gradient, the differential, and the Lie derivative all package the partial derivatives of a function differently: as a vector (gradient), a matrix (Jacobian), a linear map (differential), or a single number in a chosen direction (directional derivative) or along a vector field (Lie derivative).
Definitions
Consider a differentiable function $f : \reals^n \to \reals$. The gradient of $f$ is the vector of partial derivatives
\[\nabla f(x) = \left( \frac{\partial f}{\partial x_1}(x), \dots, \frac{\partial f}{\partial x_n}(x) \right).\]Consider a smooth function $f$ on a Riemannian manifold, with inner product $\ip{\cdot}{\cdot}$ on each tangent space. The gradient of $f$ at $x$ is the vector $\nabla f(x)$ such that
\[\ip{\nabla f(x)}{v} = \mathrm{d}f_x(v) \quad \text{for every tangent vector } v.\]Examples
- $f(x, y, z) = x^2 + y^2$ : $\nabla f(x, y, z) = (2x, 2y, 0)$
-
$f(x, y) = x^2 + 3y$
: $\nabla f(x, y) = (2x, 3)$
A vector.
Properties
- If $\nabla f(x) \neq 0$, then it points in the direction of steepest increase of $f$, and its length is the rate of increase in that direction.
- The gradient is perpendicular to the level sets of $f$.
- The gradient depends on the choice of inner product (Riemannian metric); the differential $\mathrm{d}f$ does not.
The Jacobian of a function with several inputs and several outputs is the table of all of its partial derivatives, with one row per output and one column per input.
The Jacobian of $f : \reals^n \to \reals^m$ is the $m \times n$ matrix of partial derivatives, whose entry in row $i$ and column $j$ is $\frac{\partial f_i}{\partial x_j}$.
Examples
-
$f(x, y, z) = x^2 + y^2$
: $Df(x, y, z) = \begin{bmatrix} 2x & 2y & 0 \end{bmatrix}$
The Laplacian of $f$ is the sum of its second partial derivatives, $2 + 2 + 0 = 4$, and its Jacobian is the row of its first partial derivatives.
-
$f(x, y) = x^2 + 3y$
: $Df(x, y) = \begin{bmatrix} 2x & 3 \end{bmatrix}$
A $1 \times 2$ matrix, the transpose of the gradient.
Consider a function of several variables, like the height of a hill at each point of a map. The directional derivative of the function is its vertical rate of change as you move horizontally in a particular direction.
Consider a function $f$, a point $x$, and a direction $v$. The directional derivative of $f$ at $x$ in the direction $v$ is the rate of change of $f$ in that direction:
\[D_v f(x) = \lim_{t \to 0} \frac{f(x + t v) - f(x)}{t}.\]Examples
-
$f(x, y) = x^2 + 3y$
: $D_v f(x, y) = \nabla f(x, y) \cdot v = 2x$, in the direction $v = (1, 0)$
$\nabla f \cdot (1, 0) = 2x$.
Properties
- If $f$ is differentiable, then $D_v f(x) = \nabla f(x) \cdot v$.
- A function can have directional derivatives in every direction and still not be differentiable, or even continuous.
Consider a differentiable function $f$ and a point $x$ in its domain. The differential of $f$ at $x$ is the linear map $\mathrm{d}f_x : v \mapsto Df(x) v$ that best approximates $f(x + v) - f(x)$ for small $v$. For $f : \reals^n \to \reals$,
\[\mathrm{d}f_x(v) = \frac{\partial f}{\partial x_1}(x) v_1 + \dots + \frac{\partial f}{\partial x_n}(x) v_n.\]Consider a smooth function $f : M \to \reals$ on a manifold $M$. The differential of $f$ is the one-form $\mathrm{d}f$, which assigns to each point $x$ the covector $\mathrm{d}f_x$ on the tangent space $T_x M$.
Examples
-
$f(t, x) = t x^2$
: The linear map $\mathrm{d}f_{(t, x)} : (\Delta t, \Delta x) \mapsto x^2 \Delta t + 2 t x \Delta x$
The differential of $f(t, x) = t x^2$ combines both partial derivatives into the linear approximation $Df(t, x)$.
-
$f(x, y) = x^2 + 3y$
: $\mathrm{d}f_{(x, y)} : (v_1, v_2) \mapsto 2x v_1 + 3 v_2$
A linear map that takes a direction and returns a rate of change.
Consider a function $V$ and a vector field $F$, with solutions of $\dot{x} = F(x)$. The Lie derivative of $V$ along $F$ is
\[L_F V(x) = \nabla V(x) \cdot F(x).\]It is the rate of change of $V$ along those solutions.
Consider a function $V$ and a vector field $F$ on a manifold, where $\phi_t$ is the flow of $F$. The Lie derivative of $V$ along $F$ is
\[L_F V(x) = \frac{\mathrm{d}}{\mathrm{d}t} V(\phi_t(x)) \big\rvert_{t = 0}.\]Lie derivatives are also defined for vector fields and differential forms.
Examples
-
$f(x, y) = x^2 + 3y$
: $L_F f(x, y) = \nabla f(x, y) \cdot F(x, y) = -2x^2$, along the vector field $F(x, y) = (-x, 0)$
$\nabla f \cdot F = 2x \cdot (-x) + 3 \cdot 0$. The direction varies from point to point.
Properties
- If $L_F V(x) \leq 0$ for all $x$, then $V$ never increases along solutions of $\dot{x} = F(x)$. This is how Lyapunov functions are checked, without solving the system.
Examples
Each cell is the ___ of the function in that row.
$f(x, y) = x^2 + 3y$
The gradient of $f$ is a vector, its Jacobian is the $1 \times 2$ matrix with the same entries, and its differential is the linear map given by that matrix. The directional derivative needs a direction $v$, and the Lie derivative uses a direction $F(x, y)$ that varies from point to point.
| gradient | $\nabla f(x, y) = (2x, 3)$ |
|---|---|
| Jacobian | $Df(x, y) = \begin{bmatrix} 2x & 3 \end{bmatrix}$ |
| directional derivative | $D_v f(x, y) = \nabla f(x, y) \cdot v = 2x$, in the direction $v = (1, 0)$ |
| differential | $\mathrm{d}f_{(x, y)} : (v_1, v_2) \mapsto 2x v_1 + 3 v_2$ |
| Lie derivative | $L_F f(x, y) = \nabla f(x, y) \cdot F(x, y) = -2x^2$, along the vector field $F(x, y) = (-x, 0)$ |
Notes
- At each point, the gradient is a vector and the differential is a covector. The differential needs no extra structure, but the gradient depends on an inner product: $\nabla f(x)$ is the vector with $\ip{\nabla f(x)}{v} = \mathrm{d}f_x(v)$ for every $v$. With the dot product, they have the same components in the standard basis of $\reals^n$ (or every orthonormal basis), but not in other coordinates. In polar coordinates, for example, $\mathrm{d}f$ has components $(\partial_r f, \partial_\theta f)$, while $\nabla f$ has components $(\partial_r f, \frac{1}{r^2} \partial_\theta f)$ in the coordinate basis $(\partial_r, \partial_\theta)$. See vector vs. covector.