Calculus of variations vs. optimal control
The calculus of variations and optimal control seek a function of time that minimizes a cost given by an integral. In the calculus of variations, the unknown is a curve $y(t)$, and the cost depends on $y$ and its derivative $y^{\prime}$, which can be chosen freely. In optimal control, the unknown is an input $u(t)$, which may be constrained to a set $U$, and the state $x(t)$ is determined by the input through the dynamics $\dot{x} = f(x, u)$.
calculus of variations ⊂ optimal control
Definitions
Calculus of variations is the study of finding the best curve or function among all possible curves or functions, where “best” is measured by a number computed from the whole curve, like the length of the curve or the time it takes to slide down the curve.
Calculus of variations is the study of problems of finding a function $y : [a, b] \to \reals^n$, with fixed end values $y(a) = y_a$ and $y(b) = y_b$, that minimizes (or maximizes) a functional of the form
\[J[y] = \int_a^b L\bigl(t, y(t), y^{\prime}(t)\bigr) \, \mathrm{d}t.\]Examples
-
Finding the shortest curve $y(x)$ in the plane from $(0, 0)$ to $(1, 1)$
Finding the shortest curve from $(0, 0)$ to $(1, 1)$ is a problem in the calculus of variations, because the unknown is a function $y$ with fixed end values, and the length is the integral of $L(y^{\prime}) = \sqrt{1 + (y^{\prime})^2}$. The Euler–Lagrange equation gives $y^{\prime\prime} = 0$, so the shortest curve is the straight line $y = x$.
-
Minimizing $\int_0^\infty \bigl(x(t)^2 + u(t)^2\bigr) \, \mathrm{d}t$ for the system $\dot{x} = x + u$, with $x(0) = 1$
Minimizing $\int_0^\infty (x^2 + u^2) \, \mathrm{d}t$ for $\dot{x} = x + u$ is also a problem in the calculus of variations, because the input is unconstrained, so substituting $u = \dot{x} - x$ gives the functional $\int_0^\infty \bigl(x^2 + (\dot{x} - x)^2\bigr) \, \mathrm{d}t$ of $x$ alone.
Non-examples
-
Moving a cart from rest at position $0$ to rest at position $1$ in the shortest time, with an acceleration $u(t)$ between $-1$ and $1$
Moving a cart in minimum time with acceleration between $-1$ and $1$ is not a calculus of variations problem, because the bound $\abs{\ddot{x}} \leq 1$ constrains the derivatives of the unknown function, and the optimal acceleration jumps from $1$ to $-1$, so the optimal position is not a smooth solution of an Euler–Lagrange equation.
Properties
- If a continuously differentiable function $y$ minimizes $J$, then $y$ satisfies the Euler–Lagrange equation $\frac{\mathrm{d}}{\mathrm{d}t} \frac{\partial L}{\partial y^{\prime}} = \frac{\partial L}{\partial y}$.
- A problem in the calculus of variations is the optimal control problem with the dynamics $\dot{x} = u$ and no constraints on the input $u$, where the state $x$ plays the role of $y$.
Optimal control is the study of choosing the inputs to a system over time to make the behavior of the system as good as possible, measured by a cost, while the system obeys its equations of motion.
Consider a control system $\dot{x} = f(x, u)$ with state $x(t) \in \reals^n$, input $u(t)$ in a set $U$ of allowed inputs, and initial state $x_0$. Optimal control is the study of problems of the form
\[\begin{aligned} \operatorname*{minimize}_{u(\cdot)} \quad & \varphi\bigl(x(T)\bigr) + \int_0^T \ell\bigl(x(t), u(t)\bigr) \, \mathrm{d}t \\ \text{subject to} \quad & \dot{x}(t) = f\bigl(x(t), u(t)\bigr) \\ & x(0) = x_0 \\ & u(t) \in U \text{ for all } t, \end{aligned}\]where the unknown is the input $u$ as a function of time.
Examples
-
Finding the shortest curve $y(x)$ in the plane from $(0, 0)$ to $(1, 1)$
Finding the shortest curve from $(0, 0)$ to $(1, 1)$ is an optimal control problem with the state $y$, the input $u = y^{\prime}$, the dynamics $\dot{y} = u$, and the cost $\int_0^1 \sqrt{1 + u^2} \, \mathrm{d}x$.
-
Moving a cart from rest at position $0$ to rest at position $1$ in the shortest time, with an acceleration $u(t)$ between $-1$ and $1$
Moving a cart in minimum time with acceleration between $-1$ and $1$ is an optimal control problem with the bounded input $u$. The optimal input is “bang-bang”: $u = 1$ until $t = 1$, then $u = -1$ until the cart stops at $T = 2$.
-
Minimizing $\int_0^\infty \bigl(x(t)^2 + u(t)^2\bigr) \, \mathrm{d}t$ for the system $\dot{x} = x + u$, with $x(0) = 1$
Minimizing $\int_0^\infty (x^2 + u^2) \, \mathrm{d}t$ for $\dot{x} = x + u$ is an optimal control problem, a linear-quadratic regulator problem, whose solution is the feedback $u = -(1 + \sqrt{2}) x$.
Properties
- Pontryagin’s maximum principle gives necessary conditions for an input to be optimal, in terms of a Hamiltonian $H(x, u, \lambda) = \ell(x, u) + \lambda^\top f(x, u)$ and a costate $\lambda$.
- The optimal cost, as a function of the initial state and time, satisfies the Hamilton–Jacobi–Bellman partial differential equation.
- Every problem in the calculus of variations is an optimal control problem with the dynamics $\dot{x} = u$ and $U = \reals^n$.
Examples
| Example | calculus of variations | optimal control | Why |
|---|---|---|---|
| Finding the shortest curve $y(x)$ in the plane from $(0, 0)$ to $(1, 1)$ | ✓ | ✓ | |
| Moving a cart from rest at position $0$ to rest at position $1$ in the shortest time, with an acceleration $u(t)$ between $-1$ and $1$ | ✗ | ✓ | Moving a cart in minimum time is an optimal control problem but not a problem in the calculus of variations, because the input is bounded, and the optimal input jumps between its bounds. |
| Minimizing $\int_0^\infty \bigl(x(t)^2 + u(t)^2\bigr) \, \mathrm{d}t$ for the system $\dot{x} = x + u$, with $x(0) = 1$ | ✓ | ✓ | Minimizing $\int_0^\infty (x^2 + u^2) \, \mathrm{d}t$ for $\dot{x} = x + u$ is posed as an optimal control problem, but the input is unconstrained and can be solved for from the dynamics, so the problem is also a problem in the calculus of variations. |
Notes
- A problem in the calculus of variations is the optimal control problem with the dynamics $\dot{x} = u$ and no constraints on $u$.
- Pontryagin’s maximum principle in optimal control generalizes the Euler–Lagrange equation of the calculus of variations to constrained inputs, where the optimal input can jump (“bang-bang” control).
- The Hamiltonian of Pontryagin’s maximum principle is named after the Hamiltonian of mechanics, which comes from the calculus of variations through Hamilton’s principle.