Optimal control
High school
Optimal control is the study of choosing the inputs to a system over time to make the behavior of the system as good as possible, measured by a cost, while the system obeys its equations of motion.
Undergraduate (control theory)
Consider a control system $\dot{x} = f(x, u)$ with state $x(t) \in \reals^n$, input $u(t)$ in a set $U$ of allowed inputs, and initial state $x_0$. Optimal control is the study of problems of the form
\[\begin{aligned} \operatorname*{minimize}_{u(\cdot)} \quad & \varphi\bigl(x(T)\bigr) + \int_0^T \ell\bigl(x(t), u(t)\bigr) \, \mathrm{d}t \\ \text{subject to} \quad & \dot{x}(t) = f\bigl(x(t), u(t)\bigr) \\ & x(0) = x_0 \\ & u(t) \in U \text{ for all } t, \end{aligned}\]where the unknown is the input $u$ as a function of time.
Examples
Examples
-
Finding the shortest curve $y(x)$ in the plane from $(0, 0)$ to $(1, 1)$
Undergraduate
Finding the shortest curve from $(0, 0)$ to $(1, 1)$ is an optimal control problem with the state $y$, the input $u = y^{\prime}$, the dynamics $\dot{y} = u$, and the cost $\int_0^1 \sqrt{1 + u^2} \, \mathrm{d}x$.
-
Moving a cart from rest at position $0$ to rest at position $1$ in the shortest time, with an acceleration $u(t)$ between $-1$ and $1$
Undergraduate
Moving a cart in minimum time with acceleration between $-1$ and $1$ is an optimal control problem with the bounded input $u$. The optimal input is “bang-bang”: $u = 1$ until $t = 1$, then $u = -1$ until the cart stops at $T = 2$.
-
Minimizing $\int_0^\infty \bigl(x(t)^2 + u(t)^2\bigr) \, \mathrm{d}t$ for the system $\dot{x} = x + u$, with $x(0) = 1$
Graduate
Minimizing $\int_0^\infty (x^2 + u^2) \, \mathrm{d}t$ for $\dot{x} = x + u$ is an optimal control problem, a linear-quadratic regulator problem, whose solution is the feedback $u = -(1 + \sqrt{2}) x$.
Properties
- Pontryagin’s maximum principle gives necessary conditions for an input to be optimal, in terms of a Hamiltonian $H(x, u, \lambda) = \ell(x, u) + \lambda^\top f(x, u)$ and a costate $\lambda$.
- The optimal cost, as a function of the initial state and time, satisfies the Hamilton–Jacobi–Bellman partial differential equation.
- Every problem in the calculus of variations is an optimal control problem with the dynamics $\dot{x} = u$ and $U = \reals^n$.
Distinctions involving this term
- Calculus of variations vs. optimal control: The calculus of variations and optimal control seek a function of time that minimizes a cost given by an integral. In the calculus of variations, the unknown is a curve $y(t)$, and the cost depends on $y$ and its derivative $y^{\prime}$, which can be chosen freely. In optimal control, the unknown is an input $u(t)$, which may be constrained to a set $U$, and the state $x(t)$ is determined by the input through the dynamics $\dot{x} = f(x, u)$.