In dynamical systems and mathematical control theory, it is often useful to show that a given function has a particular sign for all inputs in given sets. For example, given $f : \realsn \to \reals$ and $S\subset \realsn$, we may want to show $f(x) \geq 0$ for all $x \in S$. Prominent examples include Lyapunov functions, which are used to prove stability, and barrier functions which are used to prove set invariance.
As a simple example, consider a dynamical system with state vector \(x \in \realsn\), vector field \(f : \realsn \to \realsn\), and dynamics given by
\begin{equation} \label{eq:ode} \dot x = f(x). \end{equation}
Suppose we want to show that the origin \(0 \in \realsn\) is stable for (\ref{eq:ode}). Roughly speaking, this means that all solutions that start near the origin remain near it for all time. A common approach to show stability is to pick a neighborhood \(U\) of the origin and construct a differentiable function $V : U \to \reals$, called a Lypunov function that satisfies the following criteria:
where \(\dot V(x)\) is the rate of change of \(V(x(t))\) as \(t \mapsto x(t)\) evolves according to (\ref{eq:ode}). The existence of a Lyapunov function proves that the origin is stable [1, Theorem 4.1].
Constructing a Lyapunov is often difficult because it requires finding a function that satisfies multiple inequalities at every point in a set, and the inequalities depend on the function itself, its derivative, and the dynamics of the system. The purpose of this page is to introduce sumofsquares (SOS) programming, which can be used to automatically generate Lyapunov functions and solve other similar problems.
Remark. There are actually many different variants of Lyapunov functions for showing…
Most of these variants can be handled by SOS programming, so long as the data of the system are given in terms of polynomials.
The basis for SOS programming lies in the following fact: For any function $f : \realsn \to \reals,$ let $f^2$ indicate the function $x \mapsto (f(x))^2$. Then, for any choice of functions \(f_1,\ f_2,\ \dots,\ f_N : \realsn \to \reals\),
\[f_1^2(x) + f_2^2(x) + \cdots + f_N^2(x) \geq 0 \quad \forall x \in \realsn\]and the sum is zero if and only if \(f_1(x) = f_2(x) = \cdots = f_N(x) = 0.\)
Working with general functions is difficult, so we restrict the choices for \(f_1,\ f_2,\ \dots,\ f_N\) to polynomials. In particular, we allow for multivariate polynomials, such as $(x, y) \mapsto x^2 + xy + 3y^2 $ or $(x, y, z) \mapsto 1 + x^3  2xyz^2$.
We say that a polynomial \(p : \reals^n \to \reals\) is a sum of squares if there exist polynomials \(p_1, p_2, \dots, p_m : \reals^n \to \reals\) such that
\begin{equation} \label{eq:sos} p(x) = \sum_{i=1}^m p_i^2(x). \end{equation}
Note that the argument $x$ for $p$ is a vector $x \in \realsn$, so we may write it as $x = (x_1, x_2, \dots, x_n)$. We denote the set of all SOS polynomials over $x$ by $\SOSpolys[x]$ or $\SOSpolys[x_1, x_2, \dots, x_n]$. (This notation is used in [2], but there does not appear to be a prevailing standard. Other notations used in the literature include $\mathscr{P}^{\mathup{SOS}}$ [3], and simply $\mathup{SOS}$.)
To solve problems using SOS polynomials, we need to put problems into a standard problem format, described in the next section, which can be solved algorithmically using a computer.
In general, an SOS optimization problem (also called an SOS program) has a linear cost function and one or more SOS constraints—that is, the constraints of the optimization problem require that certain polynomials are sums of squares.
To give a general formulation of an SOS problem, we denote the decision variables as $u = (u_1, u_2, \dots, u_n) \in \reals^m$. The linear cost function is defined by $c^\top u$ for some vector $c\in \reals^m$. To specify an SOS problem with $N$ constraints requires picking $N(m+1)$ polynomials, which we write $p_{i,j}$ for $i = 1, 2, \dots, N$ and $j = 0, 1, \dots, m$. An SOS problem is then written
Thus, a solution to (\ref{eq:sos program}) is a vector $u^*\in\reals^m$ that minimizes $c^\top u^*$ while satisfying the requirement that for each $i = 1, 2, \dots, N$, the function
\[x \mapsto p_{i,0}(x) + u^*_1 p_{i,1}(x) + \cdots + u^*_m p_{i,m}(x)\]is a sum of squares.
Remark. At this point, you might be concerned about how to compute a solution to (\ref{eq:sos program}). It turns out that (\ref{eq:sos program}) can be reformulated into a semidefinite program (SDP), which is a type of convex optimization problem that can be efficiently and reliably solved using numerical solvers (assuming a solution exists). We plan to describe how to solve SOS problems in a later post.
Consider a 2D continuoustime dynamical system with state vector $x = (x_1, x_2) \in \reals^2$ and dynamics given by
We can guess the form of a Lyapunov function to be
\[V(x) = u_1 p_1(x_1, x_2) + u_2 p_2(x_1, x_2) + \cdots + u_m p_m(x_1, x_2)\]where each $p_i$ is a polynomial function. Picking $m$ and each $p_i$ is typically a process of trial and error using intuitive guesses based on the structure and complexity of the problem. We will use $m = 3$ and
\[\begin{aligned} p_1(x_1, x_2) := x_1^2 \\ p_2(x_1, x_2) := x_2^2 \\ p_3(x_1, x_2) := x_1x_2 \end{aligned}\]Therefore, the general form of $V$ is \(V(x) = u_1 x_1^2 + u_2 x_2^2 + u_3 x_1x_2.\) Calculating $\dot V,$ we find
\[\begin{aligned} x \mapsto \dot V(x) &= \ip{\nabla V(x)}{f(x)} \\ &= \ip{\begin{bmatrix}2u_1x_1 +u_3 x_2 \\ 2u_2x_2 + u_3 x_1\end{bmatrix}} {\begin{bmatrix}x_2 \\ (x_1  2x_2)(x_1 + x_2)^2 \end{bmatrix}} \\ &= 2u_1x_1x_2 + u_3 x_2^2 \\ &\quad {} + 2u_2x_2(x_1  2x_2)(x_1 + x_2)^2 \\ &\quad {} + u_3 x_1(x_1  2x_2)(x_1 + x_2)^2 \end{aligned}\]We want $V$ to be positive definite relative to the origin, so it is necessary that $V(x) \in \SOSpolys[x]$, but this is not sufficient because an SOS polynomial can be only positive semidefinite rather than positive definite. You can add constraints on $u_1$, $u_2$, and $u_3$ to ensure $V$ is positive definite. The simplest choice is to require that $u_1$ and $u_2$ are positive and $u_3$ is zero, so $V$ is a bowl shaped function:
\[V(x) = u_1 x_1^2 + u_2 x_2^2.\]The constraints are then,
\[\begin{aligned} u_1 &> 0 \\ u_2 &> 0 \\ u_3 &= 0. \end{aligned}\]For numerical reasons, strict inequalities don’t work well in optimization problems, we instead pick some $\epsilon > 0$ and change the constraints to
\[\begin{aligned} u_1 &\geq \epsilon \\ u_2 &\geq \epsilon \\ u_3 &= 0. \end{aligned}\]Rewriting these constraints using matrices, we have
\[\begin{aligned} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix} u &\leq \begin{bmatrix} \epsilon \\ \epsilon \end{bmatrix} \\ \begin{bmatrix} 0 & 0 & 1 \end{bmatrix} u &= 0. \end{aligned}\]Becuase we won’t know a good choice of $\epsilon$ beforehand, we can make it one of the decision variables and make the cost function $(u, \epsilon) \mapsto \epsilon$ so that the optimizer tries to find the largest value of $\epsilon$ such that the problem has solution. We must check the value of $\epsilon$ for the solution to the optmization problem to ensure $\epsilon > 0$.
An unfortunate side effect of this choice is that it removes a degree of freedom by setting $u_3=0$. This can be avoided by instead partitioning $V$ into the sum of a positive semidefinite function and a positive definite function, such as
\[V(x) := \underbrace{u_1 x_1^2 + u_2 x_2^2 + u_3 x_1x_2}_{\textup{positive semidefinite}} + \underbrace{u_4 x_1^2 + u_4 x_2^2}_{\textup{positive definite}}.\]The corresponding constraints are then
\[\begin{aligned} (x \mapsto u_1 x_1^2 + u_2 x_2^2 + u_3 x_1x_2) &\in \SOSpolys[x] \\ \begin{bmatrix} 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{bmatrix} u &\leq \begin{bmatrix} \epsilon \\ \epsilon \end{bmatrix} \end{aligned}\]Conversely, we want $\dot V$ to be negative semidefinite relative to the origin. Thus,
\[\begin{aligned} x \mapsto \dot V(x) \in \SOSpolys[x]. \end{aligned}\]Therefore, the SOS problem formulation is
A function \(f : \realsn \to \reals\) is said to be positive semidefinite if
\[f(x) \geq 0 \quad \forall x \in \realsn\]and \(f(0) = 0.\)
A function \(f : \realsn \to \reals\) is said to be positive definite if
\[f(x) > 0 \quad \forall x \in \realsn \setminus\{0\}\]and \(f(0) = 0.\)
Negative semidefinite and negative definite functions are defined similarly, except with the inequality signs flipped.
Example. The parabola \(x \mapsto x^2\) is positive definite.
Nonexample. The parabola \(x \mapsto x^2 + 1\) is not positive definite because the value at \(x=0\) is \(1\).
Nonexample. In two variables, the function \((x, y) \mapsto x^2\) is not positive definite because the value at \((0, 1)\) is \(0\).
changes
LaTeX package.
To import the package, add \usepackage{changes}
to your document’s preamble.
The changes
package defines four types of annotations: \added
, \deleted
, \replaced
, and \comment
and also imports the \todo
macro from the todo
package.
Example:
Here is \added{added}, \deleted{deleted} and \replaced{replaced}{replaysed} text. \comment{Maybe I shouldn't have written this?} \todo[inline]{Todo: Write something worthwhile.}
Output:
In order for annotations to be useful they must be uptodate. This raises the question of when to remove annotations. The best workflow depends on the way your reviewer gives feedback. If your reviewer will read the entire document, then you can delete annotations immediately after sending them a draft. In my case, however, my advisor only reads a portion of each draft I send, so I leave annotations until he has given feedback. To this motivates the following workflow:
Once the reviewer gives you comments:
When annotating a sentence or more, I format my LaTeX code with \added{
and }
on their own lines.
Including %
immediately after \added{
and after }
prevents LaTeX from inserting extra spaces (LaTeX treats a new line in the code the same as a space).
\added{% Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. }% End \added block
If you use Visual Studio Code, see below for snippets that will wrap selected text in annotation commands.
The commands \added
, \deleted
, and \replaced
cannot contain a paragraph break.
This precludes empty lines, such as
\added{ % Paragraph 1 Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. % Paragraph 2 (Causes error!) Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. }
To mark multiple paragraphs as changed, I define a new color called added
using the xcolor
package
\usepackage{xcolor} \colorlet{added}{blue!80!black}
Then, add \color{added}
before a multiple paragraph change, and add \color{black}
afterward.
\color{added} % Paragraph 1 (Added) Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. % Paragraph 2 (Added) Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. \color{black} % Paragraph 3 (No change) Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Output:
There are various package options. For my documents, I use the following:
\usepackage[ % import "changes" package % If any of the changes commands are already defined, then the option "commandnameprefix=ifneeded" % tells changes to append "ch" to the name of the changes command in order to avoid a name collision. % Commonly, "\comment" will be changed to "\chcomment". commandnameprefix=ifneeded, % Changes imports the "todo" package. The following options are passed to the "todo" package. todonotes={colorinlistoftodos, prependcaption, textsize=small, backgroundcolor=orange!10, textcolor=black, linecolor=orange, bordercolor=orange} % % draft, % < enable line to show annotations regardless of the document being in 'final' mode. % final, % < enable line to hide annotations regardless of the document being in 'draft' mode. ]{changes}
For comments in the margin, the margin size for many document classes is too narrow, so it is necessary to adjust it.
One way to this is with the geometry
package.
In the following snippet, we also use the ifdraft
package so that our changes to the margins only apply in draft mode.
% Create \ifdraft{}{} conditional % that switches based on whether "draft" % is passed to document class. \usepackage{ifdraft} \ifdraft{ % Adjust spacing to fit margin notes. \usepackage[inner=20mm, outer=40mm, marginparwidth=34mm]{geometry} }{}
My full LaTeX configuration file is available here.
\comment
CommandThere are several packages that define a \comment
command that would clash with the one defined by changes
.
If the prependcaption
is included in the options for changes
, then \comment
is automatically renamed to \chcomment
and a warning is shown.
I would prefer to use \comment
for the changes
command, however.
To do this, you can redefine the existing \comment
command.
If comment
is an environment, as is defined by the verbatim
package, then you must redefine both \comment
and \endcomment
, prior to importing changes
, as follows:
% Redefine "comment" environment (from "verbatim" package) % to "commentsection" so that \comments{} can be defined % by the `changes` package. \makeatletter \let\commentsection\comment \let\endcommentsection\endcomment \let\comment\@undefined \let\endcomment\@undefined \makeatother
When writing LaTeX with Visual Studio Code, you can define snippets that are automatically inserted when you type particular text.
To set up snippets, type CTRL+SHIFT+P
, type Preferences: Configure User Snippets
, and select latex.json
.
Add the following code to latex.json
:
{ "Added":{ "prefix": ["\\added"], "body": [ "\\added{$TM_SELECTED_TEXT$1}$0" ] }, "Added Block":{ "prefix": ["\\added%", "\\addedblock"], "body": [ "\\added{%", "\t$TM_SELECTED_TEXT$0", "}% End \\added block", "" // Ensure there is a new line at end ] }, "Deleted":{ "prefix": ["\\deleted"], "body": [ "\\deleted{$TM_SELECTED_TEXT$1}$0" ] }, "Deleted Block":{ "prefix": ["\\deleted%", "\\deletedblock"], "body": [ "\\deleted{%", "\t$TM_SELECTED_TEXT$0", "}% End \\deleted block", "" // Ensure there is a new line at end ] }, "Replaced":{ "prefix": ["\\replaced"], "body": [ "\\replaced{$TM_SELECTED_TEXT$1}{$TM_SELECTED_TEXT}$0" ] }, "Replaced Block":{ "prefix": ["\\replaced%", "\\replacedblock"], "body": [ "\\replaced{% New Text", "\t$TM_SELECTED_TEXT$0", "}{% Old Text", "\t$TM_SELECTED_TEXT", "}% End \\replaced block", "" // Ensure there is a new line at end ] } }
For each command \added
, \deleted
, and \replaced
, there are two versions of snippets an “inline” version and a “block” version.
To use the block version append %
or block
.
For example, to add a \replaced
block:
\replaced%
or \replacedblock
. The selected text will temporarily disappear as you type.\replaced
.I find the \replaced
block snippet particularly useful because the comments % New Text
and % Old Text
remind me of the order of the arguments, which I always forget.
In order to understand control theory, it’s helpful to first introduce the concept of a dynamical system. A dynamical system is a system that changes over time. Some examples of dynamical systems are a pendulum (mechanical), a power transformer (electronic), a stock market (economic), and populations of predators and prey (ecological). A dynamical system is described using a list of numbers that change over time. We call the list of numbers the state of the system. The main questions we ask about a dynamical system is how it behaves over time.
Consider, for example, an ecosystem with a population of a predator species and the population of its prey. The state of the system has two values at each moment in time: the population of the predator and the population of the prey. If either of these values goes to zero, then that species goes extinct.
In some dynamical systems, there are inputs that affect the behavior of the system. An input is a value that can be directly chosen at each moment in time. For a car, the inputs are the throttle (gas pedal), the brake, and the steering wheel. The position and velocity of the car cannot be controlled directly—to move the car to a new location, one must use the throttle and steering wheel to maneuver there. A dynamical system with inputs is a control system and the study of how to pick the inputs achieve various goals is called control theory. In general, our goal is to design the inputs so that the system
When choosing symbols for mathematical objects (variables, sets, etc.), the best choices are
Regarding the choice of symbols, Paul R. Halmos wrote [1]:
Good notation has a kind of alphabetical harmony and avoids dissonance. Example: either $a x+b y$ or $a_1 x_1+a_2 x_2$ is preferable to $a x_1+b x_1.$ Or: if you must use $\Sigma$ for an index set, make sure you don’t run into $\sum_{\sigma \in \Sigma} \sigma.$
This document contains guidelines for picking good symbols and examples to shorten the process.
When choosing a symbol, it is helpful to choose it such that there is a connection between the symbol and its meaning. The most basic approach is to use the Latin character that starts a word related to the symbol’s meaning, such as $g$ or $G$ for gravity. After exhausting the Latin alphabet, the Greek alphabet can be used. The name of each Greek letter generally starts with the sound it makes. For instance, gamma ($\gamma$ and $\Gamma$) makes a “g” sound, so it would be a reasonable choice for a gravity symbol if $g$ and $G$ are already used elsewhere. In the following table, the second column lists possible choices of symbols to represent a object that has a name or description that starts with the sound or letter given in the first column.
First letter/sound  Symbols 

‘a’ as in “ape” or “apple” 
$a, A, \alpha$ (\alpha ), $\aleph$ (\aleph )

‘b’ 
$b, B$, $\beta$ (\beta )

‘d’ (’distance’) 
$d, D$, $\delta$ (\delta ), $\Delta$ (\Delta ). Avoid $d$ for quantities that might appear in derivatives (for a quantity $d$, the notation "$dd/dt$ is confusing).

‘e’ as in “eat” or “egg” 
$e, E$, $\eta$ (\eta )

‘f’, 'ph’ as in “first” 
$\phi$ (\phi ), $\varphi$ (\varphi ), $\Phi$ (\Phi ), $f$, $F$

‘g’ (‘good’) 
$g$, $G$, $\gamma$ (\gamma ), $\Gamma$ (\Gamma )

‘j’ (’James’, ‘gee’)  $j, J$, $g$, $G$ 
‘k’ (’king’, ‘compact’) 
$k, K, c, C$, $\chi$ (\chi . Makes a hard "k" sound in Greek)

‘l’ (’lemma’, “Lie”) 
$l, L$, $\ell$ (\ell ), $\lambda$ (\lambda ), $\Lambda$ (\Lambda ). The symbol $\ell$ (\ell ) is generally preferable to $l$ (l ), as it’s less likely to mistaken for a $1$ (1 ) and vice versa.

‘m’ 
$m$, $M$, $\mu$ (\mu )

‘n’ 
$n$, $N$, $\nu$ (\nu )

‘o’ 
$o$, $O$, $\omega$ (\omega ), $\Omega$ (\Omega )

‘p’ 
$p, P$, $\pi$ (\pi ), $\Pi$ (\Pi )

‘r’ (’radius’) 
$r, R$, $\rho$ (\rho ), $\varrho$ (\varrho )

‘s’ (’see’, ‘psychic’, ‘cease’) 
$s, S$, $\psi$ (\psi ), $\Psi$ (\Psi ), $\sigma$ (\sigma ), $\varsigma$ (\varsigma ), $\Sigma$ (\Sigma ), $c$, $C$, $\xi$ (\xi ), $\Xi$ (\Xi )

‘t’ (’tensor’, ‘time’) 
$t, \tau$ (\tau ), $T$

‘th’ 
$\theta$ (\theta ), $\Theta$ (\Theta ), $t, T$, $\mathrm{Th}, \mathrm{th}$, $\vartheta$ (\vartheta )

‘u’ (’you’, ‘young’)  $u, U$, $y$, $Y$. Lowercase upsilon "$\upsilon$" should be avoided due to the similarity to lowercase vee "$v$". 
‘v’  $v, V$ 
‘w’  $w, W$ 
z  $z, Z, \zeta$ (\zeta )

Suppose we are using the symbol $x$ and want to introduce a second symbol that is strongly related to $x.$ The following modifications can be used to create a new symbol.
Example  Description 

$x^1, x^{(2)}, x^a, x^*, x^\circ$  Superscript. Numbers should be avoided when they could be confused with exponents. 
$x_1, x_a, ...$  Subscripts. Having more than two layers of subscripts should be avoided to preserve readability. 
$x', x'', x'''$  Prime notation. Commonly used for derivatives, so avoid when there may be confusion. Use at most three tick marks. 
$x_{I}, x_{II}, x_{III}, x_{IV}$  Annotate with Roman Numerals. I’ve never seen this notation in a publication, but I use it within my scratch work to keep track of different iterations while I develop my work. If, say, I’m trying to find a set that satisfies some properties, I might notation them $A_I, A_{II}, ...$, until I find one that works. Then, I would simply call the final choice $A$. 
$x \mapsto \hat{x}, \tilde{x}, \overline{x}, \underline{x}$  Add annotations above or below. 
$x \mapsto X$ $F \mapsto f$ 
Change capitalization 
$x \mapsto \mathrm{x}, \mathbf{x}$ $X \mapsto \mathcal{X}, \mathbb{X}, \mathbf{X}, \mathscr{X} , \mathfrak{X}$ 
Change the font. This should be used with caution because the difference between certain fonts will not be obvious to all readers, especially in handwritten text. See note below. 
$x_{\textrm{label}}$, $x^{\textrm{label}}$  Include text labels. This is a heavyhanded approach that is tedious to write, but it does not require remembering another piece of notation so it might be desirable in presentations where the audience cannot go back to review the notation. Avoid for symbols that occur often. 
$a \mapsto b, \text{or } x\mapsto y$  Use letters that are adjacent in the alphabet. 
$x \mapsto p_x, A_x$  Juxtaposition with another symbol. Use $x$ as a label for another symbol 
$V \mapsto \Lambda$  Change the [orientation of the symbol](https://tex.stackexchange.com/questions/18157/rotatingaletter). (This doesn’t work for symmetric symbols, such as $x$). 
$x \mapsto \Delta x, dx, \delta x$  Prefix with another symbol (typically, $\Delta x$ represents a change in $x$; $\delta x$ represents a small but finite change in $x$, and $dx$ is used to represent the change in $x $ in the limit as the change goes to zero.) 
$x \mapsto [x]$, $\{x\}$, $\x\$  Brackets. Use with caution as most brackets have existing meanings. 
$x \mapsto f(x)$  Function notation. 
$1 + x^2$  Sometimes, a new symbol is not actually necessary. For instance, if the new symbol depends on $x,$ you can simply write it as a function of $x$. 
In addition to the default capital Latin characters ($A$, $B$, $C$), etc., LaTeX provides script (\mathscr
) characters $\mathscr{A}$, $\mathscr{B}$, $\mathscr{C}$; calligraphy (\mathcal
) characters ($\mathcal{A}$, $\mathcal{B}$, $\mathcal{C}$); and Fraktur (\mathfrak
) characters $\mathfrak{A}$, $\mathfrak{B}$, $\mathfrak{C}$. Many letters are different enough in each style that readers can be expected to recognize them as distinct symbols. For example, one can safely use $L, \mathcal{L}$, and $\mathfrak{L}$ in the same document without much risk of confusion (although, if you are writing these characters by hand, that is a different story!).
However, many of the Fraktur characters are easily confused with each other so care should be used to avoid mixing, say $\mathfrak{I}$ and $\mathfrak{J}$ in the same document.
There are conventions for notating certain types of mathematical objects. The following rules are not universally accepted, but are merely taken from my personal observations.
Type of Object  Common Notation Classes 

Set 
Capital Latin or Greek ($A, B, C, \Lambda$); Calligraphy ($\mathcal{A, B, C}$); Blackboard bold ($\mathbb{R, N, Z}$)—typically reserved for wellknown sets; Script ($\mathscr{A, B, C}$)—commonly used for sets of sets; Setbuilder notation: $\{a, b, c\}$ 
Function 
Lowercase Latin: $d, f, g, h, u, v, w, x, y, z$ Lowercase Greek: $\alpha, \beta, \gamma$ Capital Latin: $F, G, H$ Capital Greek: $\Gamma, \Theta, \Phi, \Psi, \Omega, \Xi$ Often the choice of symbol for a function matches the convention used for objects in the function’s codomain (range). 
Vector  $x, \mathbf{x}, \vec{x}, \underline{x}$. In texts where students are newly acquainted to vectors, $\mathbf{x}$ or $\vec x$ is commonly used. The notation $\vec x$ has the advantage that it can easily be written by hand, but it makes equations more cluttered, especially when other annotations are added, such as $\dot{\vec{\widetilde{x}}}$. In advanced texts, $x$ is almost always used. 
Scalar  $a, b, c, x, y, z, \alpha, \beta, \gamma$. See notes on real numbers and integers, below. 
Unitary Operation 
Prefix: $x\mapsto x$, $f \mapsto \partial f$ Annotations: $f\mapsto \hat f, f\mapsto \tilde f, x\mapsto x^*$ Function notation: $x\mapsto f(x), f\mapsto \mathcal{L}\{f\}, x\mapsto \sin x$ Capitalization: $f \mapsto F$ Sub/superscripts: $x \mapsto x_{\text{new}}$ 
Binary Operation  infix: $a+b$, $A\cup B$, $p \wedge q;$ function: $f(x, y);$ juxtaposition: $xy, \overset{x}{y}, \underset{x}{y}, x^y;$ brackets: $(x, y), \langle x, y\rangle.$ 
nary Operation 
Prefix: $\Pi_{i=1}^n x_i$ (Abbreviated) infix: $x_1 + x_2 + \cdots + x_n$ Function: $f(x_1, x_2, \dots, x_n)$ Einstein Summation Convention: $v^i\frac{\partial}{\partial x_i}$ 
Matrix 
Capital Latin or Greek: $A, B, C, \Gamma;$ Upright Capital Latin: $\mathrm{A, B, C, M}$; Bold Capital Latin: $\mathbf{A, B, C, M};$ Elementwise notation: $[a_{ij}]$, $[\sin(i\pi)\cos(j\pi)].$ 
Sequence 
Abbreviated: $1, 1/2, 1/3, \dots;$ Sequence notation: $\{s_j\}_{j=1}^\infty;$ Juxtaposition (as a squence of heads/tails is represented in probability): $HHTHT;$ Recursive: $x_{k+1} = r(1x_k),$ Function: $i \mapsto = 1/i.$ 
Typically, lowercase Latin or Greek letters are used to represent real numbers. Less commonly, uppercase letters are used as well.
Several symbols, namely $\pi$ and $e$, have wellestablished meanings, so they should be avoided when there is any risk of ambiguity. The letters $\varepsilon$ and $\delta$ are frequently used to represent small positive numbers and capital letters such as $M$ or $R$ are sometimes used for large values.
For Latin letters, note that $e$ has a wellestablished usage as Euler’s constant. Be judicious in the use of $f, g, h$, which are commonly used for functions.
Avoid $i, j, k, m, n$ because they are commonly used for integers and avoid $l, o$ due to potential confusion with $1$ and $0$—the symbol $\ell$ can be used instead of $l$.
Typically, lowercase and (less frequently) uppercase Latin letters are used to represent integers. The letters $i, j, k, m, n$ are common choices, especially for indices. Choosing one of them offers a hint to the reader that the variable is an integer when they encounter it after its introduction. In some contexts, $i$ or $j$ is reserved for the imaginary unit $\sqrt{1}$. The letters $a, b, c, d, p, q$ are also commonly used for integers but are also used as real numbers. Avoid $l, o$ (due to potential confusion with $1$ and $0$—$\ell$ can be used instead of $l$). A capital letter is useful to convey that an integer will be “large”, e.g., defining a sequence $x_i$ in $\mathbb{R}$ to be unbounded if “for every $M > 0$, there exists $i \in \mathbb{N}$ such that $x_i > M$.”
When it comes to variables and constants, I prefer to use the beginning of the alphabet for constants and the end for variables. In particular, I tend to use $a,$ $b,$ $c,$ $d,$ $p,$ $q,$ $r$ for constants and $t,$ $u,$ $v,$ $w,$ $x,$ $y,$ $z$ are for variables.
In this section we present a case study of picking symbols for the upper and lower bounds on a real number $x$. At first, we can simply take the first two letter of the alphabet.
\[a \leq x \leq b\]Using $a$ and $b$ is fine, but the connection of $a$ and $b$ with $x$ is not implied symbolically. We also have to pick two new symbols if we need to set bounds on another value, say $y$. To show the connection between $x$ and the bounds, we might instead pick
\[x_{lb} \leq x \leq x_{ub}.\]The use of italicized text for $lb$ and $ub$ is bad form, however. A better choice is
\[x_{\mathrm{lb}} \leq x \leq x_{\mathrm{ub}}.\]This choice is pretty good, but writing letter subscripts can get tedious and makes equations somewhat messy. y For this reason, I prefer to simply underline $x$ for the lower bound and overline $x$ for the upper bound.
\[\underline{x} \leq x \leq \overline{x}.\]This notation is (1) simple, (2) visually descriptive, and (3) does not require chosing new symbols for every upper and lower bound that is introduced. There are cases where $\overline{x}$ can cause confusing, however. Namely, if $x$ is a complex number, then $\overline{x}$ could be read as the complex conjugate.
[1] Norman E. Steenrod, Paul R. Halmos, Menahem M. Schiffer, and Jean A. Dieudonné, How to Write Mathematics. 1973. [2] N. J. Higham, Handbook of writing for the mathematical sciences, 2nd ed. Philadelphia: Society for Industrial and Applied Mathematics, 1998.
]]>