Projections and Least Squares
The projection of a vector $\vect{x}$ onto a subspace $W$ is the point of $W$ closest to $\vect{x}$. A least squares solution of $A\vect{x} = \vect{b}$ makes $A\vect{x}$ as close as possible to $\vect{b}$, so $A\vect{x}$ is the projection of $\vect{b}$ onto the column space of $A$. Fitting a curve to data is a least squares problem with one equation for each data point. Choose a problem below, enter your own numbers or load an example, and see the picture and the calculations.
With more than three rows, b is in ℝ⁴ or higher, so there's no picture. The calculations below work the same way.
Top, front, and side views
Calculations
The kind of curve to fit to the data:
- Constant
- A horizontal line. The best constant is the mean of the y-values.
- Line
- A straight line, as in linear regression.
- Quadratic
- A parabola. It goes through any three points with different x-values.
- Cubic
- A polynomial of degree 3. It goes through any four points with different x-values.
Each model is linear in its coefficients c₀, c₁, …, so finding them is a least squares problem.