1 Least Squares Approximation

We spent quite a bit of time discussing Least Squares Approximation. In total we discussed the following methods…

1. $$[X^TX|X^TY]$$

2. $$[A|\vec{b}]$$

3. $$\frac{\partial}{\partial c_k} \int\limits_{\mathbb{R}} \left(f - \hat{\varphi}\right)^2$$

4. Presolved $$[A|\vec{b}]$$ for a line

While we discussed four methods for deriving $\hat{\varphi}$… all four methods were really different notations for the same techniques. Note that our focus here is on Least Squares Approximation as applied to discrete data.

Least Squares Approximation is a single method.

2 Why Interpolation?

Least Squares Approximation captures the behavior of a set of data. Interpolation also captures the behavior of a set of data. However,

• Least Squares Approximation focuses on global behavior (the entire collection of data)

• Interpolation focuses on local behavior (around and at individual points)

Interpolation refers to a collection of techniques… with a common invariant.

The error at every input point must be zero. Given a polynomial $p(x)$, the polynomial must be equal to $f(x)$ at every input point ($x_0, x_1, \ldots, x_n$). This invariant is often written as

$$ \forall_{x_k} p(x_k) = f(x_k) $$

This notation is read as for all $x_k$ $p(x_k)$ must be equal to $f(x_k)$.

3 Let Us Try that Again… In Plain English

Least Squares Approximation can be thought of as…

We have a memory of a pieces of a picture and want to capture the idea as inspired by the pieces

Interpolation can be though of as…

We have pieces of a picture and want to fill in the blanks… while preserving the orignal pieces and trying to figure out should be around each piece.