Least Squares - Using a Higher Degree Monomial

Thomas J. Kennedy

Contents:

Suppose that we want to approximate $f(x) = x$ for the domain $0 \leq x \leq \alpha$ using a monomial of degree 2… using a continuous approximation.

1 Setting Up

Let us start with what we know:

Since $\hat{\varphi}$ is a monomial… the single basis function is

which leads to…

$$ \begin{align} \hat{\varphi} &= c_0 \pi_2 \\ &= c_0 x^2 \\ \end{align} $$

Since we have one basis function… $A\vec{c} = \vec{b}$ takes the form…

$$ \left[\begin{array}{ll} \int\limits_{x=0}^{\alpha} \pi_{0} \pi_{0} dx \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ \end{array}\right] = \left[\begin{array}{l} \int\limits_{x=0}^{\alpha} \pi_0 f dx\\ \end{array}\right] $$

Note how $A$ is a 1-by-1 matrix. We have one constant to compute.

2 Plugging Everything In

We know that

Let us plug everything in.

$$ \left[\begin{array}{ll} \int\limits_{x=0}^{\alpha} x^4 dx \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ \end{array}\right] = \left[\begin{array}{l} \int\limits_{x=0}^{\alpha} x^3 dx\\ \end{array}\right] $$

3 Switching to Algebra

Let us switch to basic Algebra.

$$ \begin{eqnarray} c_0 \int\limits_{x=0}^{\alpha} x^4 dx &=& \int\limits_{x=0}^{\alpha} x^3 dx \\ c_0 \frac{\alpha^5}{5} &=& \frac{\alpha^4}{4} \\ c_0 &=& \frac{5\alpha^4}{4\alpha^5} \\ c_0 &=& \frac{5}{4\alpha} \\ \end{eqnarray} $$

4 The Result

Now that we have $c_0$… we can write the approximation function as

$$ \hat{\varphi} = \frac{5}{4\alpha} x^2 $$

5 What is the Error

Now… for the important question: How “bad” is our approximation?

The error can be evaluated using

$$ \begin{eqnarray} E^2(\alpha,x) &=& ||f(x) - \hat{\varphi}(\alpha,x)|| \\ &=& \int\limits_{\mathbb{R}} \left(f - \hat{\varphi} \right)^2 d\lambda(x) \\ &=& \int\limits_{\mathbb{R}} f^2 d\lambda(x) -2\int\limits_{\mathbb{R}} \hat{\varphi}f d\lambda(x) +\int\limits_{\mathbb{R}} \hat{\varphi}^2 d\lambda(x) \\ &=& \int\limits_{\mathbb{R}} \hat{\varphi}^2 d\lambda(x) -2\int\limits_{\mathbb{R}} \hat{\varphi}f d\lambda(x) + \int\limits_{\mathbb{R}} f^2 d\lambda(x) \\ \end{eqnarray} $$

Since we have a continuous approximation… with the domain $0 \leq x \leq \alpha$…

$$ \begin{eqnarray} E^2(\alpha,x) &=& \int\limits_{\mathbb{R}} \hat{\varphi}^2 d\lambda(x) &-2\int\limits_{\mathbb{R}} \hat{\varphi}f d\lambda(x) &+ \int\limits_{\mathbb{R}} f^2 d\lambda(x) \\ &=& \int\limits_{x=0}^{\alpha} \hat{\varphi}^2 dx &-2\int\limits_{x=0}^{\alpha} \hat{\varphi}f dx &+ \int\limits_{x=0}^{\alpha} f^2 dx \\ \end{eqnarray} $$

5.1 Plugging Everything In

Now… we plug in $f$ and $\hat{\varphi}$ (after preparing ourselves for some Calculus, Algebra, and Arithmetic).

$$ \begin{eqnarray} E^2(\alpha,x) &=& \int\limits_{x=0}^{\alpha} \hat{\varphi}^2 dx &-2\int\limits_{x=0}^{\alpha} \hat{\varphi}f dx &+ \int\limits_{x=0}^{\alpha} f^2 dx \\ &=& \int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)^2 dx &-2\int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)x dx &+ \int\limits_{x=0}^{\alpha} x^2 dx \\ &=& \int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)^2 dx &-2\int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)x dx &+ \frac{1}{3}\alpha^3 \\ &=& \int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)^2 dx &-2\int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha}x^3\right) dx &+ \frac{1}{3}\alpha^3 \\ &=& \int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)^2 dx &-\frac{10}{2\alpha}\int\limits_{x=0}^{\alpha} \left(x^3\right) dx &+ \frac{1}{3}\alpha^3 \\ &=& \int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)^2 dx &-\frac{10}{16\alpha}\left(\frac{\alpha^4}{4}\right) &+ \frac{1}{3}\alpha^3 \\ &=& \int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)^2 dx &- \frac{10\alpha^3}{16} &+ \frac{1}{3}\alpha^3 \\ &=& \int\limits_{x=0}^{\alpha} \left(\frac{5}{4\alpha} x^2\right)^2 dx &-\frac{10\alpha^3}{16} &+ \frac{1}{3}\alpha^3 \\ &=& \int\limits_{x=0}^{\alpha} \left(\frac{25}{16\alpha^2} x^4\right) dx &-\frac{10\alpha^3}{16} &+ \frac{1}{3}\alpha^3 \\ &=& \frac{25}{16\alpha^2} \left(\frac{\alpha^5}{5}\right) dx &-\frac{10\alpha^3}{16} &+ \frac{1}{3}\alpha^3 \\ &=& \frac{5}{16\alpha^2} \left(\alpha^5\right) dx &-\frac{10\alpha^3}{16} &+ \frac{1}{3}\alpha^3 \\ &=& \frac{5}{16}\alpha^3 dx &-\frac{10\alpha^3}{16} &+ \frac{1}{3}\alpha^3 \\ &=& \frac{15}{48}\alpha^3 dx &-\frac{30\alpha^3}{48} &+ \frac{16}{48}\alpha^3 \\ &=& \frac{1}{48}\left(15 - 30 + 16\right)\alpha^3 \\ &=& \frac{1}{48}(1)\alpha^3 \\ &=& \frac{1}{48}\alpha^3 \\ \end{eqnarray} $$