Least Squares - Using a Higher Degree Polynomial
Thomas J. Kennedy
Suppose that we want to approximate $f(x) = x$ for the domain $0 \leq x \leq \alpha$ using a polynomial of degree 2… using a continuous approximation.
1 Setting Up
Let us start with what we know:
- $f(x) = x$
- $x \in [0,\alpha]$
- $\hat{\varphi}$ is a quadrtaic expression
Since $\hat{\varphi}$ is quadratic… the basis functions are
- $\pi_0 = 1$
- $\pi_1 = x$
- $\pi_2 = x^2$
which leads to…
$$ \begin{align} \hat{\varphi} &= c_0 \pi_1 + c_1 \pi_1 + c_2 \pi_2 \\ &= c_0 + c_1 x + c_2 x^2 \\ \end{align} $$
Since we have three basis functions… $A\vec{c} = \vec{b}$ takes the form…
$$ \left[\begin{array}{ll} \int\limits_{x=0}^{\alpha} \pi_{0} \pi_{0} dx & \int\limits_{x=0}^{\alpha} \pi_{0} \pi_{1}dx & \int\limits_{x=0}^{\alpha} \pi_{0} \pi_{2}dx \\ \int\limits_{x=0}^{\alpha} \pi_{1} \pi_{0} dx & \int\limits_{x=0}^{\alpha} \pi_{1} \pi_{1}dx & \int\limits_{x=0}^{\alpha} \pi_{1} \pi_{2}dx \\ \int\limits_{x=0}^{\alpha} \pi_{2} \pi_{0} dx & \int\limits_{x=0}^{\alpha} \pi_{2} \pi_{1}dx & \int\limits_{x=0}^{\alpha} \pi_{2} \pi_{2}dx \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \int\limits_{x=0}^{\alpha} \pi_0 f dx\\ \int\limits_{x=0}^{\alpha} \pi_1 f dx\\ \int\limits_{x=0}^{\alpha} \pi_2 f dx\\ \end{array}\right] $$
Note how $A$ is a 3-by-3 matrix. We have two constants to compute (i.e., $c_0, c_1, c_2$).
2 Expectations…
Based on the nature of the problem, I suspect that we will end up with
- $c_0 = 0$
- $c_1 = 1$
- $c_2 = 0$
3 Plugging Everything In
We know that
- $\pi_0 = 1$
- $\pi_1 = x$
- $\pi_2 = x^2$
- $f(x) = x$
Let us plug everything in.
$$ \left[\begin{array}{ll} \int\limits_{x=0}^{\alpha} dx & \int\limits_{x=0}^{\alpha} x dx & \int\limits_{x=0}^{\alpha} x^2 dx \\ \int\limits_{x=0}^{\alpha} x dx & \int\limits_{x=0}^{\alpha} x^2 dx & \int\limits_{x=0}^{\alpha} x^3 dx \\ \int\limits_{x=0}^{\alpha} x^2 dx & \int\limits_{x=0}^{\alpha} x^3 dx & \int\limits_{x=0}^{\alpha} x^4 dx \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \int\limits_{x=0}^{\alpha} x dx\\ \int\limits_{x=0}^{\alpha} x^2 f dx\\ \int\limits_{x=0}^{\alpha} x^3 f dx\\ \end{array}\right] $$
3.1 Power Rule
Since all integrals take the form
$$ \int\limits_{x=0}^{\alpha} x^n dx \\ $$
We can specialize the definition of the power rule to this problem…
$$ \begin{align} \int\limits_{x=0}^{\alpha} x^n dx &= \frac{1}{n+1} x^{n+1} \Big|_{x=0}^{\alpha} \\ &= \frac{1}{n+1} \left(\alpha^{n+1} - (0)^{n+1}\right) \\ &= \frac{\alpha^{n+1}}{n+1} \\ \end{align} $$
3.2 Using the Extension
Now… we evaluate everything…
$$ \left[\begin{array}{ll} \alpha & \frac{1}{2} \alpha^2 & \frac{1}{3} \alpha^3 \\ \frac{1}{2}\alpha^2 & \frac{1}{3} \alpha^3 & \frac{1}{4} \alpha^4 \\ \frac{1}{3}\alpha^3 & \frac{1}{4} \alpha^4 & \frac{1}{5} \alpha^5 \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \frac{1}{2}\alpha^2 \\ \frac{1}{3}\alpha^3 \\ \frac{1}{4}\alpha^4 \\ \end{array}\right] $$
4 Linear Algebra (Solving)
The next step is to apply some elementary row operations. Let us start with:
- $\alpha^{-1}\vec{r_0}$
- $2\alpha^{-2}\vec{r_1}$
- $3\alpha^{-3}\vec{r_2}$
$$ \left[\begin{array}{ll} 1 & \frac{1}{2} \alpha & \frac{1}{3} \alpha^2 \\ 1 & \frac{2}{3} \alpha & \frac{1}{2} \alpha^2 \\ 1 & \frac{3}{4} \alpha & \frac{3}{5} \alpha^2 \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \frac{1}{2}\alpha \\ \frac{2}{3}\alpha \\ \frac{3}{4}\alpha \\ \end{array}\right] $$
Let us rewrite the fractions.
$$ \left[\begin{array}{ll} 1 & \frac{6}{12} \alpha & \frac{10}{30} \alpha^2 \\ 1 & \frac{8}{12} \alpha & \frac{15}{30} \alpha^2 \\ 1 & \frac{9}{12} \alpha & \frac{18}{30} \alpha^2 \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \frac{6}{12}\alpha \\ \frac{8}{12}\alpha \\ \frac{9}{12}\alpha \\ \end{array}\right] $$
Now… it is time for
- $\vec{r_1} - \vec{r_0}$
- $\vec{r_2} - \vec{r_0}$
$$ \left[\begin{array}{ll} 1 & \frac{6}{12} \alpha & \frac{10}{30} \alpha^2 \\ 0 & \frac{2}{12} \alpha & \frac{ 5}{30} \alpha^2 \\ 0 & \frac{3}{12} \alpha & \frac{ 8}{30} \alpha^2 \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \frac{6}{12}\alpha \\ \frac{2}{12}\alpha \\ \frac{3}{12}\alpha \\ \end{array}\right] $$
Now… let us scale $\vec{r_1}$ and $\vec{r_2}$
$$ \left[\begin{array}{ll} 1 & \frac{6}{12} \alpha & \frac{10}{30} \alpha^2 \\ 0 & 1 & \alpha \\ 0 & 1 & \frac{32}{30} \alpha \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \frac{6}{12}\alpha \\ 1 \\ 1 \\ \end{array}\right] $$
Let us simplify a few fractions.
$$ \left[\begin{array}{ll} 1 & \frac{1}{2} \alpha & \frac{1}{3} \alpha^2 \\ 0 & 1 & \alpha \\ 0 & 1 & \frac{16}{15} \alpha \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \frac{1}{2}\alpha \\ 1 \\ 1 \\ \end{array}\right] $$
Let us perform $\vec{r_2} - \vec{r_1}$
$$ \left[\begin{array}{ll} 1 & \frac{1}{2} \alpha & \frac{1}{3} \alpha^2 \\ 0 & 1 & \alpha \\ 0 & 0 & \alpha \\ \end{array}\right] \left[\begin{array}{c} c_0 \\ c_1 \\ c_2 \\ \end{array}\right] = \left[\begin{array}{l} \frac{1}{2}\alpha \\ 1 \\ 0 \\ \end{array}\right] $$
We now have an upper-triangular matrix.
5 Switching to Algebra
Let us switch to basic Algebra.
$$ \begin{array}{rrrrrl} (1)c_0 &+& (\frac{1}{2}\alpha)c_1 &+& (\frac{1}{3}\alpha^2)c_2 &=& \frac{1}{2}\alpha \\ (0)c_0 &+& (1)c_1 &+& (\alpha)c_2 &=& 1 \\ (0)c_0 &+& (0)c_1 &+& (\alpha)c_2 &=& 0 \\ \end{array} $$
The third equation simplifies to
$$ c_2 = 0 $$
Now… we can simplify the first two equations.
$$ \begin{array}{rrrrl} (1)c_0 &+& (\frac{1}{2}\alpha)c_1 &=& \frac{1}{2}\alpha \\ (0)c_0 &+& (1)c_1 &=& 1 \\ \end{array} $$
The second equation gives us
$$ c_1 = 1 $$
We are now left with the first equation…
$$ \begin{eqnarray} (1)c_0 + (\frac{1}{2}\alpha)c_1 &=& \frac{1}{2}\alpha \\ c_0 + (\frac{1}{2}\alpha)(1) &=& \frac{1}{2}\alpha \\ c_0 + (\frac{1}{2}\alpha) &=& \frac{1}{2}\alpha \\ c_0 &=& -(\frac{1}{2}\alpha) + \frac{1}{2}\alpha \\ c_0 &=& 0 \\ \end{eqnarray} $$
6 The Result
Now that we have $c_0$, $c_1$, and $c_2$ the approximation function can be written as…
$$ \hat{\varphi} = (0) 1 + (1) x + (0) x^2 $$
or in its simplified form as…
$$ \hat{\varphi} = x $$
7 What is the Error
Now… for the important question: How “good” is our approximation?
$$ f(x) = \hat{\varphi}(x,\alpha) \Rightarrow E^2(x, \alpha) = 0 $$
Since we ended up with our orignal function… the error is zero!