Reconcile the XTX|XTY and Ab Methods

Thomas J. Kennedy

Let us try to reconcile the $[X^TX|X^TY]$ and $[A|\vec{b}]$ methods for $k$ points and $\forall_\hat{\varphi}$ where $\hat{\varphi} = c_0 \pi_0 + c_1 \pi_1$ and $\pi_0$, $\pi_1$ are arbitrary monomial basis functions.

 

We know quite a bit about lines. Let

$$\pi_0 = 1$$ $$\pi_1 = x$$

and let $f$ be an arbitrary function.


Let us start by constructing $X$, $X^T$ and $Y$. By definition… $X$ takes the form

$$ \left[\begin{array}{rr} \pi_{0}(x_0) & \pi_{1}(x_0) \\ \pi_{0}(x_1) & \pi_{1}(x_1) \\ \pi_{0}(x_2) & \pi_{1}(x_2) \\ \vdots & \vdots \\ \pi_{0}(x_{k-2}) & \pi_{1}(x_{k-2}) \\ \pi_{0}(x_{k-1}) & \pi_{1}(x_{k-1}) \\ \end{array}\right] $$

That leads to

$$ X^T = \left[\begin{array}{rrrrrr} \pi_{0}(x_0) & \pi_{0}(x_1) & \pi_{0}(x_2) & \dots & \pi_{0}(x_{k-2}) \pi_{0}(x_{k-1}) \\ \pi_{1}(x_0) & \pi_{1}(x_1) & \pi_{1}(x_2) & \dots & \pi_{1}(x_{k-2}) \pi_{1}(x_{k-1}) \\ \end{array}\right] $$

By definition, $Y$ takes the form

$$ \left[\begin{array}{r} f(x_0) \\ f(x_1) \\ f(x_2) \\ \vdots \\ f(x_{k-2}) \\ f(x_{k-1}) \\ \end{array}\right] $$

Using these definitions we (after a couple matrix multiplications) end up with

$$ X^TX = \left[\begin{array}{rr} \sum\limits_{i=0}^{k-1}\pi_{0}(x_i)\pi_{0}(x_i) & \sum\limits_{i=0}^{k-1}\pi_{0}(x_i)\pi_{1}(x_i) \\ \sum\limits_{i=0}^{k-1}\pi_{1}(x_i)\pi_{0}(x_i) & \sum\limits_{i=0}^{k-1}\pi_{1}(x_i)\pi_{1}(x_i) \\ \end{array}\right] = A $$

and

$$ X^TY = \left[\begin{array}{rr} \sum\limits_{i=0}^{k-1}\pi_{0}(x_i)f(x_i) \\ \sum\limits_{i=0}^{k-1}\pi_{1}(x_i)f(x_i) \\ \end{array}\right] = \vec{b} $$

Notice how I used $\pi_0$ and $\pi_1$ the entire time? I used the line as a starting point to frame the proof in my mind. Once I knew which direction I was going to take, I used $\pi_0$ and $\pi_1$ as arbitrary basis functions.


We now have (and know) general forms of $X^TX$ and $X^TY$… These are the definitions of $A$ and $\vec{b}$, respectively, for two arbitrary monomial basis functions.