Reconcile the XTX|XTY and Ab Methods
Thomas J. Kennedy
Let us try to reconcile the $[X^TX|X^TY]$ and $[A|\vec{b}]$ methods for $k$ points and $\forall_\hat{\varphi}$ where $\hat{\varphi} = c_0 \pi_0 + c_1 \pi_1$ and $\pi_0$, $\pi_1$ are arbitrary monomial basis functions.
We know quite a bit about lines. Let
$$\pi_0 = 1$$ $$\pi_1 = x$$
and let $f$ be an arbitrary function.
Let us start by constructing $X$, $X^T$ and $Y$. By definition… $X$ takes the form
$$ \left[\begin{array}{rr} \pi_{0}(x_0) & \pi_{1}(x_0) \\ \pi_{0}(x_1) & \pi_{1}(x_1) \\ \pi_{0}(x_2) & \pi_{1}(x_2) \\ \vdots & \vdots \\ \pi_{0}(x_{k-2}) & \pi_{1}(x_{k-2}) \\ \pi_{0}(x_{k-1}) & \pi_{1}(x_{k-1}) \\ \end{array}\right] $$
That leads to
$$ X^T = \left[\begin{array}{rrrrrr} \pi_{0}(x_0) & \pi_{0}(x_1) & \pi_{0}(x_2) & \dots & \pi_{0}(x_{k-2}) \pi_{0}(x_{k-1}) \\ \pi_{1}(x_0) & \pi_{1}(x_1) & \pi_{1}(x_2) & \dots & \pi_{1}(x_{k-2}) \pi_{1}(x_{k-1}) \\ \end{array}\right] $$
By definition, $Y$ takes the form
$$ \left[\begin{array}{r} f(x_0) \\ f(x_1) \\ f(x_2) \\ \vdots \\ f(x_{k-2}) \\ f(x_{k-1}) \\ \end{array}\right] $$
Using these definitions we (after a couple matrix multiplications) end up with
$$ X^TX = \left[\begin{array}{rr} \sum\limits_{i=0}^{k-1}\pi_{0}(x_i)\pi_{0}(x_i) & \sum\limits_{i=0}^{k-1}\pi_{0}(x_i)\pi_{1}(x_i) \\ \sum\limits_{i=0}^{k-1}\pi_{1}(x_i)\pi_{0}(x_i) & \sum\limits_{i=0}^{k-1}\pi_{1}(x_i)\pi_{1}(x_i) \\ \end{array}\right] = A $$
and
$$ X^TY = \left[\begin{array}{rr} \sum\limits_{i=0}^{k-1}\pi_{0}(x_i)f(x_i) \\ \sum\limits_{i=0}^{k-1}\pi_{1}(x_i)f(x_i) \\ \end{array}\right] = \vec{b} $$
Notice how I used $\pi_0$ and $\pi_1$ the entire time? I used the line as a starting point to frame the proof in my mind. Once I knew which direction I was going to take, I used $\pi_0$ and $\pi_1$ as arbitrary basis functions.
We now have (and know) general forms of $X^TX$ and $X^TY$… These are the definitions of $A$ and $\vec{b}$, respectively, for two arbitrary monomial basis functions.