The Condition Number
Thomas J. Kennedy
1 Overview
We want to relate the error in representing $x$ to the output of a function $f$. Phrased more formally… we want to
-
take $x$ and $x^{*}$
-
compare $f(x^{*})$ to the correct $f(x)$
To do this we will need the relative error for $x$ and the relative error for $f(x)$.
2 Starting With Relative Error
Let us start by writing the relative error for input…
$$ \left| \frac{x - x^{*}}{x} \right| $$
and output…
$$ \left| \frac{f(x) - f(x^{*})}{f(x)} \right| $$
We want to examine the ratio of relative error for output to relative error for input. That leads us to…
$$ \begin{eqnarray} \left| \frac{f(x) - f(x^{*})}{f(x)} \right| \div \left| \frac{x - x^{*}}{x} \right| \\ \left| \frac{f(x) - f(x^{*})}{f(x)} \right| \times \left| \frac{x}{x - x^{*}} \right| \\ \end{eqnarray} $$
3 Some More Notation
Let us rewrite $x - x^{*}$. We know that $x^{*}$ is $x$ plus/minus some difference (absolute error). Let us denote $|x - x^{*}| = |x^{*} - x|$ as $\Delta x$.
$$ \begin{align} \left| \frac{f(x) - f(x^{*})}{f(x)} \right| \times \left| \frac{x}{x - x^{*}} \right| \\ \left| \frac{f(x) - f(x^{*})}{f(x)} \right| \times \left| \frac{x}{\Delta x} \right| \\ \end{align} $$
4 Absolute Value Properties
Now… let us recall the following absolute value properties:
-
$\left|\frac{a}{b}\right| = \frac{|a|}{|b|}$
-
$|a||b| = |ab|$
-
$||a|| = |a|$
-
$|a| = a \text{ if } a \ge 0$
These properties allow some simplification.
$$ \begin{align} \left| \frac{f(x) - f(x^{*})}{f(x)} \right| \times \left| \frac{x}{\Delta x} \right| \\ \left| \frac{f(x) - f(x^{*})}{\Delta x} \right| \times \left| \frac{x}{f(x)} \right| \\ \end{align} $$
5 Some Limit Shenanigans
We are interested in $\Delta x \to 0$ (i.e., $\Delta x$ close to zero).
$$ \begin{align} \lim_{\Delta x \to 0} \left| \frac{f(x) - f(x^{*})}{\Delta x} \right| \times \left| \frac{x}{f(x)} \right| \\ \end{align} $$
We know that $f(x^{*})$ can be written as $f(x + \Delta x)$.
$$ \begin{align} \lim_{\Delta x \to 0} \left| \frac{f(x) - f(x + \Delta x)}{\Delta x} \right| \times \left| \frac{x}{f(x)} \right| \\ \left| \lim_{\Delta x \to 0} \frac{f(x) - f(x + \Delta x)}{\Delta x} \right| \times \left| \frac{x}{f(x)} \right| \\ \end{align} $$
Do you recognize the first fraction? It is the limit definition of the derivative!
$$ \begin{align} \left| \lim_{\Delta x \to 0} \frac{f(x) - f(x + \Delta x)}{\Delta x} \right| \times \left| \frac{x}{f(x)} \right| \\ \left| f^{\prime}(x) \right| \times \left| \frac{x}{f(x)} \right| \\ \left| \frac{xf^{\prime}(x)}{f(x)} \right| \\ \end{align} $$
We now have the general form of the condition number…
$$ (\text{cond }f)(x) = \left| \frac{xf^{\prime}(x)}{f(x)} \right| $$
Keep in mind that we have two invariants…
-
$x \neq 0$
-
$f(x) \neq 0$