1 Vector Norms

To discuss the conditioning of vector functions… we first need to discuss vector norms and their properties.

We are interested in $L_1$, $L_2$, and $L_{\infty}$ norms:

$$ L_{1} = \sum\limits_{0}^{n - 1} \left| x_i \right| $$

$$ L_{2} = \sqrt{\sum\limits_{0}^{n - 1} \left| x_i \right|^{2}} $$

where $n$ is the cardinality of the vector (i.e., the number of dimensions). You probably noticed that I skipped the $L_{\infty}$ norm. It has a succinent (and slightly less approachable) definition.

$$ L_{\infty} = \max_{0 \leq i \leq n-1}\left|x_i\right| $$

which I prefer to write as

$$ L_{\infty} = \max\{|x_0|, |x_1|, x_2|, \ldots |x_{n-1}|\} $$

2 Properties of Vector Norms

1. $||\vec{x}|| \geq 0$ if $\vec{x} \neq 0$
2. $||c\vec{x}|| = c||\vec{x}||$ for $c \in \mathbb{R}$
3. $||\vec{x} + \vec{y}|| \leq ||\vec{x}|| + ||\vec{y}||$
4. $||\vec{x} - \vec{y}|| \geq ||\vec{x}|| - ||\vec{y}||$

3 Condition Number of a Vector Function

The condition number of a vector function is defined using the $L_{\infty}$-norm and the Jacobian Matrix.

$$ (cond f)(x) = \frac{||\vec{x}||_{\infty} ||f^{\prime}(\vec{x})||_{\infty}} {||f(\vec{x})||_{\infty}} \ $$

where $f^{\prime}$ is the Jacobian Matrix.

$$ f^{\prime}(\vec{x}) = \left[\begin{array}{cccc} \frac{\partial f_1}{\partial x_{1}} & \frac{\partial f_1}{\partial x_{2}} & \ldots & \frac{\partial f_1}{\partial x_{n-1}} \\ \frac{\partial f_2}{\partial x_{1}} & \frac{\partial f_2}{\partial x_{2}} & \ldots & \frac{\partial f_2}{\partial x_{n-1}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial f_{n-1}}{\partial x_{1}} & \frac{\partial f_{n-1}}{\partial x_{2}} & \ldots & \frac{\partial f_{n-1}}{\partial x_{n-1}} \\ \end{array}\right] $$

3.1 Two Well-Known Vector Functions

There are two well-known vector functions… In fact you have used the quite a few times in Math and Physics problems.

The first equation is used to convert from Cartesian Coordinates to Polar Coordinates.

$$ f(x, y) = \left[\begin{array}{c} \sqrt{x^2 + y^2} \\ \arctan\left(\frac{y}{x}\right) \end{array}\right] $$

… although we are usually only interested in distance from the origin (i.e., the first piece).

The second equation is used to convert from Polar Coordinates to Cartesian Coordinates.

$$ f(r, \theta) = \left[\begin{array}{c} |r|\cos\theta \\ |r|\sin\theta \\ \end{array}\right] $$

4 A Quick Example

For our example… we will use

$$ \vec{x} = \left[\begin{array}{c} x \\ y \end{array}\right] $$

and

$$ f(\vec{x}) = \left[\begin{array}{c} \sqrt{x^2 + y^2} \end{array}\right] $$

Let us start with $f^{\prime}$:

$$ f^{\prime}(\vec{x}) = \left[\begin{array}{cc} \frac{\partial}{\partial x_{1}} \sqrt{x^2 + y^2} & \frac{\partial f_1}{\partial x_{2}} \sqrt{x^2 + y^2} \end{array}\right] \\ = \left[\begin{array}{cc} \frac{2x}{2\sqrt{x^2 + y^2}} & \frac{2y}{2\sqrt{x^2 + y^2}} \end{array}\right] \\ = \left[\begin{array}{cc} \frac{x}{\sqrt{x^2 + y^2}} & \frac{y}{\sqrt{x^2 + y^2}} \end{array}\right] \\ $$

Now… we can plug everything into the definition…

$$ \begin{eqnarray}{rl} \text{(}cond f)(x) &=& \frac{||\vec{x}||_{\infty} ||f^{\prime}(\vec{x})||_{\infty}} {||f(\vec{x})||_{\infty}} \\ &=& \frac{ ||\vec{x}||_{\infty} \left|\left| \left[\begin{array}{cc} \frac{x}{\sqrt{x^2 + y^2}} & \frac{y}{\sqrt{x^2 + y^2}} \end{array}\right] \right|\right|_{\infty} } {||\sqrt{x^2 + y^2}||_{\infty}} \\ &=& \frac{\max\left(|x|,|y|\right) \max\left( \left|\frac{x}{\sqrt{x^2 + y^2}} \right|, \left|\frac{y}{\sqrt{x^2 + y^2}} \right| \right) } {\max\left(\left|\sqrt{x^2 + y^2}\right|\right)} \end{eqnarray} $$

Maybe… we should have evaluated the $L_{\infty}$ Norms first. But… we can simplify this expression further. We know that:

1. The square roots are all non-negative (since $x^2 + y^2 \geq 0$)
2. The square roots can be factored out of the numerator

$$ \begin{eqnarray} \text{(}cond f)(x) &=& \frac{\max\left(|x|,|y|\right) \max\left( \left|\frac{x}{\sqrt{x^2 + y^2}} \right|, \left|\frac{y}{\sqrt{x^2 + y^2}} \right| \right) } {\max\left(\left|\sqrt{x^2 + y^2}\right|\right)} \\ &=& \frac{\frac{1}{\sqrt{x^2 + y^2}} \max\left(|x|,|y|\right) \max\left( \left|\frac{x}{1} \right|, \left|\frac{y}{1} \right| \right) } {\max\left(\left|\sqrt{x^2 + y^2}\right|\right)} \\ &=& \frac{\frac{1}{\sqrt{x^2 + y^2}} \max\left(|x|,|y|\right) \max\left(\left|x\right|,\left|y\right|\right) } {\max\left(\left|\sqrt{x^2 + y^2}\right|\right)} \\ &=& \frac{\frac{1}{\sqrt{x^2 + y^2}} \max\left(|x|,|y|\right) \max\left(\left|x\right|,\left|y\right|\right) } {\sqrt{x^2 + y^2}} \\ &=& \frac{\max\left(|x|,|y|\right) \max\left(\left|x\right|,\left|y\right|\right) } {x^2 + y^2} \\ &=& \frac{\left(\max\left(|x|,|y|\right)\right)^2} {x^2 + y^2} \\ \end{eqnarray} $$

And… that is our simplified condition number.

5 A Very Specific Case

Let us use the general solution for a not-quite-randomly selected point: $(4,5)$.

$$ \begin{eqnarray} \text{(}cond f)(x) &=& \frac{\left(\max\left(|x|,|y|\right)\right)^2} {x^2 + y^2} \\ &=& \frac{\left(\max\left(|4|,|5|\right)\right)^2} {4^2 + 5^2} \\ &=& \frac{\left(\max\left(4,5\right)\right)^2} {16 + 25} \\ &=& \frac{5^2} {16 + 25} \\ &=& \frac{25} {41} \\ &\leq& 1 \end{eqnarray} $$

6 Relationship to the “Scaler” Condition Number

We now have two condition number definitions… one for vector functions ($f: \mathbb{R}^m \to \mathbb{R}^n$)…

$$ (cond f)(x) = \frac{||\vec{x}||_{\infty} ||f^{\prime}(\vec{x})||_{\infty}} {||f(\vec{x})||_{\infty}} \ $$

and one for functions on scaler values ($f: \mathbb{R} \to \mathbb{R}$)…

$$ (cond f)(x) = \left|\frac{x f^{\prime}(x)} {f(\vec{x})}\right| \ $$

We can actual show that if we have:

1. a vector with single element
2. a vector function with a single sub-function

that both definitions are equivalent.

6.1 A Quick Change of Notation

Let us rewrite the Condition Number defintion for vector functions.

$$ (cond v)(r) = \frac{||\vec{r}||_{\infty} ||v^{\prime}(\vec{r})||_{\infty}} {||v(\vec{r})||_{\infty}} \ $$

Why change $f$ to $v$ and $x$ to $r$? Subscripts… we can forgo subscripts. Let us define $v$ and $\vec{r}$ as

$$ v = [f(x)] $$

and

$$ \vec{r} = [x] $$

We now have two vectors… one that contains a single real number (i.e., $x$) and one that contains a function ($f$) on that element.

6.2 Using Norm Definitions

If we apply the definition of the $L_{\infty}$-Norm to each term…

$$ \begin{eqnarray} ||\vec{r}||_{\infty} &=& ||[x]||_{\infty} \\ &=& \max(|x|) \\ \end{eqnarray} $$

$$ \begin{eqnarray} ||v||_{\infty} &=& ||[f(x)]||_{\infty} \\ &=& \max(|f(x)|) \\ \end{eqnarray} $$

$$ \begin{eqnarray} ||v^\prime||_{\infty} &=& ||[f(x)]||_{\infty} \\ &=& \max(\left|\frac{\partial}{\partial x}f(x)\right|) \\ \end{eqnarray} $$

Note that since

1. $f(x)$ is a function on a scaler value
2. $\vec{r}$ contains a single element

we can simplify the last norm a bit more.

$$ \begin{eqnarray} ||v^\prime||_{\infty} &=& ||[f(x)]||_{\infty} \\ &=& \max(\left|\frac{\partial}{\partial x}f(x)\right|) \\ &=& \max(\left|\frac{d}{d x}f(x)\right|) \\ &=& \max(\left|f^{\prime}(x)\right|) \\ \end{eqnarray} $$

We can use each of the definitions in the Condition Number formula.

$$ \begin{eqnarray} (cond\phantom{1}v)(\vec{r}) &=& \frac{||\vec{r}||_{\infty} ||v^{\prime}(\vec{r})||_{\infty}} {||v(\vec{r})||_{\infty}} \\ &=& \frac{\max(|x|) \max(\left|f^{\prime}(x)\right|)} {\max(\left|f(x)\right|)} \\ \end{eqnarray} $$

The max of a single value is that value…

$$ \begin{eqnarray} (cond\phantom{1}v)(\vec{r}) &=& \frac{\max(|x|) \max(\left|f^{\prime}(x)\right|)} {\max(\left|f(x)\right|)} \\ &=& \frac{|x| * \left|f^{\prime}(x)\right|} {\left|f(x)\right|} \\ \end{eqnarray} $$

And… we can use the properties of the absolute value to get…

$$ \begin{eqnarray} (cond\phantom{1}v)(\vec{r}) &=& \frac{|x| * \left|f^{\prime}(x)\right|} {\left|f(x)\right|} \\ &=& \frac{\left|x f^{\prime}(x)\right|} {\left|f(x)\right|} \\ &=& \left| \frac{x f^{\prime}(x)} {f(x)} \right|\\ \end{eqnarray} $$

And… we now have the familiar form of the Condition Number.