  | 1 of 21
 |   |
| | 1 of 21
| |
We’ll illustrate the techniques we’ve learned with some common functions, including some of the array manipulation functions that we converted to iterator style.
This block of code might be found in an early step in a linear algebra application. It sets up an NxN matrix with 1.0 on the diagonals and 0.0 on all of the off-diagonal elements:
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N];
for (int j = 0; j < N; ++j)
{
if (i == j)
matrix[i][j] = 1.0; ➀
else
matrix[i][j] = 0.0; ➁
}
}
The most deeply nested statements in this code are ➀ and ➁. Looking at them, we can see that they involve address calculations for the array indexing, and assignment of a single double. All of those calculations are $O(1)$.
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N];
for (int j = 0; j < N; ++j)
{
if (i == j)
matrix[i][j] = 1.0; // O(1)
else
matrix[i][j] = 0.0; // O(1)
}
}
The if statement condition is $O(1)$,
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N];
for (int j = 0; j < N; ++j)
{
if (i == j) // cond: O(1)
matrix[i][j] = 1.0; // O(1)
else
matrix[i][j] = 0.0; // O(1)
}
}
which means that
\[ \begin{align} t_{\mbox{if}} &= t_{\mbox{condition}} + \max(t_{\mbox{then}}, t_{\mbox{else}}) \\ &= O(1) + \max(O(1), O(1)) \\ &= O(1) \end{align} \]
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N];
for (int j = 0; j < N; ++j)
{
if (i == j) // cond: O(1) total: O(1)
matrix[i][j] = 1.0; // O(1)
else
matrix[i][j] = 0.0; // O(1)
}
}
Collapsing,
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N];
for (int j = 0; j < N; ++j)
{
// O(1)
}
}
The inner for loop has initialization, condition, and increment all O(1). It executes N times.
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N];
// init: O(1)
for (int j = 0; j < N; ++j) // cond: O(1) incr: O(1) #: N
{
// O(1)
}
}
So by our rule for for loops,
\[\begin{align} t_{\mbox{for}} &= t_{\mbox{init}} + \sum_{\mbox{iterations}}(t_{\mbox{condition}} + t_{\mbox{increment}} + t_{\mbox{body}}) + t_{\mbox{condition}}^{\mbox{final}} \\ &= O(1) + \sum_{j=0}^{N-1}(O(1) + O(1) + O(1)) + O(1) \\ &= O(1) + N * (O(1) + O(1) + O(1)) + O(1) \\ &= O(1) + N * (O(1)) + O(1) \\ &= O(1) + O(N) + O(1) \\ &= O(N) \end{align}\]
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N];
// init: O(1)
for (int j = 0; j < N; ++j) // cond: O(1) incr: O(1) #: N total: O(N)
{
// O(1)
}
}
Collapsing:
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N];
// init: O(1)
// O(N)
}
Now look at the first statement in the remaining loop body. We have talked a bit already about what happens when arrays are allocated. First a block of memory is obtained, then the array elements are each initialized by the data type’s default constructor — unless it is a primitive type in which case no such initialization occurs.
In this case we are dealing with an array of double, a primitive type. So no initialization occurs. The only time spent is to get a block of memory from the operating system, which, interestingly enough, does not depend upon the size of the block requested.
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
matrix[i] = new double[N]; // O(1)
// init: O(1)
// O(N)
}
So the statement sequence making up this function body adds up to $O(N)$
double** matrix = new double*[N];
for (int i = 0; i < N; ++i)
{
// O(N)
}
The remaining for loop has initialization, condition, and increment all O(1). It executes N times.
double** matrix = new double*[N];
for (int i = 0; i < N; ++i) // cond: O(1) incr: O(1) #: N
{
// O(N)
}
And so the time for this loop is
\[\begin{align} t_{\mbox{for}} &= t_{\mbox{init}} + \sum_{\mbox{iterations}}(t_{\mbox{condition}} + t_{\mbox{increment}} + t_{\mbox{body}}) + t_{\mbox{condition}}^{\mbox{final}} \\ &= O(1) + \sum_{i=0}^{N-1}(O(1) + O(1) + O(N)) + O(1) \\ &= O(1) + N * (O(1) + O(1) + O(N)) + O(1) \\ &= O(1) + N * (O(N)) + O(1) \\ &= O(1) + O(N^2) + O(1) \\ &= O(N^2) \end{align}\]
double** matrix = new double*[N];
for (int i = 0; i < N; ++i) // cond: O(1) incr: O(1) #: N total: O(N^2)
{
// O(N)
}
Collapsing:
double** matrix = new double*[N];
// O(N^2)
The first statement allocates an array of pointers. Pointer are primitive types and are not initialized, so we have only the $O(1)$ time to get the block of memory:
double** matrix = new double*[N]; // O(1)
// O(N^2)
and the entire statement sequence adds up to $O(N^2)$.
There are a few things that I hope you take notice of here:
Continuing with our NxN matrix, suppose that later in the code we saw:
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j)
{
matrix[i][j] += matrix[i][i];
matrix[j][i] += matrix[i][i];
if (i == j)
{
for (int k = 0; k < N; ++k)
matrix[i][i] *= 2.0;
}
}
}
Here we have three nested loops, each apparently repeating N times, so we might guess that this will be $O(N^3)$. let’s see if this guess holds up.
Again, we can start by marking simple statements that involve only arithmetic and assignment of primitive types.
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j)
{
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j)
{
for (int k = 0; k < N; ++k)
matrix[i][i] *= 2.0; // O(1)
}
}
}
Looking at the innermost loop,
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j)
{
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j)
{
for (int k = 0; k < N; ++k) //cond: O(1) iter: O(1) #: N
matrix[i][i] *= 2.0; // O(1)
}
}
}
and the loop time is
\[\begin{align} t_{\mbox{for}} &= t_{\mbox{init}} + \sum_{\mbox{iterations}}(t_{\mbox{condition}} + t_{\mbox{increment}} + t_{\mbox{body}}) + t_{\mbox{condition}}^{\mbox{final}} \\ &= O(1) + \sum_{k=0}^{N-1}(O(1) + O(1) + O(1)) + O(1) \\ &= O(1) + N * (O(1) + O(1) + O(1)) + O(1) \\ &= O(1) + N * (O(1)) + O(1) \\ &= O(1) + O(N) + O(1) \\ &= O(N) \end{align}\]
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j)
{
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j)
{
for (int k = 0; k < N; ++k) //cond: O(1) iter: O(1) #: N total: O(N)
matrix[i][i] *= 2.0; // O(1)
}
}
}
Collapsing:
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j)
{
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j)
{
// O(N)
}
}
}
The if condition is $O(1)$.
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j)
{
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j) // cond: O(1)
{
// O(N)
}
}
}
And we can then evaluate the if as
\[ \begin{align} t_{\mbox{if}} &= t_{\mbox{condition}} + \max(t_{\mbox{then}}, t_{\mbox{else}}) \\ &= O(1) + \max(O(N), O(1)) \\ &= O(N) \end{align} \]
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j)
{
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j) // cond: O(1) total: O(N)
{
// O(N)
}
}
}
Now, commonly we would, at this point, collapse the if code and just retain the total complexity. But there are three things that hint to me that this might not be the best approach:
if is inside a loop.then and else part complexities are different, andif condition seems highly selective (will take one path much more often than the other).Whenever I see those three characteristics together, I think it’s worth asking whether the more expensive of the two if parts gets executed often enough, in the worst case, to actually dominate the overall loop.
Ignore the outermost loop for the moment.
for (int j = 0; j < N; ++j)
{// O(N)
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j) // cond: O(1) total: O(N)
{
// O(N)
}
}
Question: How many times, total, is the if statement going to be executed by this loop?
It will be executed a total of $N$ times.
Question: How many times, total, will the if statement take the “then” (true) branch?
Only one time.
Question: How many times, total, will the if statement take the “else” (false) branch?
$N-1$ times.
OK, now the rest of that loop is easy enough to handle:
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j) // cond: O(1) iter: O(1) #: N
{// O(N)
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j) // cond: O(1) total: O(N)
{
// O(N)
}
}
}
so that
\[\begin{align} t_{\mbox{for}} &= t_{\mbox{init}} + \sum_{\mbox{iterations}}(t_{\mbox{condition}} + t_{\mbox{increment}} + t_{\mbox{body}}) + t_{\mbox{condition}}^{\mbox{final}} \\ &= O(1) + \sum_{j=0}^{N-1}(O(1) + O(1) + t_{\mbox{body}}) + O(1) \\ &= O(1) + \sum_{j=0}^{N-1}(O(1) + O(1)) + \sum_{j=0}^{N-1} t_{\mbox{body}} + O(1) \\ &= O(1) + N * O(1) + \sum_{j=0}^{N-1} t_{\mbox{body}} \\ &= O(1) + O(N) + \sum_{j=0}^{N-1} t_{\mbox{body}} \\ &= O(N) + \sum_{j=0}^{N-1} t_{\mbox{body}} \\ \end{align}\]
OK, now let’s think about what our earlier questions told us about the sum of the loop body times over all of the iterations. That sum will add together a total of N executions of the body. Exactly one of those will take $O(N)$ time and $N-1$ of those will take $O(1)$ time. So
\[\begin{align} t_{\mbox{for}} &= O(N) + \sum_{j=0}^{N-1} t_{\mbox{body}} \\ &= O(N) + O(N) + (N-1)O(1) \\ &= O(N) + O(N) + O(N) - O(1) \\ &= O(N) \\ \end{align}\]
So what we can see is that the expensive option of the if is done so rarely that it does not dominate the rest of the sum – it kind of disappears into the overall sum.
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < N; ++j) // cond: O(1) iter: O(1) #: N total: O(N)
{ // O(N)
matrix[i][j] += matrix[i][i]; // O(1)
matrix[j][i] += matrix[i][i]; // O(1)
if (i == j) // cond: O(1) total: O(N)
{
// O(N)
}
}
}
Now we can collapse that “j” loop:
for (int i = 0; i < N; ++i)
{
// O(N)
}
The final loop follows the familiar form:
for (int i = 0; i < N; ++i) // cond: O(1) iter: O(1) #: N
{
// O(N)
}
\[\begin{align} t_{\mbox{for}} &= t_{\mbox{init}} + \sum_{\mbox{iterations}}(t_{\mbox{condition}} + t_{\mbox{increment}} + t_{\mbox{body}}) + t_{\mbox{condition}}^{\mbox{final}} \\ &= O(1) + \sum_{i=0}^{N-1}(O(1) + O(1) + O(N)) + O(1) \\ &= O(1) + N * (O(1) + O(1) + O(N)) + O(1) \\ &= O(1) + N * (O(1)) + O(1) \\ &= O(1) + O(N^2) + O(1) \\ &= O(N^2) \end{align}\]
And the entire block of code is $O(N^2)$.
That is not what we predicted by just looking at the depth of nesting of the loops. Surprises demand explanation. In this case we have the explanation: the innermost loop is executed only (1/N)th of the time, so we wind up being a factor of $N$ faster than we had originally predicted.
We start with a function for inserting an element into a sorted array.
/**
*
* Assume the elements of the array are already in order
* Find the position where value could be added to keep
* everything in order, and insert it there.
*
* Return the position where it was inserted
* - Assumes that we have a separate integer (size) indicating how
* many elements are in the array
* - and that the "true" size of the array is at least one larger
* than the current value of that counter
*
* @param array array into which to add an element
* @param size number of data elements in the array. Must be less than
* the number of elements allocated for the array.
* Incremented upon output from this function.
* @param value value to add into the array
* @return the position where the element was added
*/
int addInOrder (int* array, int& size, int value)
{
// Make room for the insertion
int toBeMoved = size - 1; ➀
while (toBeMoved >= 0 && value < array[toBeMoved]) { ➁
array[toBeMoved+1] = array[toBeMoved]; ➂
--toBeMoved;
}
// Insert the new value
array[toBeMoved+1] = value; ➃
++size;
return toBeMoved+1;
}
We start from the high end of the array (➀) and check to see if that’s where we want to insert the data (➁).
value into the “hole” left behind. We repeat this step as necessary.Once we exit from the loop (➃), we insert the value into the hole at our chosen position.
You can run this algorithm until you are comfortable with your understanding of how it works.
From the “Algorithm” menu, select “generate an array”. Create an array with 5-10 elements.
Again, from the “Algorithm” menu, select “add in order” to try the algorithm. Press the forward button to step through the execution.
Try inserting values that should fall near the middle of the array.
Try inserting values that are smaller than any already in the array.
Try inserting values that are larger than any already in the array.
Then we can move on to the analysis.
We start our analysis with the easy stuff – mark all of the non-compound statements.
int addInOrder (int* array, int& size, int value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
while (toBeMoved >= 0 && value < array[toBeMoved]) {
array[toBeMoved+1] = array[toBeMoved]; // O(1)
--toBeMoved; // O(1)
}
// Insert the new value
array[toBeMoved+1] = value; // O(1)
++size; // O(1)
return toBeMoved+1; // O(1)
}
Next, looking at the while loop, we see that its condition can be evaluated in O(1) time, and that the loop repeats at most size times.
int addInOrder (int* array, int& size, int value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
while (toBeMoved >= 0 && value < array[toBeMoved]) { // cond: O(1) #: size
array[toBeMoved+1] = array[toBeMoved]; // O(1)
--toBeMoved; // O(1)
}
// Insert the new value
array[toBeMoved+1] = value; // O(1)
++size; // O(1)
return toBeMoved+1; // O(1)
}
The loop body is $O(1)$, so
\[ \begin{align} t_{\mbox{while}} &= \sum_{\mbox{iterations}}(t_{\mbox{condition}} + t_{\mbox{body}}) + t_{\mbox{condition}}^{\mbox{final}} \\ &= \sum_{i=1}^{\mbox{size}} (O(1) + O(1)) + O(1) \\ &= O\left(\sum_{i=1}^{\mbox{size}} 1\right) + O(1) \\ &= O(\mbox{size}) + O(1) \\ &= O(\mbox{size}) \end{align} \]
int addInOrder (int* array, int& size, int value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
while (toBeMoved >= 0 && value < array[toBeMoved]) { // cond: O(1) #: size total: O(size)
array[toBeMoved+1] = array[toBeMoved]; // O(1)
--toBeMoved; // O(1)
}
// Insert the new value
array[toBeMoved+1] = value; // O(1)
++size; // O(1)
return toBeMoved+1; // O(1)
}
Replacing the loop by this quantity … ,
int addInOrder (int* array, int& size, int value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
// O(size)
// Insert the new value
array[toBeMoved+1] = value; // O(1)
++size; // O(1)
return toBeMoved+1; // O(1)
}
… we get down to a statement sequence composed of $O(1)$ and $O(\mbox{size})$ terms. The total run time is therefore
\[ t_{\mbox{addInOrder}} = O(1) + O(\mbox{size}) + O(1) + O(1) + O(1) = O(\mbox{size}) \]
template <typename int>
int addInOrder (int* array, int& size, int value)
{
// O(size)
}
Note that, because none of the inputs to this function are actually named “n”, it is not proper to say the function is “O(n)” unless we explicitly define “n” to be equal to size.
int addInOrder (int* array, int& size, int value)
{
// Make room for the insertion
int toBeMoved = size - 1;
while (toBeMoved >= 0 && value < array[toBeMoved]) {
array[toBeMoved+1] = array[toBeMoved];
--toBeMoved;
}
// Insert the new value
array[toBeMoved+1] = value;
++size;
return toBeMoved+1;
}
A special point worth noting:
If we are adding a value that is greater than all elements already in array, this algorithm does 0 iterations of the loop.
Suppose we are given a series of values to insert into an initially empty array, and that these values are already sorted.
Then each new value will be greater than all the ones already inserted into the array.
Each call to addInOrder will use 0 iterations.
and so each call runs in $O(1)$ time (for this special case of inserting sorted elements)
We’ll make use of this special case later when we incorporate this function into more complex algorithms.
Suppose that, instead of working on arrays of integers, we wrote addInOrder as a template that would work on arrays of any data type that supported assignment and comparison with <:
template <typename Comparable>
int addInOrder (Comparable* array, int& size, Comparable value)
{
// Make room for the insertion
int toBeMoved = size - 1;
while (toBeMoved >= 0 && value < array[toBeMoved]) {
array[toBeMoved+1] = array[toBeMoved];
--toBeMoved;
}
// Insert the new value
array[toBeMoved+1] = value;
++size;
return toBeMoved+1;
}
Again, we might start by marking certain “obvious” lines as $O(1)$:
template <typename Comparable>
int addInOrder (Comparable* array, int& size, Comparable value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
while (toBeMoved >= 0 && value < array[toBeMoved]) {
array[toBeMoved+1] = array[toBeMoved]; ➀
--toBeMoved; // O(1)
}
// Insert the new value
array[toBeMoved+1] = value; ➁
++size; // O(1)
return toBeMoved+1; // O(1)
}
But look at the lines ➀ and ➁. What are the complexity of these? It’s actually hard to say.
Comparable value to another. We have no idea, really, what kinds of data types might be supplied for Comparable, other than the fact that they will all need to support operator= and operator<. Not all assignments are $O(1)$. In fact, we know that programmers can write their own assignment operators, so these could be arbitrarily complicated.So the best that we can do is to introduce a symbol for the time to perform an assignment of a Comparable value. Let’s call that $t_{a}$.
template <typename Comparable>
int addInOrder (Comparable* array, int& size, Comparable value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
while (toBeMoved >= 0 && value < array[toBeMoved]) {
array[toBeMoved+1] = array[toBeMoved]; // O(t_a)
--toBeMoved; // O(1)
}
// Insert the new value
array[toBeMoved+1] = value; // O(t_a)
++size; // O(1)
return toBeMoved+1; // O(1)
}
That means that the entire loop body it $O(t_a)$ (i.e., $O(t_a + 1) = O(t_a)$, because nothing can be smaller than $O(1)$.
template <typename Comparable>
int addInOrder (Comparable* array, int& size, Comparable value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
while (toBeMoved >= 0 && value < array[toBeMoved]) {
// O(t_a)
}
// Insert the new value
array[toBeMoved+1] = value; // O(t_a)
++size; // O(1)
return toBeMoved+1; // O(1)
}
Now, let’s look at the while loop. We have another problem in determining the complexity of the loop condition. We don’t know what the complexity of operator< will be for an arbitrary Comparable type. So let $t_c$ denote the time required to compare two Comparable values using <. Then the condition is $O(1)$ for the integer comparison and the boolean && plus $t_c$ for the Comparable comparison:
template <typename Comparable>
int addInOrder (Comparable* array, int& size, Comparable value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
while (toBeMoved >= 0 && value < array[toBeMoved]) { // cond: O(t_c)
// O(t_a)
}
// Insert the new value
array[toBeMoved+1] = value; // O(t_a)
++size; // O(1)
return toBeMoved+1; // O(1)
}
At worst, the loop repeats size times:
int addInOrder (Comparable* array, int& size, Comparable value)
{
// Make room for the insertion
int toBeMoved = size - 1; // O(1)
while (toBeMoved >= 0 && value < array[toBeMoved]) { // cond: O(t_c) #: size
// O(t_a)
}
// Insert the new value
array[toBeMoved+1] = value; // O(t_a)
++size; // O(1)
return toBeMoved+1; // O(1)
}
That makes the total time for the while loop:
\[ \begin{align} t_{\mbox{while}} &= \sum_{\mbox{iterations}}(t_{\mbox{condition}} + t_{\mbox{body}}) + t_{\mbox{condition}}^{\mbox{final}} \\ &= \sum_{i=1}^{\mbox{size}} (O(t_c) + O(t_a)) + O(t_c) \\ &= O\left(\mbox{size} * t_a + (\mbox{size}+1)*t_c\right) \\ &= O(\mbox{size} * t_a + \mbox{size} * t_c) \end{align} \]
We would summarize this in English by stating that in the worst case, addInOrder performs O(size) assignments and O(size) comparisons of the Comparable type.
A more modern style of C++ would urge writing it in terms of iterators so that it could be used with containers other than conventional arrays:
/**
*
* Assume the elements of a container are already in order
* Find the position where value could be added to keep
* everything in order, and insert it there.
*
* Return the position where it was inserted. This assumes that
* we have room in the container for one more element at the
* given stop position.
*
* @param start beginning position of the sequence within which
* to insert a value.
* @param stop Position just after the last element to be examined
* in determining where to place value. In other words,
* the range (start,stop] is considered to contain
* already ordered data.
* @param value value to add into the container
* @return the position where the element was added.
* Upon exit from this function, all data in the range
* (start,stop) will be ordered.
*/
template <typename Iterator, typename Comparable>
int addInOrder (Iterator start, Iterator stop, const Comparable& value)
{
Iterator preStop = stop;
--preStop;
while (stop != start && value < *preStop) {
*stop = *preStop;
--stop;
--preStop;
}
// Insert the new value
*stop = value;
return stop;
}
toBeMoved as in the earlier array-based version, we here use stop, decrementing it until we find the right position at which to insert the value.preStop in this code fulfills the same purpose as the expression array[toBeMoved-1] – it holds the position just in front of the one we are considering as a candidate insertion position.In all analyses in this course, we will assume, unless stated otherwise, that iterator operations have the same complexity as pointer operations. Thus *stop, --stop, iterator assignment, and iterator comparison are all $O(1)$. So we can, gain, start by marking some “obvious” statements:
template <typename Iterator, typename Comparable>
int addInOrder (Iterator start, Iterator stop, const Comparable& value)
{
Iterator preStop = stop; // O(1)
--preStop; // O(1)
while (stop != start && value < *preStop) {
*stop = *preStop; ➀
--stop; // O(1)
--preStop; // O(1)
}
// Insert the new value
*stop = value; ➁
return stop; // O(1)
}
As before, the assignments (➀, ➁) of Comparable objects are of unknown complexity, so we introduce a symbol to denote the time required for such assignments:
template <typename Iterator, typename Comparable>
int addInOrder (Iterator start, Iterator stop, const Comparable& value)
{
Iterator preStop = stop; // O(1)
--preStop; // O(1)
while (stop != start && value < *preStop) {
*stop = *preStop; // O(t_a)
--stop; // O(1)
--preStop; // O(1)
}
// Insert the new value
*stop = value; // O(t_a)
return stop; // O(1)
}
and introduce a symbol $t_c$ for the time to perform a comparison:
template <typename Iterator, typename Comparable>
int addInOrder (Iterator start, Iterator stop, const Comparable& value)
{
Iterator preStop = stop; // O(1)
--preStop; // O(1)
while (stop != start && value < *preStop) { // comp: O(t_c)
*stop = *preStop; // O(t_a)
--stop; // O(1)
--preStop; // O(1)
}
// Insert the new value
*stop = value; // O(t_a)
return stop; // O(1)
}
We can collapse the loop body:
template <typename Iterator, typename Comparable>
int addInOrder (Iterator start, Iterator stop, const Comparable& value)
{
Iterator preStop = stop; // O(1)
--preStop; // O(1)
while (stop != start && value < *preStop) { // comp: O(t_c)
// O(t_a)
}
// Insert the new value
*stop = value; // O(t_a)
return stop; // O(1)
}
How do we express the worst-case number of iterations of the loop? We no longer have the parameter size to tell us how many elements were in the container, but that’s the idea that we need.
There is a way to express that notion within C++. There is a std function distance(iter1,iter2) that, for a pair of iterators, returns the number of distinct positions in the range starting at iter1 and going up to, but not including, iter2.
So we could annotate the loop as
while (stop != start && value < *preStop) { // comp: O(t_c) #: distance(start,stop)
I like distance(...) as a notation for discussing iterators, but would strongly discourage you from ever using it in actual code. The reason is that distance is one of those functions that is implemented differently (using the same iterator traits trick that we used earlier) depending on what variety of iterators it is given.
If the two iterators are random-access iterators, distance is implemented as
template <typename Iterator>
size_t distance (Iterator start, Iterator stop)
{
return stop - start;
}
which is $O(1)$.
But if the two iterators are not random access, then
distance is implemented as
template <typename Iterator>
size_t distance (Orator start, Iterator stop)
{
size_t counter = 0;
while (start != stop)
{
++counter;
++stop;
}
return counter;
}
which is O(distance(start,stop)).
So I have seen some horrible code like
template <typename Iterator>
void doSomething (Iterator start, Iterator stop)
{
Iterator pos = start;
for (int i = 0; i < distance(start, stop); ++i)
{
doSomethingWith(*pos);
++pos;
}
}
which looks like it should be linear in the number of elements being processed, but is often quadratic instead.
but the expression distance(start,stop) has two problems:
First, the value of stop is constantly changing inside the loop. What we really want to say is distance(start,value_of_stop_when_we_first_entered_the_function). This is actually a common enough problem that we’ll adopt a convention in this course:
For any parameter $x$ to a function that is reassigned new values as the function runs, we will use the notation $x_0$ to denote its value upon entry to the function. Inside the code, we write this as
x_0.
So we could annotate that loop as
while (stop != start && value < *preStop) { // comp: O(t_c) #: distance(start,stop_0)
But even that is more than I want to be typing out within mathematical expressions, so…
Let $N$ denote distance(start,stop_0). Then we have:
template <typename Iterator, typename Comparable>
int addInOrder (Iterator start, Iterator stop, const Comparable& value)
{
Iterator preStop = stop; // O(1)
--preStop; // O(1)
while (stop != start && value < *preStop) { // comp: O(t_c) #: N
// O(t_a)
}
// Insert the new value
*stop = value; // O(t_a)
return stop; // O(1)
}
We can then quickly conclude that
\[ t_{\mbox{while}} = O(N t_a + N t_c) \]
template <typename Iterator, typename Comparable>
int addInOrder (Iterator start, Iterator stop, const Comparable& value)
{
Iterator preStop = stop; // O(1)
--preStop; // O(1)
while (stop != start && value < *preStop) { // comp: O(t_c) #: N total: O(N t_a + N t_c)
// O(t_a)
}
// Insert the new value
*stop = value; // O(t_a)
return stop; // O(1)
}
and eventually that the entire function is $O(N t_a + N t_c)$ where $N$ is distance(start,stop) and $t_a$ and $t_c$ denote the time required to assign and compare Comparable values, respectively.
Another common utility function is to search through a range of data looking for a “search key”, returning the position where it is found and some “impossible” position if it cannot be found.
You might have seen code like this, for example,:
/*
* Search an array for a given value, returning the index where
* found or -1 if not found.
*
* From Malik, C++ Programming: From Problem Analysis to Program Design
*
* @param list the array to be searched
* @param listLength the number of data elements in the array
* @param searchItem the value to search for
* @return the position at which value was found, or -1 if not found
*/
template <typename T>
int seqSearch(const T list[], int listLength, T searchItem)
{
int loc;
for (loc = 0; loc < listLength; loc++)
if (list[loc] == searchItem)
return loc;
return -1;
}
The code discussed here is available as an animation that you can run to see how it works.
But we’re going to look at the standard, iterator-based version. The std library has a function find that could be implemented as
template <typename Iterator, typename T>
Iterator find (Iterator start, Iterator stop, const T& key)
{
while (start != stop && !(key == *start))
++start;
return start;
}
start points to a value equal to key. If so, we return the start position. Otherwise, we increment it and keep going.start has become equal to stop), we exit the loop and return start, which is equal to stop.Note that we never examine the data at position stop, because ranges in the C++ iterator style are always “up to but not including” the second iterator. So we could not possibly find the key at position stop because we will never look there. Hence returning a value of stop is the “impossible” value return that says we could not find the key.
Again, we can start the analysis by marking the “easy” stuff – the non-compound statements.
template <typename Iterator, typename T>
Iterator find (Iterator start, Iterator stop, const T& key)
{
while (start != stop && !(key == *start))
++start; // O(1)
return start; // O(1)
}
Looking at the while loop condition, start != stop and the &&, !, and * operations are $O(1)$. But key == *start depends on the underlying type T.
Let $t_c$ denote the complexity of comparing two T values of equality. Then
template <typename Iterator, typename T>
Iterator find (Iterator start, Iterator stop, const T& key)
{
while (start != stop && !(key == *start)) // cond: O(t_c)
++start; // O(1)
return start; // O(1)
}
Let $N$ denote distance(start,stop), the number of elements we can search through.
template <typename Iterator, typename T>
Iterator find (Iterator start, Iterator stop, const T& key)
{
while (start != stop && !(key == *start)) // cond: O(t_c) #: N
++start; // O(1)
return start; // O(1)
}
The while loop is therefore $O(N t_c)$:
template <typename Iterator, typename T>
Iterator find (Iterator start, Iterator stop, const T& key)
{
while (start != stop && !(key == *start)) // cond: O(t_c) #: N total: O(N t_c)
++start; // O(1)
return start; // O(1)
}
template <typename Iterator, typename T>
Iterator find (Iterator start, Iterator stop, const T& key)
{
// O(N t_c)
return start; // O(1)
}
and the entire function is $O(N t_c)$ where $N$ denotes distance(start,stop) (the number of elements we can search through) and $t_c$ denotes the complexity of comparing two T values of equality.
In our introduction to iterators we looked at the interesting function std::lower_bound, which makes a compile-time choice of doing either an ordered search or a binary search depending on what kinds of iterators it is passed.
We wrap up these case studies by analyzing those two search functions, starting with the ordered sequential search.
template <typename Iterator, typename Value>
Iterator orderedSearch (Iterator start, Iterator stop, const Value& key)
{
while ((start != stop) && (*start < key))
++start;
return start;
}
We can mark the simple statements that are $O(1)$:
template <typename Iterator, typename Value>
Iterator orderedSearch (Iterator start, Iterator stop, const Value& key)
{
while ((start != stop) && (*start < key))
++start; //O(1)
return start; //O(1)
}
Looking at the while loop condition, the (start != stop) part will be $O(1)$, as will the applications of the && and * operators. The < operator, however, is an operation provided by the unknown Value type, so
Value type.The loop condition is then $O(t_c)$.
Iterator orderedSearch (Iterator start, Iterator stop, const Value& key)
{
while ((start != stop) && (*start < key)) //cond: O(t_c)
++start; //O(1)
return start; //O(1)
}
Question: What is the worst case input for this function (for any fixed input size)?
The worst case occurs when key is nowhere in the range being searched. In that case, the loop repeats until start is advanced all the way forward tostop.
Question: How many times does the loop repeat in that worst case?
$\mbox{distance}(\mbox{start}_0,\mbox{stop})$, a.k.a., the number of positions between (the initial value of) start and stop.
start and stop.Then we have
Iterator orderedSearch (Iterator start, Iterator stop, const Value& key)
{
while ((start != stop) && (*start < key)) //cond: O(t_c) #: N
++start; //O(1)
return start; //O(1)
}
So we have a loop that repeats $N$ times, and the times of its body and condition do not depend on which iteration they are evaluated in. We’ve seen that pattern enough times to know how it works:
\[ t_{\mbox{while}} = N*O(t_c) = O(N * t_c) \]
Iterator orderedSearch (Iterator start, Iterator stop, const Value& key)
{
while ((start != stop) && (*start < key)) //cond: O(t_c) #: N total: O(N*t_c)
++start; //O(1)
return start; //O(1)
}
Collapsing the loop:
Iterator orderedSearch (Iterator start, Iterator stop, const Value& key)
{
// O(N*t_c)
return start; //O(1)
}
and the total complexity is $O(N t_c)$.
This function’s time is linear in the number of elements being searched.
Finally, let’s look at binary search.
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0;
auto high = stop - start - 1;
while( low <= high )
{
auto mid = ( low + high ) / 2;
RandomAccessIterator midPos = start + mid;
if( *midPos < key )
low = mid + 1;
else if( key < *midPos )
high = mid - 1;
else
return midPos; // Found
}
return start + low;
}
Let’s mark the simple $O(1)$ statements to start:
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{
auto mid = ( low + high ) / 2; // O(1)
RandomAccessIterator midPos = start + mid; // O(1)
if( *midPos < key )
low = mid + 1; // O(1)
else if( key < *midPos )
high = mid - 1; // O(1)
else
return midPos; // Found // O(1)
}
return start + low; // O(1)
}
Now look at the innermost if (the “else if”). The then parts and else parts are already marked as $O(1)$. The condition, however, depends on the time required to compare two elements of the unknown Value type.
Value type.Then the condition is $O(t_c)$:
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{
auto mid = ( low + high ) / 2; // O(1)
RandomAccessIterator midPos = start + mid; // O(1)
if( *midPos < key )
low = mid + 1; // O(1)
else if( key < *midPos ) //cond: O(t_c)
high = mid - 1; // O(1)
else
return midPos; // Found // O(1)
}
return start + low; // O(1)
}
So that if takes time $O(t_c) + \max{O(1), O(1)} = O(t_c)$.
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{
auto mid = ( low + high ) / 2; // O(1)
RandomAccessIterator midPos = start + mid; // O(1)
if( *midPos < key )
low = mid + 1; // O(1)
else if( key < *midPos ) //cond: O(t_c) total: O(t_c)
high = mid - 1; // O(1)
else
return midPos; // Found // O(1)
}
return start + low; // O(1)
}
Collapsing:
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{
auto mid = ( low + high ) / 2; // O(1)
RandomAccessIterator midPos = start + mid; // O(1)
if( *midPos < key )
low = mid + 1; // O(1)
else
// O(t_c)
}
return start + low; // O(1)
}
The remaining if also has a condition that takes time $O(t_c)$:
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{
auto mid = ( low + high ) / 2; // O(1)
RandomAccessIterator midPos = start + mid; // O(1)
if( *midPos < key ) //cond: O(t_c)
low = mid + 1; // O(1)
else
// O(t_c)
}
return start + low; // O(1)
}
which means that this if takes time $O(t_c) + \max{O(1), O(t_c)}$.
Despite the more expensive else part than in the previously analyzed if, this still simplifies to $O(t_c)$.
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{
auto mid = ( low + high ) / 2; // O(1)
RandomAccessIterator midPos = start + mid; // O(1)
if( *midPos < key ) //cond: O(t_c) total: O(t_c)
low = mid + 1; // O(1)
else
// O(t_c)
}
return start + low; // O(1)
}
Collapsing,
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{
auto mid = ( low + high ) / 2; // O(1)
RandomAccessIterator midPos = start + mid; // O(1)
// O(t_c)
}
return start + low; // O(1)
}
which means that the loop body is $O(1) + O(1) + O(t_c) = O(t_c)$.
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{//total: O(t_c)
auto mid = ( low + high ) / 2; // O(1)
RandomAccessIterator midPos = start + mid; // O(1)
// O(t_c)
}
return start + low; // O(1)
}
Collapsing,
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high )
{
// O(t_c)
}
return start + low; // O(1)
}
The loop condition is $O(1)$:
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high ) //cond: O(1)
{
// O(t_c)
}
return start + low; // O(1)
}
Now, how many times does this loop repeat?
To answer this question let’s go back to the original listing.
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0;
auto high = stop - start - 1;
while( low <= high )
{
auto mid = ( low + high ) / 2;
RandomAccessIterator midPos = start + mid;
if( *midPos < key )
low = mid + 1;
else if( key < *midPos )
high = mid - 1;
else
return midPos; // Found
}
return start + low;
}
Remember how this function actually works.
low … high define our current search area. Initially, there are distance(start, stop) items in this area.
The two values $\mbox{low}$ and $\mbox{high}$ define our current search range. If the value we are looking for is somewhere in the array, it is going to be at a position $\geq \mbox{low}$ and $\leq \mbox{high}$. So the difference, $\mbox{high} - \mbox{low} + 1$, defines how many values we have left in the search range.
On any given iteration of the loop:
there are $\mbox{high} - \mbox{low} + 1$ items in the search area
Each time around the loop, we cut this area in half.
We stop when the search area has been reduced to a single item.
Let $N$ denote distance(start, stop), the number of positions being searched.
How many times can we divide $N$ things into $2$ equal parts before getting down to only $1$?
That’s the interesting question!
If I start with $N$ things in this search area, how many times can I keep cutting that search area of $N$ things in half till I get to only a single item? (An item that must be the value we are looking for, if that value is anywhere in the array at all.)
Let’s assume for the sake of our argument that $N$ is some power of 2 to start with. So we cut $N$ in half – we get $N/2$. Next time we cut that half which makes $N/4$, then $N/8$ and then $N/16$ and so on. And the question is, how often can we keep doing that until we reduce the number down to just $1$?
The answer may be a bit clearer if we turn the problem around. Start at $1$ and keep doubling until we get $N$. So we proceed
$$1, 2, 4, \ldots $$
and we keep going until we actually get up to $N$. The number of steps we did in doubling is same as number of steps when we start at $N$ and kept dividing by $2$.
How many steps is that? Well, what power of $2$ do we reach when we finally get to $N$? Suppose we took $k$ steps. Then we are saying that $N = 2^k$. Solving for $k$, we get $k = \log_2 N$.
The use of $\log_2\mbox{}$ is so common in Computer Science that we generally assume that any logarithm written without its base is to the base 2. In other words, computer scientists would usually write this as $k = \log N$. (That’s in contrast to other mathematical fields, where a logarithm with no base is generally assumed to be base 10.)
What if $N$ is not an exact power of 2? In that case, $\log_2 N$ is not an integer, and we would need to divide or double
$$\left\lceil \log_2 N \right\rceil$$
times, where $\lceil \; \rceil$ denotes the ceiling or “next integer higher than”.
In practice, we won’t need to worry about that fractional part, because we are going to use this inside a big-O expression, where the fractional part will be dominated in any sum by the logarithm itself.
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high ) //cond: O(1) #: log N
{
// O(t_c)
}
return start + low; // O(1)
}
Again, the condition and body times are independent of which iteration we are in, so
\[ t_{\mbox{while}} = (\log N)*(O(1) + O(t_c)) + O(1) = O(t_c \log N) \]
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
while( low <= high ) //cond: O(1) #: log N total: O(t_c log(N))
{
// O(t_c)
}
return start + low; // O(1)
}
Collapsing,
template <typename RandomAccessIterator, typename Value>
RandomAccessIterator binarySearch (RandomAccessIterator start,
RandomAccessIterator stop,
const Value& key)
{
auto low = 0; // O(1)
auto high = stop - start - 1; // O(1)
// O(t_c log(N))
return start + low; // O(1)
}
and the total complexity of the function is $O(t_c \log(N))$ where $t_c$ is the time required to compare two elements and where $N$ is the number of positions being searched.
Class member functions are analyzed in the same way as standalone functions.
The only “trick” is to remember that a member function has the implicit parameter, this, which provides access to the object on which the member function is being applied, i.e., the one on the left when we make a call like
anObject.memberFunction(param1, param2);
Every (non-static) member function has an implicit this parameter. We may declare something like:
class Book
{
⋮
void addAuthor (const Author& au);
⋮
}
but the compiler treats this function as if we had actually written:
void addAuthor (Author* this, const Author& au);
Or, for a const member function, e.g.,
class Book
{
⋮
std::string getTitle () const;
⋮
}
as if we had written
std::string getTitle (const Author* this);
It’s via the this pointer that a member function has access to its own data and function members, even though writing out this-> is optional in C++.
std::string getTitle() const
{
return title; // could have written this->title
}
The section below needs to be rewritten to be compatible with the
Book::code actually used in the earlier lessons.
Consider this code from our early Book ADT
void Book::addAuthor (const Author& au)
{
assert (numAuthors < MaxAuthors);
addIfNotPresent(authors, numAuthors, au.getName());
}
First, look at the declaration of addAuthor. We expect any function’s complexity to be written in terms of its inputs, so what are the inputs to this function?
The inputs are au and this, the pointer to the current value of the Book that we want to modify. Don’t forget “this”!
We expect the answer to be written in terms of some numerical properties of that author and/or that book.
OK, so, let’s look at the first statement in addAuthor:
void Book::addAuthor (const Author& au)
{
assert (numAuthors < MaxAuthors);
addIfNotPresent(authors, numAuthors, au.getName());
}
What is the complexity of numAuthors < MaxAuthors ?
This is a simple comparison of two integers, so it is O(1).
What about the call to assert? assert is a functions used for defensive programming purposes. During development, it is implemented roughly as
void assert (bool test)
{
if (!test)
sendErrorMEssageAndAbort();
}
but when compiling for production code, it is simply a comment and does not translate to any code at all. Consequently, we will treat it as $O(1)$.
How should we annotate the assert statement?
void Book::addAuthor (const Author& au)
{
assert (numAuthors < MaxAuthors); // O(1)
addIfNotPresent(authors, numAuthors, au.getName());
}
Now let’s look at the second statement:
addIfNotPresent(authors, numAuthors, au.getName());
To get the complexity of this statement, we would need to know the complexity of the two functions addIfNotPresent and Book::getName.
We’ve seen that getName is implemented like this:
class Author
{
⋮
std::string getName() const {return name;}
⋮
}
What is the complexity of getName()?
The return statement copies a string, and string copies take time proportional to the number of characters being copied, so the complexity would be the length of the string name.
class Author
{
⋮
std::string getName() const {return name;} // O(this->name.length())
⋮
}
Returning to
addIfNotPresent(authors, numAuthors, au.getName());
In this instance, we have called getName() on au.
What is the complexity of au.getName()?
When we look at a call to a function, we substitute the actual parameters in the fall for the formal parameters in the function complexity, so au.getName() is O(au.name.length()).
Now, to go any further, we need to know the complexity of addIfNotPresent. We have not actually looked at the code for this before, so here it is:
template <typename T>
void addIfNotPresent (T* array, int& N, const T& key)
{
if ( find(array, array+N, key) == array+N )
{
array[N] = key;
++N;
}
}
As always, we work from inside out, so start with the then part of the if.
The line
array[N] = key;
copies an element of type T. But because this is a template, we don’t know what T is and so do not know the complexity of copying that. So the best we can do is to define a symbol $t_{\mbox{copy}}$ to denote the time required to copy an element of type T.
How do we annotate this?
template <typename T>
void addIfNotPresent (T* array, int& N, const T& key)
{
if ( find(array, array+N, key) == array+N )
{
array[N] = key; // O(t_copy) where t_copy is the time to copy a T object
++N;
}
}
The next statement is much simpler, a simple increment of an int, so this would be $O(1)$.
template <typename T>
void addIfNotPresent (T* array, int& N, const T& key)
{
if ( find(array, array+N, key) == array+N )
{
array[N] = key; // O(t_copy) where t_copy is the time to copy a T object
++N; // O(1)
}
}
So the total complexity of the if’s then part is $O(t_{\mbox{copy}} + 1)$ (adding up the statement complexities of a sequence), which simplifies to $O(t_{\mbox{copy}})$.
How do we annotate that?
template <typename T>
void addIfNotPresent (T* array, int& N, const T& key)
{
if ( find(array, array+N, key) == array+N )
{ //total: O(t_copy)
array[N] = key; // O(t_copy) where t_copy is the time to copy a T object
++N; // O(1)
}
}
Next we turn our attention to the if condition. This follows our iterator-based pattern of operating on a range of positions (from array up to but not including array+N) and comparing to the end position (array+N) to see if we have gone through the entire range. Luckily, we have already analyzed find.
What is the complexity of the condition to the if statement?
When we did the analysis of the function find, we used “N” to denote the number of elements in the range. But that’s just a mathematical symbol of our own selection and we can call it anything we want. I’ve chosen to call it “M” here to avoid confusion with the variable named “N” that is part of the code we are analyzing.
Remember that it is critical in mathematics (as it is in programming) that we not try to use one symbol to mean multiple things. In this case, I was being careful to make a point about the difference between the prior analysis of the function find and the application of that result to this particular call to find, even though, in the end, we determine that $M=$N.
The find function is $O(M t_{\mbox{comp}})$ where $M$ is the number number of elements in the range to be searched and $t_{\mbox{comp}}$ is the time required to compare two of those elements for equality.
In this call, we have $N$ elements in the range being searched. We don’t know how long it takes to compare two elements of the unknown type T, so let’s keep the symbol $t_{\mbox{comp}}$ for that purpose.
So the call to find takes time $O(Nt_{\mbox{comp}})$ after its parameters have been evaluated and passed. The total time to evaluate the parameters was $O(t_{\mbox{copy}})$ because of the getName() call. So the total complexity of the find call is $O(Mt_{\mbox{comp}} + t_{\mbox{copy}})$.
template <typename T>
void addIfNotPresent (T* array, int& N, const T& key)
{
if ( find(array, array+N, key) == array+N ) //cond:O(N*t_comp + t_copy)
{ //total: O(t_copy)
array[N] = key; // O(t_copy) where t_copy is the time to copy a T object
++N; // O(1)
}
}
What is the complexity of the entire if statement?
We now know the complexity of the condition and the then-part. The else-part is empty, making it $O(1)$. So, by our rule for if statements, the total time is
\[ t_{\mbox{cond}} + \max(t_{\mbox{then}}, t_{\mbox{else}}) \]
\[ = O(Nt_{\mbox{comp}} + t_{\mbox{copy}}) + \max(O(t_{\mbox{copy}}), O(1)) \]
\[ = O(N t_{\mbox{comp}} + t_{\mbox{copy}}) + O(t_{\mbox{copy}}) \]
\[ = O(N t_{\mbox{comp}} + t_{\mbox{copy}} + t_{\mbox{copy}}) \]
\[ = O(N t_{\mbox{comp}} + 2 t_{\mbox{copy}}) \]
\[ = O(N t_{\mbox{comp}} + t_{\mbox{copy}}) \]
template <typename T>
void addIfNotPresent (T* array, int& N, const T& key)
{
if ( find(array, array+N, key) == array+N ) //cond:O(N*t_comp + t_copy) total: O(N*t_comp + t_copy)
{ //total: O(t_copy)
array[N] = key; // O(t_copy) where t_copy is the time to copy a T object
++N; // O(1)
}
}
And, because that if statement is the only statement in the function body, we conclude that addIfNotPresent has complexity $O(N t_{\mbox{comp}} + t_{\mbox{copy}})$.
Finally, we can return to the function that we were trying to analyze in the first place:
void Book::addAuthor (const Author& au)
{
assert (numAuthors < MaxAuthors); // O(1)
addIfNotPresent(authors, numAuthors, au.getName());
}
In this call, N is replaced by numAuthors (or this->numAuthors if you want to spell it out). T is replaced by std::string, so $t_{\mbox{copy}}$ and $t_{\mbox{comp}}$ now refer to the times to copy and compare strings. So let $L$ denote the longest author name in our publisher’s records and we can say that $t_{\mbox{copy}} \in O(L)$ and $t_{\mbox{comp}} \in O(L)$.
That makes the complexity of the second statement $O(\mbox{numAuthors}*L + L)$, which simplifies to $O(\mbox{numAuthors}*L)$.
void Book::addAuthor (const Author& au)
{ // Let L be the length of the longest author name
assert (numAuthors < MaxAuthors); // O(1)
addIfNotPresent(authors, numAuthors, au.getName()); // O(numAuthors*L)
}
Actually, though, there is a good chance that $L$ has a relatively small bound. After all, the name has fit attractively on the book cover. If Raymond Algernon Luxury-Yacht1 gets a publishing contract, he will probably be asked to select a shorter pen-name. So we would not be amiss to claiming that there is a small constant bound for $L$, in which case $O(L) = O(1)$ and we wind up with
void Book::addAuthor (const Author& au)
{ // Assume that all author names are fairly short
assert (numAuthors < MaxAuthors); // O(1)
addIfNotPresent(authors, numAuthors, au.getName()); // O(numAuthors)
}
Then adding up the statements that make up the statement sequence in the body:
void Book::addAuthor (const Author& au)
{ // Assume that all author names are fairly short
//total: O(numAuthors)
assert (numAuthors < MaxAuthors); // O(1)
addIfNotPresent(authors, numAuthors, au.getName()); // O(numAuthors)
}
This is a very good example of the point I made at the start of the analysis. The complexity of a function must be described entirely in terms of the inputs to that function, but for member functions, the this object is an implicit input.
The following functions and function templates have been analyzed in this lesson and are now available for use in quizzes, assignments, and later lessons:
template <typename Iterator, typename T>
Iterator find (Iterator start, Iterator stop, const T& key)
distance(start,stop) (the # of positions between start and stop) and $t_c$ is the time required to compare two items of type T for equality.template <typename Iterator, typename Value>
Iterator lower_bound (Iterator start, Iterator stop, const Value& key)
Let $n$ be distance(start,stop) (the # of positions between start and stop) and let $t_c$ is the time required to compare two items of type T using operator<.
Iterator is a random access iterator, this function uses binary search and is $O(t_c \log n)$.Iterator is not a random access iteratir (e.g., a forward or bi-directional iterator) , this function uses ordered sequential search and is $O(t_c n)$.1: Actually, his name is spelled “Luxury-Yacht”, but it’s pronounced “Throatwarbler Mangrove”.
| | 1 of 21
| |