Review - Head to Head Testing

Thomas J. Kennedy

Contents:
1 Setting Up an Example
2 A Few Quick Revisions & Manual Output Inspection
3 Checking Output with Some Help

One of the most useful tools in a programmer’s toolkit is head to head testing. Head to head testing is the process of comparing two or more pieces of output, e.g.,

1 Setting Up an Example

Suppose that you have been tasked with implementing a program that outputs all even numbers between lower_bound and upper_bound. Let us generate some example “correct” outputs. Make sure to save the reference outputs in reference-1.txt and reference-2.txt.

Example 1: reference-1.txt

When run with ./even_gen 15 32 the following output must be generated:

Range [15, 32]:

    16
    18
    20
    22
    24
    26
    28
    30
    32
Example 2: reference-2.txt

When run with ./even_gen 8 16 the following output must be generated:

Range [8, 16]:

    8
    10
    12
    14
    16

Let us tackle the first piece of output together.

2 A Few Quick Revisions & Manual Output Inspection

Suppose we just wrote the following code…

even_gen_1.cpp 
#include <iostream>

using namespace std;

int main(int argc, char** argv)
{
    // Check and parse command line args
    if (argc < 3) {
       cout << " Usage: ./even_gen [lower_bound] [upper_bound]" << "\n";
       return 1;
    }

    // Assume all args are well formed (i.e., can be parsed as integers).
    int lower_bound = atoi(argv[1]);
    int upper_bound = atoi(argv[2]);

    // The core even output logic
    cout << "Range [" << lower_bound << ", " << upper_bound << "]" << "\n";
    cout << "\n";

    for (int next_even = lower_bound; next_even <= upper_bound; next_even++) {
        cout << next_even << "\n";
    }

    return 0;
}

compiled it into even_gen and ran it with ./even_gen 15 32. We would obtain the following output…

Range [15, 32]

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

Notice the issues? The first couple issues are obvious… there are odd numbers and the first number is 15!

Our first mistake is obvious… next_even++ should be next_even += 2. Let us make the correction and move the loop to its own function (i.e., no monolithic functions).

even_gen_2.cpp 
#include <iostream>

using namespace std;

/**
 * Output all even numbers in a given range.
 *
 * @param lower lower integer bound (a)
 * @param upper upper integer bound (b)
 */
void output_even_integers(const int lower, const int upper)
{
    for (int next_even = lower; next_even <= upper; next_even += 2) {
        cout << next_even << "\n";
    }
}

int main(int argc, char** argv)
{
    // Check and parse command line args
    if (argc < 3) {
       cout << " Usage: ./even_gen [lower_bound] [upper_bound]" << "\n";
       return 1;
    }

    // Assume all args are well formed (i.e., can be parsed as integers).
    int lower_bound = atoi(argv[1]);
    int upper_bound = atoi(argv[2]);

    // The core even output logic
    cout << "Range [" << lower_bound << ", " << upper_bound << "]" << "\n";
    cout << "\n";

    output_even_integers(lower_bound, upper_bound);

    return 0;
}

If we run the program again… we obtain…

Range [15, 32]

15
17
19
21
23
25
27
29
31

We have fewer numbers.. but they are all odd! Let us fix that by checking to see if lower_limit is even.

even_gen_3.cpp 
#include <iostream>

using namespace std;

/**
 * Output all even numbers in a given range.
 *
 * @param lower lower integer bound (a)
 * @param upper upper integer bound (b)
 */
void output_even_integers(const int lower, const int upper)
{
    const int first_even = (lower % 2 == 0 ? lower : lower + 1);

    for (int next_even = first_even; next_even <= upper; next_even += 2) {
        cout << next_even << "\n";
    }
}

int main(int argc, char** argv)
{
    // Check and parse command line args
    if (argc < 3) {
       cout << " Usage: ./even_gen [lower_bound] [upper_bound]" << "\n";
       return 1;
    }

    // Assume all args are well formed (i.e., can be parsed as integers).
    int lower_bound = atoi(argv[1]);
    int upper_bound = atoi(argv[2]);

    // The core even output logic
    cout << "Range [" << lower_bound << ", " << upper_bound << "]" << "\n";
    cout << "\n";

    output_even_integers(lower_bound, upper_bound);

    return 0;
}

The output looks correct..

Range [15, 32]

16
18
20
22
24
26
28
30
32

However, we are humans. We will miss certain differences.

3 Checking Output with Some Help

Instead of manually inspecting output, let us use a few Linux utilities. Rerun the program with:

./even_gen 15 32 > actual-1.txt

This command uses output redirection (through >) to save all output to actual-1.txt. Run

diff reference-1.txt actual-1.txt

You should see something similar to…

1c1
< Range [15, 32]:
---
> Range [15, 32]
3,11c3,11
<     16
<     18
<     20
<     22
<     24
<     26
<     28
<     30
<     32
---
> 16
> 18
> 20
> 22
> 24
> 26
> 28
> 30
> 32

The output definitely differs. However, diff normally falls short (i.e., the differences are difficult to analyze). Let us try 

sdiff reference-1.txt actual-1.txt

You should see…

Range [15, 32]:						      |	Range [15, 32]

    16							      |	16
    18							      |	18
    20							      |	20
    22							      |	22
    24							      |	24
    26							      |	26
    28							      |	28
    30							      |	30
    32							      |	32

That is a little better. However, I am partial to vimdiff. Run

vimdiff reference-1.txt actual-1.txt

You should see output similar to…

vimdiff

Take note of how vimdiff highlights the lines that differ and (in most cases) the parts of each line that differ. You can exit by pressing <Esc> then typing :qa and pressing enter.

Notice the last mistake? Our implementation is not outputting 4 spaces before each number. Let us make one last correction…

even_gen_4.cpp 
#include <iostream>

using namespace std;

/**
 * Output all even numbers in a given range.
 *
 * @param lower lower integer bound (a)
 * @param upper upper integer bound (b)
 */
void output_even_integers(const int lower, const int upper)
{
    const int first_even = (lower % 2 == 0 ? lower : lower + 1);

    for (int next_even = first_even; next_even <= upper; next_even += 2) {
        cout << "    " << next_even << "\n";
    }
}

int main(int argc, char** argv)
{
    // Check and parse command line args
    if (argc < 3) {
       cout << " Usage: ./even_gen [lower_bound] [upper_bound]" << "\n";
       return 1;
    }

    // Assume all args are well formed (i.e., can be parsed as integers).
    int lower_bound = atoi(argv[1]);
    int upper_bound = atoi(argv[2]);

    // The core even output logic
    cout << "Range [" << lower_bound << ", " << upper_bound << "]" << "\n";
    cout << "\n";

    output_even_integers(lower_bound, upper_bound);

    return 0;
}

Rerun ./even_gen 15 32 > actual-1.txt followed by the diff, sdiff and vimdiff commands. There is one more mistake… a missing : on the first line. I will leave adding the missing : as an exercise to the reader (i.e., you).

© 2015-2021, Old Dominion Univ.