1 General Testing Techniques

Let us focus on general testing techniques first.

1.1 Head-to-Head Testing

Head-to-head-testing is essentially comparing to sets of output:

• “correct” (reference) output generated by hand (e.g., based on a requirements document) or from a known “correct” program.

• actual output generated by the program being evaluated.

1.2 Black-Box Testing

Black-Box testing involves writing tests for a “black box.” What is a “black box?” This type of testing focuses creating test data for a function, class, or program with a known interface, but whose implementation details are unknown. In fact… this technique is used in Test Driven Development (TDD) to write unit tests before actually implementing any code.

Let us go through a quick example. Suppose you are provided the following function declaration and told to design a few tests.

Example 1: Simple Addition Function

/**
 * Add two integers (any combination of positive, negative, or zero) together.
 *
 * @param num1 - first number
 * @param num2 - second number
 *
 * @returns sum of num1 and num23  (i.e., num1 + num2)
 *
 * @pre (num1 >= -10000 && num1 <= 10000) 
 *   && (num2 >= -10000 && num2 <= 10000) 
 */
int addTwoIntegers(const int num1, const int num2);

The function has full documentation. The constraints (e.g., preconditions) of the function are explicitly stated. All numbers must fall in the range $-10,000$ to $10,000$. We would want to test a few things:

• boundary conditions
  • combinations of 10,000 and -10,000
  • combinations close 10,000 and -10,000 but still in the defined range
• combinations of positive numbers
• combinations of negative numbers
• two zeroes
• combinations of negative numbers, zeroes, and positive numbers

Since we know how to add (usually…), the correct sums can be computed by hand. When we obtain the definition of addTwoIntegers the results generated by the function can be compared to the know correct results.

1.3 White-Box Testing

White-Box testing complements Black-Box testing. While Black-Box testing ignores the code itself (instead focusing on interfaces and requirements), White-Box testing focuses on the code and all but ignores the requirements.

White-Box testing often examines which lines of code are executed and which branches are taken. The process general makes use of a tool such as gcov (C++), or jacoco (Java), coverage.py (Python), Tarpaulin (Rust). These are each code coverage tools… used to track which lines of code are “touched” when during a program’s normal execution (i.e., using the main driver function) or when some type of tests (e.g., Black-Box or Unit) are run.

Examples of such tools will be discussed briefly in CS 330 (Object Oriented Programming & Design) and in future coursework (e.g., CS 350).

1.4 Formal Testing Frameworks