The most basic of all white-box methods, statement coverage is the selection of tests so that every statement has been executed at least once.
does not have to happen in a single test - as long as one test in the suite executes a statement, it's covered
How would you test this function so that every statement is covered?
template <typename T> int seqSearch(const T list[], int listLength, T searchItem) { int loc; for (loc = 0; loc < listLength; loc++) if (list[loc] == searchItem) return loc; return -1; }
Can we cover this statement with a test?
int BidderCollection::add (const Bidder& value) // Adds this bidder //Pre: getSize() < getMaxSize() { if (size < MaxSize) { addToEnd (elements, size, value); return size - 1; } else { <[+]>cerr << "BidderCollection::add - collection is full" << endl;<[-]> exit(1); } }
During unit test, certainly.
During an integration or system test, probably not.
Typical of code inserted as "defensive programming"
We can check whether we have covered statements by adding debugging output.
add a unique output to each block of "straight line" code
#define COVER cerr << "Block " << __FILE__ << ":" << __LINE__ << endl template <typename T> int seqSearch(const T list[], int listLength, T searchItem) { <[+]>COVER;<[-]> int loc; for (loc = 0; loc < listLength; loc++) if (list[loc] == searchItem) <[+]>{ COVER;<[-]> return loc; <[+]>}<[-]> <[+]>COVER;<[-]> return -1; }
However, g++ and other compilers offer tools that can automate this process.
We can go a long way by simply being aware of the idea of statement coverage when we develop our tests.
Recognize that we want tests to cover every branch
But as programs get more complicated, even experienced testers do a poor job of stmt coverage
Automatic coverage tools can help
Coverage tools need to understand the control flow of your code
typically have to duplicate most of the work of a compiler
often cost big $$
It's a lot easier to do this if the compiler provides support in the first place
The gcc/g++ suite includes a coverage tool,
gcov
We will look at doing a unit test of the three search functions in arrayUtils.h
First, we need a test driver. We can go with either of two styles.
In one, we write a main program that reads data from a text stream (e.g., standard in), uses that data to construct arrays, and invokes each function on those arrays, printing the results of each.
Alternatively this version uses
no external input but instead contains code to construct the
data we need, then assert
s that the search
functions produce the expected results.
For this example we'll use the first style.
To use gcov
, we compile with special options
-fprofile-arcs -ftest-coverage
In Unix, we would add these to the compilation commands or add them to the C++ options in a makefile.
In Code::Blocks, we can add these to "Project ->
Build Options -> Compiler Settings -> Other
options", and also by going to ""Project -> Build
Options -> Linker Settings" and, under "Link
libraries", adding the libgcov.a
file found
inside the Code::Blocks installation directory, probably
under MingW\lib\gcc\mingw32\3.4.5
(the
version number at the end may vary).
When the code has been compiled, in addition to the usual
files there will be several files with endings like
.bb
, .bbg
. These hold data on where
the statements and branches in our code are.
Run your tests normally.
As you test, various *.da
files will accumulate
data on which statements have been covered.
Run gcov
mainProgram
the immediate output will be a report on the percentages of statements covered in each source code file.
Also creates a detailed report for each source code file. e.g.,
-: 69:template <typename T> -: 70:int seqSearch(const T list[], int listLength, T searchItem) -: 71:{ 1: 72: int loc; -: 73: 2: 74: for (loc = 0; loc < listLength; loc++) 2: 75: if (list[loc] == searchItem) 1: 76: return loc; -: 77: #####: 78: return -1; -: 79:}
Lists number of times each statement has been executed
Lists #### if a statement has never been executed
Focus on these as you choose additional tests
Code::Blocks users will find gcov
in the same
directory as the g++
executables. The easiest way to
handle this is to start a Windows cmd
window, cd to
the directory where Code::Blocks has placed your compiled code,
then do
pathToExecutables\gcov mainProgram