Abstract

It would be nice if every new ADT we wrote worked correctly the first time it compiled properly, but the real world just doesn’t work that way.

One advantage of organizing your code into a collection of ADTs is that ADTs provide, not only a convenient way to package up the functionality of your code, but also a convenient basis for testing.

The *Unit test frameworks give us a powerful tool for unit testing. But we still need to pick the tests.

In this lesson we look at how to use a unit test framework to “thoroughly” test an ADT. Our approach is based upon dividing the ADT interface into

• mutators: functions that alter the state or value of the ADT, and
• accessors: functions that examine the state of the ADT.

Our approach then becomes:

1. Each test case explores one mutator, and each mutator has at least one test case.
2. In each test case, assert the effect of that mutator as revealed through each accessor.

# 1 Be Smart: Generate and Check

In our earlier test for addContact, we weren’t particularly thorough:

TEST_F (MailingListTests, addContact) {
EXPECT_TRUE (mlist.contains("Jones"));
EXPECT_EQ ("Jones", mlist.getContact("Jones").getName());
EXPECT_EQ (4, ml.size());
}


• Missing boundary/special case: adding to an empty container

• Missing special cases: Adding to beginning or end of an ordered sequence

Some of our concerns could be addressed by adding tests but varying the parameters:

TEST_F (MailingListTests, addExistingContact) {
EXPECT_TRUE (mlist.contains("Baker"));
EXPECT_EQ ("Baker", mlist.getContact("Baker").getName());
EXPECT_EQ (3, ml.size());
}



## 1.1 Generate and Check

A useful design pattern for producing well-varied tests is

for v: varieties of ADT {
result = applyFunctionBeingTested(x);
check (x, result);
}

• generate and check could be separate functions or in-line

• depends on whether you can re-use in multiple tests
• Most common way to introduce “variety” would be size

• Could also be different constructors, in which case you might not literally have a loop:
   ADT x (constructor1);
result = applyFunctionBeingTested(x);
check (x, result);

result = applyFunctionBeingTested(x2);
check (x2, result);



A more elaborate fixture will aid in generating mailing lists of different sizes:

fixture.cpp
// The fixture for testing class MailingList.
class MailingListTests  {
public:
Contact jones;
vector<Contact> contacts;

MailingListTests() {
jones = Contact("Jones", "21 Penn. Ave.");
contacts.clear();
contacts.push_back (Contact ("Baker", "Drury Ln."));
contacts.push_back (Contact ("Holmes", "221B Baker St."));
contacts.push_back(jones);
contacts.push_back (Contact ("Wolfe", "454 W. 35th St."));
}

~MailingListTests() {
}

MailingList generate(int n) const
{
MailingList m;
for (int i = 0; i < n; ++i)
return m;
}

};


BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) {
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml (ml0);
BOOST_CHECK (ml.contains("Jones"));
BOOST_CHECK_EQUAL ("Jones", ml.getContact("Jones").getName());
BOOST_CHECK_EQUAL (ml.size(), sz);
else
BOOST_CHECK_EQUAL (ml.size(), sz+1);
}
}

• Here we generate mailing lists of a variety of sizes and add Jones to them

• At larger sizes, Jones is already in the list – functional case covered

• sz == 0 covers one of our boundary/special cases

BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) {
for (unsigned sel = 0; sel < contacts.size(); ++sel)
{
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml (ml0);
BOOST_CHECK_EQUAL (ml.size(), sz);
else
BOOST_CHECK_EQUAL (ml.size(), sz+1);
}
}
}

• We an also explore adding to different positions (beginning, middle, end)
• by varying which element we add

# 2 Be Thorough: Mutators and Accessors

OK, let’s say we want to design a unit test suite for our MailingList ADT.

Just staring at the ADT interface, where do we start? The criteria we suggested earlier (typical values, extremal values, special values) may be of help, but it’s far from obvious how to apply those ideas.

There is an organized way to approach this test design. It starts with the recognition that the member functions of an ADT can usually be divided into two groups - the mutators and the accessors.

Mutator functions are the functions that alter the value of an object. These include constructors and assignment operators.

Accessor functions are the functions that “look at” the value of an object but do not change it. Some functions may both alter an object and return part of its value - we’ll have to wing it a bit with those.

Mutators and Accessors

To test an ADT, divide the public interface into

• mutators: functions that alter the value of the object

• accessors: functions that “look at” the current value of an object

• Occasional functions will fall in both classes

The basic procedure for writing an ADT unit test is to

1. Consider each mutator in turn.

2. Write a test that begins by applying that mutator function.

3. Then consider how that mutator will have affected the results of each accessor.

4. Write assertions to test those effects.

Commonly, each mutator will be tested in a separate function.

## 2.1 Example: Unit Testing of the MailingList Class

mailinglist.h
#ifndef MAILINGLIST_H
#define MAILINGLIST_H

#include <iostream>
#include <string>

#include "contact.h"

/**
A collection of names and addresses
*/
class MailingList
{
public:
MailingList();
MailingList(const MailingList&);
~MailingList();

const MailingList& operator= (const MailingList&);

// Add a new contact to the list

// Does the list contain this person?
bool contains (const Name&) const;

// Find the contact
const Contact& getContact (const Name& nm) const;
//pre: contains(nm)

// Remove one matching contact
void removeContact (const Contact&);
void removeContact (const Name&);

// combine two mailing lists
void merge (const MailingList& otherList);

// How many contacts in list?
int size() const;

bool operator== (const MailingList& right) const;
bool operator< (const MailingList& right) const;

private:

struct ML_Node {
Contact contact;
ML_Node* next;

ML_Node (const Contact& c, ML_Node* nxt)
: contact(c), next(nxt)
{}
};

ML_Node* first;
ML_Node* last;
int theSize;

// helper functions
void clear();
void remove (ML_Node* previous, ML_Node* current);

friend std::ostream& operator<< (std::ostream& out, const MailingList& addr);
};

// print list, sorted by Contact
std::ostream& operator<< (std::ostream& out, const MailingList& list);

#endif


Look at our MailingList class.

Question: What are the mutators? What are the accessors?

Unit Test Skeleton

Here is the basic skeleton for our test suite.

skeleton.cpp
#define BOOST_TEST_MODULE MailingList test

#include "mailinglist.h"

#include <string>
#include <sstream>
#include <vector>

#include <boost/test/included/unit_test.hpp>

using namespace std;


Now we start going through the mutators.

### 2.1.1 Testing the Constructors

The first mutators we listed were the two constructors. Let’s start with the simpler of these.

Apply The Mutator

BOOST_AUTO_TEST_CASE ( constructor ) {
MailingList ml;
⋮



First we start by applying the mutator.

Apply the Accessors to the Mutated Object

Then we go down the list of accessors and ask what we expect each one to return.

BOOST_AUTO_TEST_CASE ( constructor ) {
MailingList ml;
BOOST_CHECK_EQUAL (0, ml.size());
BOOST_CHECK (!ml.contains("Jones"));
BOOST_CHECK_EQUAL (ml, MailingList());
BOOST_CHECK (!(ml < MailingList()));
}


1. It’s pretty clear, for example, that the size() of the list will be 0
2. contains() would always return false
3. The EQUAL test checks the operator==
4. We check for consistency of operator<

We can’t check the accessors

• getContact

• can’t satisfy the pre-condition, and
• operator<<

• behavior unspecified

Testing the Copy Constructor

testCons1.cpp
BOOST_FIXTURE_TEST_CASE ( copyConstructor,  MailingListTests) {
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);             ➀
MailingList ml1 = ml0; // copy constructor  ➁
shouldBeEqual(ml0, ml1);                    ➂
}
{
// Minimal check for deep copy - changing one should not affect the other
MailingList ml0 = generate(2);
MailingList ml1 = ml0; // copy constructor
BOOST_CHECK_EQUAL (2, ml0.size());
BOOST_CHECK_NE (ml0, ml1);
}
}

• We use the generate-and-check pattern.

• Here we invoke the function under test.

• The check function - we’ll look at this in a moment

• The second half of this is more subtle. I expect that copies are distinct. Once a copy is made, updating one object should not change the other.

A failure suggests that we have done an improper shallow copy.

The Check Function for Copying

shouldBeEqual.cpp
// The fixture for testing class MailingList.
class MailingListTests  {
public:
Contact jones;
vector<Contact> contacts;

MailingListTests() {
⋮

void shouldBeEqual (const MailingList& ml0, const MailingList& ml1) const
{
BOOST_CHECK_EQUAL (ml1.size(), ml0.size());  ➀
for (int i = 0; i < ml0.size(); ++i)         ➁
{
BOOST_CHECK_EQUAL(ml1.contains(contacts[i].getName()), ml0.contains(contacts[i].getName()));
if (ml1.contains(contacts[i].getName()))
BOOST_CHECK_EQUAL(ml1.getContact(contacts[i].getName()), ml0.getContact(contacts[i].getName()));
}

BOOST_CHECK_EQUAL (ml0, ml1);    ➂
BOOST_CHECK (!(ml0 < ml1));
BOOST_CHECK (!(ml1 < ml0));

ostringstream out0;              ➂
out0 << ml0;
ostringstream out1;
out1 << ml1;

BOOST_CHECK_EQUAL (out0.str(), out1.str());

}

};

• Sizes should be equal

• Boths lists should agree as to what they contain.

• Relational ops - should be equal and not less/greater

• Whatever they print, it should match

### 2.1.2 Testing the Assignment Operator

testAsst.cpp
BOOST_FIXTURE_TEST_CASE ( assignment,  MailingListTests) {
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml1;
MailingList ml2 = (ml1 = ml0); // assignment   ➀
shouldBeEqual(ml0, ml1);                       ➁
shouldBeEqual(ml0, ml2);  // assignment returns a value
}
{
// Minimal check for deep copy - changing one should not affect the other
MailingList ml0 = generate(2);
MailingList ml1;
ml1 = ml0; // copy constructor
BOOST_CHECK_EQUAL (2, ml0.size());
BOOST_CHECK_NE (ml0, ml1);
}
}

• Very similar to testing the copy constructor

• But remember that assignment operators not only change the value on the left, but also return a copy of the assigned value.

• We need to check both.
• Fortunately, we can re-use the check function developed for the copy constructor.

BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) {
for (unsigned sel = 0; sel < contacts.size(); ++sel)
{
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml (ml0);
{
BOOST_CHECK_EQUAL (ml.size(), (int)sz);
BOOST_CHECK_EQUAL (ml0, ml);
BOOST_CHECK (!(ml0 < ml));
BOOST_CHECK (!(ml < ml0));
}
else
{
BOOST_CHECK_EQUAL (ml.size(), (int)sz+1);
BOOST_CHECK_NE (ml0, ml);
BOOST_CHECK ((ml0 < ml) || (ml < ml0));    // one must be true
BOOST_CHECK (!((ml0 < ml) && (ml < ml0))); // ...but not both
}
ostringstream out;
out << ml;
}
}
}

• Similar to version developed earlier, but now we go systematically through all the accessors
• Needed to add checks on relational operators and output operator

### 2.1.4 And so on…

We continue in this manner until done.

fullTest.cpp
#define BOOST_TEST_MODULE MailingList test

#include "mailinglist.h"

#include <string>
#include <sstream>
#include <vector>

#include <boost/test/included/unit_test.hpp>
//#include <boost/test/unit_test.hpp>

using namespace std;

// The fixture for testing class MailingList.
class MailingListTests  {
public:
Contact jones;
vector<Contact> contacts;

MailingListTests() {
jones = Contact("Jones", "21 Penn. Ave.");
contacts.clear();
contacts.push_back (Contact ("Muffin Man", "Drury Ln."));
contacts.push_back (Contact ("Holmes", "221B Baker St."));
contacts.push_back(jones);
contacts.push_back (Contact ("Wolfe", "454 W. 35th St."));
}

~MailingListTests() {
}

MailingList generate(int n) const
{
MailingList m;
for (int i = 0; i < n; ++i)
return m;
}

void shouldBeEqual (const MailingList& ml0, const MailingList& ml1) const
{
BOOST_CHECK_EQUAL (ml1.size(), ml0.size());
for (int i = 0; i < ml0.size(); ++i)
{
BOOST_CHECK_EQUAL(ml1.contains(contacts[i].getName()), ml0.contains(contacts[i].getName()));
if (ml1.contains(contacts[i].getName()))
BOOST_CHECK_EQUAL(ml1.getContact(contacts[i].getName()), ml0.getContact(contacts[i].getName()));
}

BOOST_CHECK_EQUAL (ml0, ml1);
BOOST_CHECK (!(ml0 < ml1));
BOOST_CHECK (!(ml1 < ml0));

ostringstream out0;
out0 << ml0;
ostringstream out1;
out1 << ml1;

BOOST_CHECK_EQUAL (out0.str(), out1.str());

}

};

BOOST_AUTO_TEST_CASE ( constructor ) {
MailingList ml;
BOOST_CHECK_EQUAL (0, ml.size());
BOOST_CHECK (!ml.contains("Jones"));
BOOST_CHECK_EQUAL (ml, MailingList());
BOOST_CHECK (!(ml < MailingList()));
}

BOOST_FIXTURE_TEST_CASE ( copyConstructor,  MailingListTests) {
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml1 = ml0; // copy constructor
shouldBeEqual(ml0, ml1);
}
{
// Minimal check for deep copy - changing one should not affect the other
MailingList ml0 = generate(2);
MailingList ml1 = ml0; // copy constructor
BOOST_CHECK_EQUAL (2, ml0.size());
BOOST_CHECK_NE (ml0, ml1);
}
}

BOOST_FIXTURE_TEST_CASE ( assignment,  MailingListTests) {
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml1;
MailingList ml2 = (ml1 = ml0); // assignment
shouldBeEqual(ml0, ml1);
shouldBeEqual(ml0, ml2);  // assignment returns a value
}
{
// Minimal check for deep copy - changing one should not affect the other
MailingList ml0 = generate(2);
MailingList ml1;
ml1 = ml0; // copy constructor
BOOST_CHECK_EQUAL (2, ml0.size());
BOOST_CHECK_NE (ml0, ml1);
}
}

for (unsigned sel = 0; sel < contacts.size(); ++sel)
{
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml (ml0);
{
BOOST_CHECK_EQUAL (ml.size(), (int)sz);
BOOST_CHECK_EQUAL (ml0, ml);
BOOST_CHECK (!(ml0 < ml));
BOOST_CHECK (!(ml < ml0));
}
else
{
BOOST_CHECK_EQUAL (ml.size(), (int)sz+1);
BOOST_CHECK_NE (ml0, ml);
BOOST_CHECK ((ml0 < ml) || (ml < ml0));    // one must be true
BOOST_CHECK (!((ml0 < ml) && (ml < ml0))); // ...but not both
}
ostringstream out;
out << ml;
}
}
}

BOOST_FIXTURE_TEST_CASE (removeContact, MailingListTests) {
for (unsigned sel = 0; sel < contacts.size(); ++sel)
{
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml (ml0);
{
BOOST_CHECK_EQUAL (ml.size(), (int)sz);
BOOST_CHECK_EQUAL (ml0, ml);
BOOST_CHECK (!(ml0 < ml));
BOOST_CHECK (!(ml < ml0));
}
else
{
BOOST_CHECK_EQUAL (ml.size(), (int)sz-1);
BOOST_CHECK_NE (ml0, ml);
BOOST_CHECK ((ml0 < ml) || (ml < ml0));    // one must be true
BOOST_CHECK (!((ml0 < ml) && (ml < ml0))); // ...but not both
}
ostringstream out;
out << ml;
string outs = out.str();
}
}
}

BOOST_FIXTURE_TEST_CASE (removeContactByName, MailingListTests) {
for (unsigned sel = 0; sel < contacts.size(); ++sel)
{
for (unsigned sz = 0; sz < contacts.size(); ++sz)
{
MailingList ml0 = generate(sz);
MailingList ml (ml0);
{
BOOST_CHECK_EQUAL (ml.size(), (int)sz);
BOOST_CHECK_EQUAL (ml0, ml);
BOOST_CHECK (!(ml0 < ml));
BOOST_CHECK (!(ml < ml0));
}
else
{
BOOST_CHECK_EQUAL (ml.size(), (int)sz-1);
BOOST_CHECK_NE (ml0, ml);
BOOST_CHECK ((ml0 < ml) || (ml < ml0));    // one must be true
BOOST_CHECK (!((ml0 < ml) && (ml < ml0))); // ...but not both
}
ostringstream out;
out << ml;
}
}
}

BOOST_FIXTURE_TEST_CASE ( merging,  MailingListTests) {

MailingList ml0;

MailingList ml1;

ml1.merge(ml0);
shouldBeEqual(ml1, generate(4));
}


• Yes, BTW, I did discover quite a few bugs in my MailingList code while putting this together.

## 2.2 Testing for Pointer/Memory Faults

In our example, we did not write explicit tests for the destructor. Partly that’s because the destructor is already being invoked a lot - every time we exit a function or a {} statement block that declares a local variable. So we have some good reason to hope that the destructor has been well exercised already.

Also, most of the effects of a destructor are hard to see directly. How do you check to see if memory has been successfully deleted?

Most destructors simply delete pointers, and pointer issues are particularly difficult to test and debug.

• One way that pointer/memory use can be tested is to employ specialized tools that replace the system functions for allocating and deallocating memory (alloc and free) with special versions that
• keep track of all blocks of memory that have been allocated,

• whether those blocks have been freed,

• whether any block has been freed more than once, etc.

Tools for Testing Pointer/Memory Faults

• Purify is a well known commercial package for this purpose, but is not cheap.

• I have had good results with a free tool called LeakTracer.

## 2.3 Case Study 1: Unit Test of NumericLiteral

One of the stories in our spreadsheet example involves numeric literals, the component of an expression that represents a constant numeric value like “3.14159” or “42”.

As an API user I would like to add a numeric literal to a cell in a spreadsheet.

Numeric literals are a kind of Expression, a central idea in the spreadsheet. This story offers an opportunity for a look at a realistic set of tests.

### 2.3.1 Expressions

Expression.java
package edu.odu.cs.espreadsheet.expressions;

/**
* Expressions can be thought of as trees.  Each non-leaf node of the tree
* contains an operator, and the children of that node are the subexpressions
* (operands) that the operator operates upon.  Constants, cell references,
* and the like form the leaves of the tree.
*
* For example, the expression (a2 + 2) * c26 is equivalent to the tree:
*
*              *
*             / \
*            +   c26
*           / \
*         a2   2
*
* @author zeil
*
*/
public abstract class Expression implements Cloneable
{

/**
*  How many operands does this expression node have?
*
* @return # of operands required by this operator
*/
public abstract int arity();

/**
*  Get the k_th operand
* @param k operand number
* @return k_th operand if 0 < k < arity()
* @throws IndexOutOfBoundsException if k is outside of those bounds
*/
public abstract Expression operand(int k) throws IndexOutOfBoundsException;

/**
*  Evaluate this expression, using the provided spreadsheet to resolve
*  any cell referneces.
*
* @param usingSheet spreadsheet form which to obtain values of
* cells referenced by this expression
*
* @return value of the expression or null if the cell is empty
*/

/**
* Copy this expression (deep copy), altering any cell references
* by the indicated offsets except where the row or column is "fixed"
* by a preceding $. E.g., if e is 2*D4+C$2/$A$1, then
* e.copy(1,2) is 2*E6+D$2/$A$1, e.copy(-1,4) is 2*C8+B$2/$A$1
*
* @param colOffset number of columns to offset this copy
* @param rowOffset number of rows to offset this copy
* @return a copy of this expression, suitable for placing into
*           a cell (ColOffSet,rowOffset) away from its current position.
*
*/
public abstract Expression clone (int colOffset, int rowOffset);

/**
* Copy this expression.
*/
@Override
public Expression clone ()
{
return clone(0,0);
}

/**
* Attempt to convert the given string into an expression.
* @param in
* @return
*/
public static Expression parse (String in) throws ExpressionParseError
{
try {
parser p = new parser(new ExpressionScanner(new StringReader(in)));
Expression expr = (Expression)p.parse().value;
return expr;
} catch (Exception ex) {
throw new ExpressionParseError("Cannnot parse " + in);
}
}

@Override
public String toString ()
{
⋮
}

@Override
public abstract boolean equals (Object obj);

@Override
public abstract int hashCode ();

// The following control how the expression gets printed by
// the default implementation of toString

/**
* If true, print in inline form.
* If false, print as functionName(comma-separated-list).
*
* @return indication of whether to print in inline form.
*
*/
public abstract boolean isInline();

/**
* Parentheses are placed around an expression whenever its precedence
* is lower than the precedence of an operator (expression) applied to it.
* E.g., * has higher precedence than +, so we print 3*(a1+1) but not
* (3*a1)+1
*
* @return precedence of this operator
*/
public abstract int precedence();

/**
* Returns the name of the operator for printing purposes.
* For constants/literals, this is the string version of the constant value.
*
* @return the name of the operator for printing purposes.
*/
public abstract String getOperator();

}


A lot of this design was lifted from an earlier project of mine.

• Expression is an abstract class – we never expect to see any actual objects of this type.
• But we will have lots of interesting subclasses (one per operator) that will have “real” instances.

### 2.3.2 Numeric Literals

The NumericLiteral class is a subclass of Expression and needs to supply function bodies for all of the unimplemented functions declared in Expression.

NumericLiteral.java
package edu.odu.cs.espreadsheet.expressions;

/**
* This class represents numeric constants appearing within an expression.
*
* @author zeil
*
*/
public
class NumericLiteral extends Expression
{
⋮
/**
* Create a default numeric literal. Equivalent to NumericLiteral("0");
*/
public NumericLiteral ()
{
⋮
}

public NumericLiteral (String lit)
{
⋮
}

/**
*  How many operands does this expression node have?
*
* @return # of operands required by this operator
*/
public int arity()                    {return 0;}

/**
*  Get the k_th operand
* @param k operand number
* @return k_th operand if 0 < k < arity()
* @throws IndexOutOfBoundsException if k is outside of those bounds
*/
public Expression operand(int k)
{
⋮
}

/**
*  Evaluate this expression, using the provided spreadsheet to resolve
*  any cell references.
*
* @param usingSheet spreadsheet form which to obtain values of
* cells referenced by this expression
*
* @return value of the expression or null if the cell is empty
*/
{
⋮
}

/**
* Copy this expression (deep copy), altering any cell references
* by the indicated offsets except where the row or column is "fixed"
* by a preceding $. E.g., if e is 2*D4+C$2/$A$1, then
* e.copy(1,2) is 2*E6+D$2/$A$1, e.copy(-1,4) is 2*C8+B$2/$A$1
*
* @param colOffset number of columns to offset this copy
* @param rowOffset number of rows to offset this copy
* @return a copy of this expression, suitable for placing into
*           a cell (ColOffSet,rowOffset) away from its current position.
*
*/
@Override
public NumericLiteral clone (int colOffset, int rowOffset)
{
⋮
}

// The following control how the expression gets printed by
// the default implementation of put(ostream&)

/**
* Attempt to convert the given string into an expression.
* @param in
* @return
*/
public boolean isInline()      {return true;}

/**
* If true, print in inline form.
* If false, print as functionName(comma-separated-list).
*
* @return indication of whether to print in inline form.
*
*/
public int precedence()        {return 1000;}

/**
* Returns the name of the operator for printing purposes.
* For constants/literals, this is the string version of the constant value.
*
* @return the name of the operator for printing purposes.
*/
public String getOperator()    {return literal;}

@Override
public boolean equals(Object obj) {
⋮
}

@Override
public int hashCode() {
⋮
}

}


I’m not showing the actual implementaiton (function bodies) here because

• It’s not relevant to the design of the tests.
• In fact, as we’ll see very shortly, I usually write the tests before I implement the ADT. So, in practice, those function bodies would be empty when I do this.

### 2.3.3 Writing the Unit tests

NumericLiteral desperately needs a good set of unit test cases because we need to be sure that we have covered all of the various input format possibilities:

TestNumericLiteral.java
/**
*
*/

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;

import org.junit.Test;

/**
* @author zeil
*
*/
public class TestNumericLiteral {

public final NumericLiteral nl0 = new NumericLiteral("2.345");

/**
*/
@Test
public final void testNumericLiteral() {
NumericLiteral nl = new NumericLiteral();  ➀
assertEquals(0,  nl.arity());              ➁
try {
nl.operand(0);
fail("Expected IndexOutOfBoundsException");
} catch (IndexOutOfBoundsException ex) {
// OK
}
assertEquals (0.0, nl.evaluate(ss).toDouble(), 1.0E-10);
assertEquals ("0", nl.toString());
assertEquals (true, nl.isInline());
assertTrue (nl.precedence() > 100);
assertEquals ("0", nl.getOperator());
assertFalse (nl0.equals(nl));
}

/**
*/
@Test
public final void testNumericLiteralString() {
NumericLiteral nl = new NumericLiteral("2.3450");
assertEquals(0,  nl.arity());
try {
nl.operand(0);
fail("Expected IndexOutOfBoundsException");
} catch (IndexOutOfBoundsException ex) {
// OK
}
assertEquals (2.345, nl.evaluate(ss).toDouble(), 1.0E-10);
assertEquals ("2.3450", nl.toString());
assertEquals (true, nl.isInline());
assertTrue (nl.precedence() > 100);
assertEquals ("2.3450", nl.getOperator());
assertFalse (nl0.equals(nl));
}

/**
*/
@Test
public final void testCloneIntInt() {
NumericLiteral nl = nl0.clone(1,1);
assertEquals(0,  nl.arity());
try {
nl.operand(0);
fail("Expected IndexOutOfBoundsException");
} catch (IndexOutOfBoundsException ex) {
// OK
}
assertEquals (nl0.evaluate(ss), nl.evaluate(ss));
assertEquals (nl0.toString(), nl.toString());
assertEquals (nl0.isInline(), nl.isInline());
assertEquals (nl0.precedence(), nl.precedence());
assertEquals (nl0.getOperator(), nl.getOperator());
assertEquals (nl0, nl);
}

/**
*/
@Test
public final void testClone() {
NumericLiteral nl = (NumericLiteral)nl0.clone();
assertEquals(0,  nl.arity());
try {
nl.operand(0);
fail("Expected IndexOutOfBoundsException");
} catch (IndexOutOfBoundsException ex) {
// OK
}
assertEquals (nl0.evaluate(ss), nl.evaluate(ss));
assertEquals (nl0.toString(), nl.toString());
assertEquals (nl0.isInline(), nl.isInline());
assertEquals (nl0.precedence(), nl.precedence());
assertEquals (nl0.getOperator(), nl.getOperator());
assertEquals (nl0, nl);
}

/**
* @throws ExpressionParseError
*/
@Test
public final void testNumericParse() throws ExpressionParseError {
String input = "12.34";                          ➂
Expression e = Expression.parse(input);
assertTrue (e instanceof NumericLiteral);
assertEquals (new NumericLiteral(input), e);

input = "1.2E25";
e = Expression.parse(input);
assertTrue (e instanceof NumericLiteral);
NumericLiteral nl = (NumericLiteral)e;
assertEquals (1.2E25, new Double(nl.getOperator()).doubleValue(), 1.0E20);

input = "1.2E-5";
e = Expression.parse(input);
assertTrue (e instanceof NumericLiteral);
nl = (NumericLiteral)e;
assertEquals (1.2E-5, new Double(nl.getOperator()).doubleValue(), 1.0E-10);

}

}

• These tests are good examples of our mutator/accessor approach to devising unit tests.

• The mutators for this class are the two constructors and the two clone functions.

• The accessors are arity(), operand(i), evaluate(ss), isInLine(), precedence(), getOperator(), ’equals(…), hashCode(), andtoString().

In this test we invoke one of the mutators (a constructor)

• Then we run through each of the accessors in turn.

This pattern is repeated for each of the mutators.

• A more subtle “mutator”: Expression has a static function for parsing strings. It is, more or less, a mutator for all subclasses of expression. So we test it here as well.
• Could have made this part of a unit test for Expression, but that would have led to a single test that had to know about every subclass of Expression, which would have grown quickly unwieldy.

## 2.4 Is this Overkill?

Many, perhaps most, ADTs provide a small number of operations that manipulate the data in a non-trivial fasion, and a lage number of get/set attribute functions that, conceptually at least, simply store and retrieve a private data member.

For example, suppose we wanted to test the following ADT:

package mailinglist;
/**
* A contact is a name and address.
* <p>
* For the purpose of this example, I have simplified matters
* a bit by making both of these components simple strings.
* In practice, we would expect Address, at least, to be a
* more structured type.
*
* @author zeil
*
*/
public class Address implements Cloneable {

private String name;
private String city;
private String state;
private String zipCode;

/**
* Create an address with all empty fields.
*
*/
{
name = "";
city = "";
state = "";
zipCode = "";
}

/**
*/
String state, String zip)
{
name = nm;
this.city = city;
this.state = state;
zipCode = zip;
}

/**
* @return the theName
*/
public String getName() {
return name;
}

/**
* @param theName the theName to set
*/
public void setName(String theName) {
this.name = theName;
}

/**
*/
}

/**
*/
}

/**
* @return the city
*/
public String getCity() {
return city;
}

/**
* @param city the city to set
*/
public void setCity(String city) {
this.city = city;
}

/**
* @return the state
*/
public String getState() {
return state;
}

/**
* @param state the state to set
*/
public void setState(String state) {
this.state = state;
}

/**
* @return the zipCode
*/
public String getZipCode() {
return zipCode;
}

/**
* @param zipCode the zipCode to set
*/
public void setZipCode(String zipCode) {
this.zipCode = zipCode;
}

/**
* True if the names and addresses are equal
*/
public boolean equals (Object right)
{
return name.equals(r.name)
&& city.equals(r.city)
&& state.equals(r.state)
&& zipCode.equals(r.zipCode);
}

public int hashCode ()
{
return name.hashCode() + 3 * streetAddress.hashCode()
+ 5 * city.hashCode()
+ 7 * state.hashCode()
+ 11 * zipCode.hashCode();
}

public String toString()
{
return name + ": " + streetAddress + ": "
+ city + ", " + state + " " + zipCode;
}

public Object clone()
{
state, zipCode);
}

}


You can see that, in this case, the bulk of the operations are simple gets and sets. There are a few operations that work, in some sense, on the whole ADT, mainly for the purpose of output and comparisons.

If we were to test this, we would identify the mutators (the two constructors, the five set… functions, and the clone function) and accessors (the five get… functions and the toString, equals, and hashCode functions). Then we would create a test function for each mutator. For example, for the setCity function, we might write:

public class TestAddress {

final private String name0 = "John Doe";
final private String street0 = "221B Baker St.";
final private String city0 = "Podunk";
final private String state0 = "IL";
final private String zip0 = "01010";

⋮

@Test
public final void testSetCity() {

String city1 = "Norfolk";   ➀

}


This follows a pattern that should be increasingly familiar:

• Set up the data values that we will need.
• Invoke the mutator being tested.
• Evaluate and test each of the accessor functions on the mutated ADT value.

You can see that four out of the first five assertions actually assert that this value was unaffected by the mutator. Some students look at this and wonder why we bother. Isn’t that a lot of wasted code? It doesn’t seem to really be relevant to the mutator function (setCity) that we are testing in this function. And when you consider that there will be similar “waste” in each of the other functions for testing the other mutator functions, this can seem excessive.

But there are a number of reasons why these “does not change” assertions are worth performing:

• Our intuition is that a working implementation of setCity won’t have any effect on the street address, zip code, etc.. But, if we knew that the code was working, we wouldn’t be testing it!

True story: I often write an ADT like this by typing out the first pair of get and set functions (e.g., getName and setName), then copying and pasting those multiple times, and then using my editor’s search-and-replace function to alter the attribute names in the subsequent pairs.

On more than one occasion, I have forgotten to change the name of the data member being retrieved by a get/set pair, winding up with a bug something like like this:

public String getCity() {
return name;
}

public void setCity(String city) {
this.name = name;
}


This bug was detected by the "getName() does not change when setCity is called) assertion in the testSetCity test function.

• Our intuition that the behavior of “getCity” has no bearing on the testing of “setCity” is based on a misunderstanding of what the test functions are doing. Although we allocate one test function per mutator, each such function is not merely testing its associated mutator. It is testing both that mutator and all the accessors.

The collected tests of all of accessor functions are distributed over the entire set of test functions. If we eliminated all checks of an accessor getX but the one in the testSetX function, that accessor would wind up being woefully under-tested.

On the other hand, what would be excessive would be to devote separate test functions to both mutators and accessors (depeite the fact that this is what Eclipse volunteers to do if you use its “New … Junit Test Case” helper). The only way to test an accessor is to use some mutator to set the ADT value, and we’re already doing that the the test functions for mutators.

• Our intuition that the behavior of a getX accessor is independent of a setY mutator for a different attribute is colored by our assumption that these get/set functions are doing nothing but retrieving simple data members.

But that’s not always the case, and, even if that’s how the ADT is implemented now, it might not be implemented that way in the future.

• If the program is distributed, so that address values are actually being transferred over the network, we will likely receive and sometimes store addresses in a serialized form, essentially a single string.

In that case, we would need to tease out the individual field values like cities, states, etc., and the possibility of “cross-talk” among the fields due to buggy code becomes significant.

• Similar serialized forms are common if we have a program that mixes code written in different programming langauges.

• A similar problem can arise if the actual Address values need to be stored in and retrieved from a database. Perhaps the databse is set up to combine, for example, the city and state in a single field.

These are Black-box tests, and we want them to remain valid even if the implementation changes.

In general, it’s always a bit dangerous to argue that tests are unnecessary based on our intuition about how the code will behave when it runs correctly. Our choice of tests really have to be more informed by the possibilities of how the code might misbehave when it has bugs.

Furthermore, these “extra” assertions are not really all that oppressive. Most of them, written once, can be copied and pasted into the next test function. For example, the testSetState function could be:

@Test
public final void testSetState() {

String state1 = "VA";

}


The highlighted portions are the only changes from the earlier testSetCity function.

A final comment on the “Is this excessive?” question:

• These tests are not, by any measure, a perfect or full set of tests that will detect all bugs. Having done this first pass, we would still want to look at the tests from the perspective of good black-box testing, possibly adjusting the vlues used for inputs or maybe doing more than one invocation of some mutators.

• That said, this is still more testing and better testing than most programmers perform if they don’t let themselves by guided by some approach, like this mutator/accessor approach, for being thorough and systematic in their test design.

• Finally, many students’ perception of what makes testing difficult and time-consuming is rooted in the practice of running tests “manually” and inspecting all outputs by eye. No wonder, then, that even running a small test set seems like drudgery.

But we’ve just automated the checking part of testing.

And with Eclipse and other IDE’s we can run those tests with single mouse action. And, shortly, we’ll integrate the tests into our automated build process so that launching tests does not even require that small effort.

# 3 Be Independent:

• A true unit test should not depend on other modules of the program
• The *unit frameworks we have looked at provide drivers.
• But scaffolding can also involve stubs.
• Which are often harder to write
• We could
• Wait until integration test, or

• Prepare stubs for unit testing and accept that they can only be rough approximations, or

• Create stubs for unit testing and replace them later when the lower-level modules are ready.

Is there a virtue to independence?

• Integration tests are not independent.

• Can lead to slower testing.
• Can complicate debugging
• But are certainly more realistic than unit tests with simple stubs
• Increasingly, *`unit frameworks are providing support for stubbing.

• We’ll cover this in the next lesson on mock objects.