Testing ADTs in C++

Steven J Zeil

Last modified: Feb 27, 2014

Contents:
1. Be Smart: Generate and Check
2. Be Thorough: Mutators and Accessors
2.1 Example: Unit Testing of the MailingList Class
2.2 Testing for Pointer/Memory Faults
3. Be Independent:
4. Be Pro-active: Write the Tests First

It would be nice if every new ADT we wrote worked correctly the first time it compiled properly, but the real world just doesn’t work that way.

One advantage of organizing your code into a collection of ADTs is that ADTs provide, not only a convenient way to package up the functionality of your code, but also a convenient basis for testing.

The *Unit test frameworks give us a powerful tool for unit testing. But we still need to pick the tests.

1. Be Smart: Generate and Check

Testing addContact

In our earlier test for addContact, we weren’t particularly thorough:

TEST_F (MailingListTests, addContact) {
  mlist.addContact (jones);
  EXPECT_TRUE (mlist.contains("Jones"));
  EXPECT_EQ ("Jones", mlist.getContact("Jones").getName());
  EXPECT_EQ (4, ml.size());
}

Adding Variety

Some of our concerns could be addressed by adding tests but varying the parameters:

TEST_F (MailingListTests, addExistingContact) {
  mlist.addContact (baker);
  EXPECT_TRUE (mlist.contains("Baker"));
  EXPECT_EQ ("Baker", mlist.getContact("Baker").getName());
  EXPECT_EQ (3, ml.size());
}

Generate and Check

A useful design pattern for producing well-varied tests is

for v: varieties of ADT {
   ADT x = generate(v);
   result = applyFunctionBeingTested(x);
   check (x, result); 
}


   ADT x (constructor1);
   result = applyFunctionBeingTested(x);
   check (x, result); 

   ADT x2 (constructor2,  ... );
   result = applyFunctionBeingTested(x2);
   check (x2, result);

Example: addContact

A more elaborate fixture will aid in generating mailing lists of different sizes:

fixture.cpp 
// The fixture for testing class MailingList.
class MailingListTests  {
public:
  Contact jones;
  vector<Contact> contacts;
  
  MailingListTests() {
    jones = Contact("Jones", "21 Penn. Ave.");
    contacts.clear();
    contacts.push_back (Contact ("Baker", "Drury Ln."));
    contacts.push_back (Contact ("Holmes", "221B Baker St."));
    contacts.push_back(jones);
    contacts.push_back (Contact ("Wolfe", "454 W. 35th St."));
  }

  ~MailingListTests() {
  }


  MailingList generate(int n) const
  {
    MailingList m;
    for (int i = 0; i < n; ++i)
      m.addContact(contacts[i]);
    return m;
  }

};

Example: addContact - many sizes

testAdd1.cpp 
BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) {
  for (unsigned sz = 0; sz < contacts.size(); ++sz)
    {
      MailingList ml0 = generate(sz);
      MailingList ml (ml0);
      bool alreadyContained = ml.contains("Jones");
      ml.addContact (jones);
      BOOST_CHECK (ml.contains("Jones"));
      BOOST_CHECK_EQUAL ("Jones", ml.getContact("Jones").getName());
      if (alreadyContained)
      BOOST_CHECK_EQUAL (ml.size(), sz);
      else
    BOOST_CHECK_EQUAL (ml.size(), sz+1);
    }
}

Example: addContact - ordering

testAdd2.cpp 
BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) {
  for (unsigned sel = 0; sel < contacts.size(); ++sel)
    {
      Contact& toAdd = contacts[sel];
      const Name& nameToAdd = contacts[sel];
      for (unsigned sz = 0; sz < contacts.size(); ++sz)
        {
          MailingList ml0 = generate(sz);
          MailingList ml (ml0);
          bool alreadyContained = ml.contains(nameToAdd);
          ml.addContact (toAdd);
          BOOST_CHECK (ml.contains(nameToAdd));
          BOOST_CHECK_EQUAL (nameToAdd, ml.getContact(nameToAdd).getName());
          if (alreadyContained)
            BOOST_CHECK_EQUAL (ml.size(), sz);
          else
            BOOST_CHECK_EQUAL (ml.size(), sz+1);
        }
    }
}

2. Be Thorough: Mutators and Accessors

OK, let’s say we want to design a unit test suite for our MailingList ADT.

Just staring at the ADT interface, where do we start? The criteria we suggested earlier (typical values, extremal values, special values) may be of help, but it’s far from obvious how to apply those ideas.

There is an organized way to approach this test design. It starts with the recognition that the member functions of an ADT can usually be divided into two groups - the mutators and the accessors.

Mutator functions are the functions that alter the value of an object. These include constructors and assignment operators.

Accessor functions are the functions that “look at” the value of an object but do not change it. Some functions may both alter an object and return part of its value - we’ll have to wing it a bit with those.

Mutators and Accessors

To test an ADT, divide the public interface into

Organizing ADT Tests

The basic procedure for writing an ADT unit test is to

  1. Consider each mutator in turn.

  2. Write a test that begins by applying that mutator function.

  3. Then consider how that mutator will have affected the results of each accessor.

  4. Write assertions to test those effects.

Commonly, each mutator will be tested in a separate function.

2.1 Example: Unit Testing of the MailingList Class

Example: Unit Testing of the MailingList Class

mailinglist.h 
#ifndef MAILINGLIST_H
#define MAILINGLIST_H

#include <iostream>
#include <string>

#include "contact.h"

/**
   A collection of names and addresses
*/
class MailingList
{
public:
  MailingList();
  MailingList(const MailingList&);
  ~MailingList();

  const MailingList& operator= (const MailingList&);

  // Add a new contact to the list
  void addContact (const Contact& contact);

  // Does the list contain this person?
  bool contains (const Name&) const;

  // Find the contact
  const Contact& getContact (const Name& nm) const;
  //pre: contains(nm)

  // Remove one matching contact
  void removeContact (const Contact&);
  void removeContact (const Name&);

  // combine two mailing lists
  void merge (const MailingList& otherList);

  // How many contacts in list?
  int size() const;


  bool operator== (const MailingList& right) const;
  bool operator< (const MailingList& right) const;

private:

  struct ML_Node {
    Contact contact;
    ML_Node* next;

    ML_Node (const Contact& c, ML_Node* nxt)
      : contact(c), next(nxt)
    {}
  };

  ML_Node* first;
  ML_Node* last;
  int theSize;

  // helper functions
  void clear();
  void remove (ML_Node* previous, ML_Node* current);

  friend std::ostream& operator<< (std::ostream& out, const MailingList& addr);
};

// print list, sorted by Contact
std::ostream& operator<< (std::ostream& out, const MailingList& list);


#endif

Look at our MailingList class.

Question: What are the mutators?

What are the accessors?

Answer:

Unit Test Skeleton

Here is the basic skeleton for our test suite.

skeleton.cpp 
#define BOOST_TEST_MODULE MailingList test

#include "mailinglist.h"

#include <string>
#include <sstream>
#include <vector>

#include <boost/test/included/unit_test.hpp>

using namespace std;

Now we start going through the mutators.

Testing the Constructors

The first mutators we listed were the two constructors. Let’s start with the simpler of these.

Apply The Mutator

BOOST_AUTO_TEST_CASE ( constructor ) {
    MailingList ml;
      ⋮

First we start by applying the mutator.

Apply the Accessors to the Mutated Object

Then we go down the list of accessors and ask what we expect each one to return. 

BOOST_AUTO_TEST_CASE ( constructor ) {
    MailingList ml;
    BOOST_CHECK_EQUAL (0, ml.size());
    BOOST_CHECK (!ml.contains("Jones"));
    BOOST_CHECK_EQUAL (ml, MailingList());
    BOOST_CHECK (!(ml < MailingList()));
}
  1. It’s pretty clear, for example, that the size() of the list will be 0
  2. contains() would always return false
  3. The EQUAL test checks the operator==
  4. We check for consistency of operator<

We can’t check the accessors

Testing the Copy Constructor

testCons1.cpp 
BOOST_FIXTURE_TEST_CASE ( copyConstructor,  MailingListTests) {
  for (unsigned sz = 0; sz < contacts.size(); ++sz)
    {
      MailingList ml0 = generate(sz);             ➊
      MailingList ml1 = ml0; // copy constructor  ➋
      shouldBeEqual(ml0, ml1);                    ➌
    }
  {
    // Minimal check for deep copy - changing one should not affect the other
    MailingList ml0 = generate(2);
    MailingList ml1 = ml0; // copy constructor
    ml1.addContact(jones);                        ➍
    BOOST_CHECK_EQUAL (2, ml0.size());
    BOOST_CHECK_NE (ml0, ml1);
  }
}

The Check Function for Copying

shouldBeEqual.cpp 
// The fixture for testing class MailingList.
class MailingListTests  {
 public:
  Contact jones;
  vector<Contact> contacts;
  
  MailingListTests() {
    ⋮
  
  void shouldBeEqual (const MailingList& ml0, const MailingList& ml1) const
  {
    BOOST_CHECK_EQUAL (ml1.size(), ml0.size());  ➊
    for (int i = 0; i < ml0.size(); ++i)         ➋
      {
    BOOST_CHECK_EQUAL(ml1.contains(contacts[i].getName()), ml0.contains(contacts[i].getName()));
    if (ml1.contains(contacts[i].getName()))
      BOOST_CHECK_EQUAL(ml1.getContact(contacts[i].getName()), ml0.getContact(contacts[i].getName()));
      }
    
    BOOST_CHECK_EQUAL (ml0, ml1);    ➌
    BOOST_CHECK (!(ml0 < ml1));
    BOOST_CHECK (!(ml1 < ml0));
    
    ostringstream out0;              ➌
    out0 << ml0;
    ostringstream out1;
    out1 << ml1;
    
    BOOST_CHECK_EQUAL (out0.str(), out1.str());
    
  }
  
};

Testing the Assignment Operator

testAsst.cpp 
BOOST_FIXTURE_TEST_CASE ( assignment,  MailingListTests) {
  for (unsigned sz = 0; sz < contacts.size(); ++sz)
    {
      MailingList ml0 = generate(sz);
      MailingList ml1;
      MailingList ml2 = (ml1 = ml0); // assignment   /*co1*/
      shouldBeEqual(ml0, ml1);                       /*co2*/
      shouldBeEqual(ml0, ml2);  // assignment returns a value
    }
  {
    // Minimal check for deep copy - changing one should not affect the other
    MailingList ml0 = generate(2);
    MailingList ml1;
    ml1 = ml0; // copy constructor
    ml1.addContact(jones);
    BOOST_CHECK_EQUAL (2, ml0.size());
    BOOST_CHECK_NE (ml0, ml1);
  }
}

Testing addContact

testAdd.cpp 
BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) {
  for (unsigned sel = 0; sel < contacts.size(); ++sel)
    {
      Contact& toAdd = contacts[sel];
      const Name& nameToAdd = toAdd.getName();
      for (unsigned sz = 0; sz < contacts.size(); ++sz)
        {
          MailingList ml0 = generate(sz);
          MailingList ml (ml0);
          bool alreadyContained = ml.contains(nameToAdd);
          ml.addContact (toAdd);
          BOOST_CHECK (ml.contains(nameToAdd));
          BOOST_CHECK_EQUAL (toAdd, ml.getContact(nameToAdd));
          if (alreadyContained)
            {
              BOOST_CHECK_EQUAL (ml.size(), (int)sz);
              BOOST_CHECK_EQUAL (ml0, ml);
              BOOST_CHECK (!(ml0 < ml));
              BOOST_CHECK (!(ml < ml0));
            }
          else
            {
              BOOST_CHECK_EQUAL (ml.size(), (int)sz+1);
              BOOST_CHECK_NE (ml0, ml);
              BOOST_CHECK ((ml0 < ml) || (ml < ml0));    // one must be true
              BOOST_CHECK (!((ml0 < ml) && (ml < ml0))); // ...but not both
            }
          ostringstream out;
          out << ml;
          BOOST_CHECK_NE (out.str().find(nameToAdd), string::npos);
        }
    }
}

And so on…

We continue in this manner until done.

fullTest.cpp 
#define BOOST_TEST_MODULE MailingList test

#include "mailinglist.h"

#include <string>
#include <sstream>
#include <vector>

#include <boost/test/included/unit_test.hpp>
//#define  BOOST_TEST_DYN_LINK 1
//#include <boost/test/unit_test.hpp>

using namespace std;

// The fixture for testing class MailingList.
class MailingListTests  {
 public:
  Contact jones;
  vector<Contact> contacts;
  
  MailingListTests() {
    jones = Contact("Jones", "21 Penn. Ave.");
    contacts.clear();
    contacts.push_back (Contact ("Muffin Man", "Drury Ln."));
    contacts.push_back (Contact ("Holmes", "221B Baker St."));
    contacts.push_back(jones);
    contacts.push_back (Contact ("Wolfe", "454 W. 35th St."));
  }
  
  ~MailingListTests() {
  }
  
  
  MailingList generate(int n) const
  {
    MailingList m;
    for (int i = 0; i < n; ++i)
      m.addContact(contacts[i]);
    return m;
  }
  
  void shouldBeEqual (const MailingList& ml0, const MailingList& ml1) const
  {
    BOOST_CHECK_EQUAL (ml1.size(), ml0.size());
    for (int i = 0; i < ml0.size(); ++i)
      {
        BOOST_CHECK_EQUAL(ml1.contains(contacts[i].getName()), ml0.contains(contacts[i].getName()));
        if (ml1.contains(contacts[i].getName()))
          BOOST_CHECK_EQUAL(ml1.getContact(contacts[i].getName()), ml0.getContact(contacts[i].getName()));
      }
    
    BOOST_CHECK_EQUAL (ml0, ml1);
    BOOST_CHECK (!(ml0 < ml1));
    BOOST_CHECK (!(ml1 < ml0));
    
    ostringstream out0;
    out0 << ml0;
    ostringstream out1;
    out1 << ml1;
    
    BOOST_CHECK_EQUAL (out0.str(), out1.str());
    
  }
  
};

BOOST_AUTO_TEST_CASE ( constructor ) {
  MailingList ml;
  BOOST_CHECK_EQUAL (0, ml.size());
  BOOST_CHECK (!ml.contains("Jones"));
  BOOST_CHECK_EQUAL (ml, MailingList());
  BOOST_CHECK (!(ml < MailingList()));
}


BOOST_FIXTURE_TEST_CASE ( copyConstructor,  MailingListTests) {
  for (unsigned sz = 0; sz < contacts.size(); ++sz)
    {
      MailingList ml0 = generate(sz);
      MailingList ml1 = ml0; // copy constructor
      shouldBeEqual(ml0, ml1);
    }
  {
    // Minimal check for deep copy - changing one should not affect the other
    MailingList ml0 = generate(2);
    MailingList ml1 = ml0; // copy constructor
    ml1.addContact(jones);
    BOOST_CHECK_EQUAL (2, ml0.size());
    BOOST_CHECK_NE (ml0, ml1);
  }
}

BOOST_FIXTURE_TEST_CASE ( assignment,  MailingListTests) {
  for (unsigned sz = 0; sz < contacts.size(); ++sz)
    {
      MailingList ml0 = generate(sz);
      MailingList ml1;
      MailingList ml2 = (ml1 = ml0); // assignment
      shouldBeEqual(ml0, ml1);
      shouldBeEqual(ml0, ml2);  // assignment returns a value
    }
  {
    // Minimal check for deep copy - changing one should not affect the other
    MailingList ml0 = generate(2);
    MailingList ml1;
    ml1 = ml0; // copy constructor
    ml1.addContact(jones);
    BOOST_CHECK_EQUAL (2, ml0.size());
    BOOST_CHECK_NE (ml0, ml1);
  }
}


BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) {
  for (unsigned sel = 0; sel < contacts.size(); ++sel)
    {
      Contact& toAdd = contacts[sel];
      const Name& nameToAdd = toAdd.getName();
      for (unsigned sz = 0; sz < contacts.size(); ++sz)
        {
          MailingList ml0 = generate(sz);
          MailingList ml (ml0);
          bool alreadyContained = ml.contains(nameToAdd);
          ml.addContact (toAdd);
          BOOST_CHECK (ml.contains(nameToAdd));
          BOOST_CHECK_EQUAL (toAdd, ml.getContact(nameToAdd));
          if (alreadyContained)
            {
              BOOST_CHECK_EQUAL (ml.size(), (int)sz);
              BOOST_CHECK_EQUAL (ml0, ml);
              BOOST_CHECK (!(ml0 < ml));
              BOOST_CHECK (!(ml < ml0));
            }
          else
            {
              BOOST_CHECK_EQUAL (ml.size(), (int)sz+1);
              BOOST_CHECK_NE (ml0, ml);
              BOOST_CHECK ((ml0 < ml) || (ml < ml0));    // one must be true
              BOOST_CHECK (!((ml0 < ml) && (ml < ml0))); // ...but not both
            }
          ostringstream out;
          out << ml;
          BOOST_CHECK_NE (out.str().find(nameToAdd), string::npos);
        }
    }
}

BOOST_FIXTURE_TEST_CASE (removeContact, MailingListTests) {
  for (unsigned sel = 0; sel < contacts.size(); ++sel)
    {
      Contact& toAdd = contacts[sel];
      const Name& nameToAdd = toAdd.getName();
      for (unsigned sz = 0; sz < contacts.size(); ++sz)
        {
          MailingList ml0 = generate(sz);
          MailingList ml (ml0);
          bool alreadyContained = ml.contains(nameToAdd);
          ml.removeContact (toAdd);
          BOOST_CHECK (!ml.contains(nameToAdd));
          if (!alreadyContained)
            {
              BOOST_CHECK_EQUAL (ml.size(), (int)sz);
              BOOST_CHECK_EQUAL (ml0, ml);
              BOOST_CHECK (!(ml0 < ml));
              BOOST_CHECK (!(ml < ml0));
            }
          else
            {
              BOOST_CHECK_EQUAL (ml.size(), (int)sz-1);
              BOOST_CHECK_NE (ml0, ml);
              BOOST_CHECK ((ml0 < ml) || (ml < ml0));    // one must be true
              BOOST_CHECK (!((ml0 < ml) && (ml < ml0))); // ...but not both
            }
          ostringstream out;
          out << ml;
          string outs = out.str();
          BOOST_CHECK_EQUAL (outs.find(nameToAdd), string::npos);
        }
    }
}

BOOST_FIXTURE_TEST_CASE (removeContactByName, MailingListTests) {
  for (unsigned sel = 0; sel < contacts.size(); ++sel)
    {
      Contact& toAdd = contacts[sel];
      const Name& nameToAdd = toAdd.getName();
      for (unsigned sz = 0; sz < contacts.size(); ++sz)
        {
          MailingList ml0 = generate(sz);
          MailingList ml (ml0);
          bool alreadyContained = ml.contains(nameToAdd);
          ml.removeContact (nameToAdd);
          BOOST_CHECK (!ml.contains(nameToAdd));
          if (!alreadyContained)
            {
              BOOST_CHECK_EQUAL (ml.size(), (int)sz);
              BOOST_CHECK_EQUAL (ml0, ml);
              BOOST_CHECK (!(ml0 < ml));
              BOOST_CHECK (!(ml < ml0));
            }
          else
            {
              BOOST_CHECK_EQUAL (ml.size(), (int)sz-1);
              BOOST_CHECK_NE (ml0, ml);
              BOOST_CHECK ((ml0 < ml) || (ml < ml0));    // one must be true
              BOOST_CHECK (!((ml0 < ml) && (ml < ml0))); // ...but not both
            }
          ostringstream out;
          out << ml;
          BOOST_CHECK_EQUAL (out.str().find(nameToAdd), string::npos);
        }
    }
}

BOOST_FIXTURE_TEST_CASE ( merging,  MailingListTests) {
  
  MailingList ml0;
  ml0.addContact(contacts[0]);
  ml0.addContact(contacts[1]);
  ml0.addContact(contacts[2]);
  
  MailingList ml1;
  ml1.addContact(contacts[2]);
  ml1.addContact(contacts[3]);
  
  ml1.merge(ml0);
  shouldBeEqual(ml1, generate(4));
}

2.2 Testing for Pointer/Memory Faults

In our example, we did not write explicit tests for the destructor. Partly that’s because the destructor is already being invoked a lot - every time we exit a function or a {…} statement block that declares a local variable. So we have some good reason to hope that the destructor has been well exercised already.

Also, most of the effects of a destructor are hard to see directly. How do you check to see if memory has been successfully deleted?

Most destructors simply delete pointers, and pointer issues are particularly difficult to test and debug.

Tools for Testing Pointer/Memory Faults

3. Be Independent:

Is there a virtue to independence?

4. Be Pro-active: Write the Tests First

Ideally, we have made it easier to write self-checking unit tests than to write the actual code to be tested.

Debugging: How Can You Fix What You Can’t See?

The test-first philosophy is easiest to understand in a maintenance/debugging context.

Test-Writing as a Design Activity

Every few years, software designers rediscover the principle of writing tests before implementing code.

Agile and TDD (Test-Driven Development) are just the latest in this long chain.