Automating the Testing Oracle

Steven J Zeil

Last modified: Dec 21, 2019
Contents:
1 Automating the Oracle
1.1 How can oracles be automated?
1.2 Examining Output
1.3 Limitations of Capture-and-Examine Tests
2 Self-Checking Unit & Integration Tests
2.1 First Cut at a Self-Checking Test
2.2 Better Idea: Test the Public Functions Against Each Other
2.3 assert() might not be quite what we want
3 JUnit Testing
3.1 JUnit Basics
3.2 Structure of a JUnit Test
3.3 A JUnit Example
3.4 Setting Up Tests
3.5 Writing Expressive Tests
3.6 Matchers
3.7 JUnit in Eclipse
4 C++ Unit Testing
4.1 Google Test
4.2 Boost Test Framework
4.3 CppUnitLite

 

Earlier we introduced this model of the basic testing process.

In this lesson, we turn our attention to the oracle, the process for determining whether a test has failed.

We will argue that the economics of testing provide a powerful incentive to automate the oracle. Traditionally, this has been done by capture-and-examine, capturing outputs and then using a separate automated process to examine and pass judgement upon those outputs.

Current practice, however, emphasizes self-checking tests, drivers that both feed input to the code under test and immediately evaluating its responses. Self-checking tests are supported by a *Unit framework. We will look at popular frameworks for both Java and C++.

1 Automating the Oracle

Why Automate the Oracle?

If a bug is reported or a new feature requested, the first step is to add new tests that fail because of the bug or missing feature. Then we will know we have successfully fixed the problem when we are able to pass that test.

The best way to be sure programmers rerun the tests on a regular basis is to make the test run part of the regular build process (e.g., build the test runs into the project make file) and to make them self-checking.

1.1 How can oracles be automated?

Output Capture

If we are doing systems/regression tests, the first step towards automation is to capture the output:

If a program produces output files, one can self-check by creating a file representing the expected correct output, then running the program to get the actual output file and using a simple comparison utility like the Unix diff or cmp commands to see if the actual output is identical to the expected output.

For system-level regression tests, this is even simpler. Once we have a program that passes our system tests, we run it on those tests and save the outputs. Those become the expected output files for later regression testing. (Remember, the point of regression testing is to determine if any behavior has changed due to recent updates to the code.)

If the program updates a database, it may be possible to capture entire databases in a similar fashion. Alternatively, we write database queries to check for changes in the records most likely to have been affected by a test.

On the other hand, if the program’s main function is to present information on a screen, self-checking is very difficult. Screen captures are often not much use, because we are unlikely to want to deal with changes where, say one window is a pixel wider or a pixel to the left of where it had been in a prior test. Self-checking tests for programs like this either require extremely fine control over all possible interactive inputs and graphics device characteristics, or they require a careful “design for testability” to record, in a testing log file, information about what is being rendered on the screen. (We’ll revisit this idea later in the semester when we discuss the MVC pattern for designing user interfaces.)

Output Capture and Drivers

At the unit and integration test level, we are testing functions and ADT member functions that most often produce data, not files, as their output. That data could be of any type.

How can we capture that output in a fashion that allows automated examination?

1.2 Examining Output

1.2.1 File Tools

Alternatives

Custom oracles

1.2.2 expect

expect is a shell for testing interactive programs.

Key expect Commands

Sample Expect Script

Log in to other machine and ignore “authenticity” warnings.

#!/usr/local/bin/expect
set timeout 60
spawn ssh $argv
while {1} {
  expect {

    eof                          {break}
    "The authenticity of host"   {send "yes\r"}
    "password:"                  {send "$password\r"}
    "$argv"                      {break} # assume machine name is in prompt
  }
}
interact
close $spawn_id

Expect: Testing a program

    puts "in test0: $programdir/testsets\n"
catch {
   spawn $programdir/testsets                          ➀

   expect \                                           ➁
    "RESULT: 0" {fail "testsets"} \                   ➂
    "missing expected element" {fail "testsets"} \
    "contains unexpected element" {fail "testsets"} \
    "does not match" {fail "testsets"} \
    "but not destroyed" {fail "testsets"} \
    {RESULT: 1} \{pass "testsets"} \                 ➃
    eof {fail "testsets"; puts "eofbsl nl"} \
    timeout {fail "testsets"; puts "timeout\n"}
}
catch {
    close
    wait
}

1.3 Limitations of Capture-and-Examine Tests

Structured Output

For unit/integration test, output is often a data structure.

Repository Output

For system and high-level unit/integration tests, output may be updates to a database or other repository.

Graphics Output

2 Self-Checking Unit & Integration Tests

In testing an ADT, we are not testing an individual function, but a collection of related functions. In some ways that makes thing easier, because we can use many of these functions to help test one another.

2.1 First Cut at a Self-Checking Test

Suppose you were testing a SetOfInteger ADT and had to test the add function in isolation, you would need to know how the data was stored in the set and would have to write code to search that storage for a value that you had just added in your test. E.g.,

void testAdd (SetOfInteger aSet)
{
   aSet.add (23);
   bool found = false;
   for (int i = 0; i < aSet.numMembers && !found; ++i)
      found = (aSet[i] == 23);
   assert(found);
}

2.1.1 What’s Good and Bad About This?

void testAdd (SetOfInteger aSet)
{
   aSet.add (23);
   bool found = false;
   for (int i = 0; i < aSet.numMembers && !found; ++i)
      found = (aSet.data[i] == 23);
   assert(found);
}

2.2 Better Idea: Test the Public Functions Against Each Other

On the other hand, if you are testing the add and the contains function, you could use the second function to check the results of the first:

void testAdd (SetOfInteger aSet)
{
  aSet.add (23);
  assert (aSet.contains(23));
}

Not only is this code simpler than writing your own search function as part of the test driver, it continues to work even if the data structure used to implement the ADT should be changed. What’s more, it is, in a sense, a more thorough test, since it tests two functions at once. Finally, there’s the simple fact that the test with the explicit loop probably won’t even compile, since it refers directly to data members that are almost certainly private.

In a sense, we have made a transition from white-box towards black-box testing. The new test case deliberately ignores the underlying structure.

2.2.1 Idiom: Preserve and Compare

void testAdd (SetOfInteger startingSet)
{
  SetOfInteger aSet = startingSet;
  aSet.add (23);
  assert (aSet.contains(23));
  if (startingSet.contains(23))
     assert (aSet.size() == startingSet.size());
  else
     assert (aSet.size() == startingSet.size() + 1);
}

Note that we

2.2.2 More Thorough Tests

You can see the usefulness of “preserve and comapre” in this more thorough test.

void testAdd (SetOfInteger aSet)
{
  for (int i = 0; i < 1000; ++i)
    {
     int x = rand() % 500;
     bool alreadyContained = aSet.contains(x);
     int oldSize = aSet.size();
     aSet.add (23);
     assert (aSet.contains(x));
     if (alreadyContained)
       assert (aSet.size() == oldSize);
     else
       assert (aSet.size() == oldSize + 1);
    }
}

2.3 assert() might not be quite what we want

Our use of assert() in these examples has mixed results

3 JUnit Testing

JUnit is a testing framework that has seen wide adoption in the Java community and spawned numerous imitations for other programming languages.

3.1 JUnit Basics

Getting Started: Eclipse & JUnit

Let’s suppose we are building a mailing list application. the mailing list is a collection of contacts.

Contact.java 
package mailinglist;
/**
 * A contact is a name and address.
 * <p>
 * For the purpose of this example, I have simplified matters 
 * a bit by making both of these components simple strings. 
 * In practice, we would expect Address, at least, to be a
 * more structured type.
 * 
 * @author zeil
 *
 */
public class Contact 
  implements Cloneable, Comparable<Contact> {

 private String theName;
 private String theAddress;

 /**
  * Create a contact with empty name and address.
  *
  */
 public Contact ()
 {
  theName = "";
  theAddress = "";
 }

 /**
  * Create a contact
  * @param nm  name 
  * @param addr address
  */
 public Contact (String nm, String addr)
 {
  theName = nm;
  theAddress = addr;
 }


 /**
  * Get the name of the contact
  * @return the name
  */
 public String getName()
 {
  return theName;
 }
 
 /**
  * Change the name of the contact
  * @param nm new name
  */
 public void setName (String nm) 
 {
  theName= nm;
 }

 /**
  * Get the address of the contact
  * @return the address
  */
 public String getAddress()
 {
  return theAddress;
 }
 
 /**
  * Change the address of the contact 
  * @param addr new address
  */
 public void setAddress (String addr) 
 {
  theAddress = addr;
 }

 /**
  * True if the names and addresses are equal 
  */
 public boolean equals (Object right)
 {
  Contact r = (Contact)right;
  return theName.equals(r.theName)
     && theAddress.equals(r.theAddress);
 }

 public int hashCode ()
 {
  return theName.hashCode() + 3 * theAddress.hashCode();
 }

 public String toString()
 {
  return theName + ": " + theAddress;
 }
 
 public Object clone()
 {
  return new Contact(theName, theAddress);
 }

 /**
  * Compare this contact to another.
  * Return value > 0 if this contact precedes the other,
  * == 0 if the two are equal, and < 0 if this contact
  * follows the other.  
  */
 public int compareTo (Contact c)
 {
  int nmcomp = theName.compareTo(c.theName);
  if (nmcomp != 0)
   return nmcomp;
  else
   return theAddress.compareTo(c.theAddress);
 }
}
MailingList.java 
package mailinglist;

/**
 * A collection of names and addresses
 */
public class MailingList implements Cloneable {

	/**
	 * Create an empty mailing list
	 *
	 */
	public MailingList() {
		first = null;
		last = null;
		theSize = 0;
	}

	/**
	 *  Add a new contact to the list
	 *  @param contact new contact to add
	 */
	public void addContact(Contact contact) {
		if (first == null) {
			// add to empty list
			first = last = new ML_Node(contact, null);
			theSize = 1;
		} else if (contact.compareTo(last.contact) > 0) {
			// add to end of non-empty list
			last.next = new ML_Node(contact, null);
			last = last.next;
			++theSize;
		} else if (contact.compareTo(first.contact) < 0) {
			// add to front of non-empty list
			first = new ML_Node(contact, first);
			++theSize;
		} else {
			// search for place to insert
			ML_Node previous = first;
			ML_Node current = first.next;
			assert (current != null);
			while (contact.compareTo(current.contact) < 0) {
				previous = current;
				current = current.next;
				assert (current != null);
			}
			previous.next = new ML_Node(contact, current);
			++theSize;
		}
	}

	/**
	 *  Remove one matching contact 
	 *  @param c remove a contact equal to c
	 */
	public void removeContact(Contact c) {
		ML_Node previous = null;
		ML_Node current = first;
		while (current != null && c.getName().compareTo(current.contact.getName()) > 0) {
			previous = current;
			current = current.next;
		}
		if (current != null && c.getName().equals(current.contact.getName()))
			remove(previous, current);
	}

	/**
	 * Remove a contact with the indicated name
	 * @param name name of contact to remove
	 */
	public void removeContact(String name) {
		ML_Node previous = null;
		ML_Node current = first;
		while (current != null && name.compareTo(current.contact.getName()) > 0) {
			previous = current;
			current = current.next;
		}
		if (current != null && name == current.contact.getName())
			remove(previous, current);

	}

	/**
	 * Search for contacts
	 * @param name name to search for
	 * @return true if a contact with an equal name exists
	 */
	public boolean contains(String name) {
		ML_Node current = first;
		while (current != null && name.compareTo(current.contact.getName()) > 0) {
			current = current.next;
		}
		return (current != null && name == current.contact.getName());
	}

	/**
	 * Search for contacts
	 * @param name name to search for
	 * @return contact with that name if found, null if not found
	 */
	public Contact getContact(String name) {
		ML_Node current = first;
		while (current != null && name.compareTo(current.contact.getName()) > 0) {
			current = current.next;
		}
		if (current != null && name == current.contact.getName())
			return current.contact;
		else
			return null;
	}

	/**
	 * combine two mailing lists
	 *
	 */
	public void merge(MailingList anotherList) {
		// For a quick merge, we will loop around, checking the 
		// first item in each list, and always copying the smaller
		// of the two items into result
		MailingList result = new MailingList();
		ML_Node thisList = first;
		ML_Node otherList = anotherList.first;
		while (thisList != null && otherList != null) {
			int comp = thisList.contact.compareTo(otherList.contact);
			if (comp <= 0) {
				result.addContact(thisList.contact);
				thisList = thisList.next;
				/* if (comp == 0)
					otherList = otherList.next;
                                 [Deliberate bug ]  */
			} else {
				result.addContact(otherList.contact);
				otherList = otherList.next;
			}
		}
		// Now, one of the two lists has been entirely copied. 
		// The other might still have stuff to copy. So we just copy
		// any remaining items from the two lists. Note that one of these
		// two loops will execute zero times.
		while (thisList != null) {
			result.addContact(thisList.contact);
			thisList = thisList.next;
		}
		while (otherList != null) {
			result.addContact(otherList.contact);
			otherList = otherList.next;
		}
		// Now result contains the merged list. Transfer that into this list.
		first = result.first;
		last = result.last;
		theSize = result.theSize;
	}

	/**
	 * How many contacts in list?
	 */
	public int size() {
		return theSize;
	}

	/**
	 * Return true if mailing lists contain equal contacts
	 */
	public boolean equals(Object anotherList) {
		MailingList right = (MailingList) anotherList;
		if (theSize != right.theSize) // (easy test first!)
			return false;
		else {
			ML_Node thisList = first;
			ML_Node otherList = right.first;
			while (thisList != null) {
				if (!thisList.contact.equals(otherList.contact))
					return false;
				thisList = thisList.next;
				otherList = otherList.next;
			}
			return true;
		}
	}

	public int hashCode() {
		int hash = 0;
		ML_Node current = first;
		while (current != null) {
			hash = 3 * hash + current.contact.hashCode();
			current = current.next;
		}
		return hash;
	}

	public String toString() {
		StringBuffer buf = new StringBuffer("{");
		ML_Node current = first;
		while (current != null) {
			buf.append(current.contact.toString());
			current = current.next;
			if (current != null)
				buf.append("\n");
		}
		buf.append("}");
		return buf.toString();
	}

	/**
	 * Deep copy of contacts
	 */
	public Object clone() {
		MailingList result = new MailingList();
		ML_Node current = first;
		while (current != null) {
			result.addContact((Contact) current.contact.clone());
			current = current.next;
		}
		return result;
	}

	private class ML_Node {
		public Contact contact;

		public ML_Node next;

		public ML_Node(Contact c, ML_Node nxt) {
			contact = c;
			next = nxt;
		}
	}

	private int theSize;

	private ML_Node first;

	private ML_Node last;

	// helper functions
	private void remove(ML_Node previous, ML_Node current) {
		if (previous == null) {
			// remove front of list
			first = current.next;
			if (last == current)
				last = null;
		} else if (current == last) {
			// remove end of list
			last = previous;
			last.next = null;
		} else {
			// remove interior node
			previous.next = current.next;
		}
		--theSize;
	}

}

3.2 Structure of a JUnit Test

3.2.1 Assertions

Each of these can have an optional first argument of a string that will be printed when the assertion fails. For example,

Integer pos = search(arr, x);
assertNotNull (pos);
assertNotNull ("Could not find x in arr", pos);

3.3 A JUnit Example

3.3.1 A First Test Suite

Right-click on the project and select New ... JUnit Test Case.

3.3.2 A First Test Suite (cont.)

You’ll get something like this:

package mailinglist;

import static org.junit.Assert.*;

import org.junit.Before;
import org.junit.Test;

public class TestMailingList {

    @Test
    public void testMailingList() {
       fail("Not yet implemented");
    }

    @Test
    public void testAddContact() {
       fail("Not yet implemented");
    }

}

Save this, and run it (right-click on the new file in the Package Explorer, and select Run As ... JUnit Test)

3.3.3 A First Test

Let’s implement a test for the mailing list constructor.

/**
 * Create an empty mailing list
 *
 */
public MailingList() {

Glancing through the MailingList interface, what would we expect of an empty mailing list?

3.3.4 Testing the Constructor

Change the testMailingList function to

testmlConst.java 
@Test
public void testMailingList() {
    MailingList ml = new MailingList();  ➀
    assertFalse (ml.contains("Jones"));  ➁
    assertNull (ml.getContact("Smith"));
    assertEquals (0, ml.size());         ➂
}

3.3.5 Running the Constructor Test

3.3.6 Testing addContact

Now let’s add a test for addContact:

After adding a contact

Testing addContact - first pass

Try this test (run it!)

@Test
public void testAddContact() {
    MailingList ml = new MailingList();
    Contact jones = new Contact("Jones",
                    "21 Penn. Ave.");
    ml.addContact (jones);
    assertTrue (ml.contains("Jones"));
    assertFalse (ml.contains("Smith"));
    assertEquals ("Jones", ml.getContact("Jones").getName());
    assertNull (ml.getContact("Smith"));
    assertEquals (1, ml.size());
}

It works. But it feels awfully limited.

3.4 Setting Up Tests

Before we try making that test more rigorous, let’s look at how we can organize the set up of auxiliary data like our contacts:

Contact jones;

@Before
public void setUp() throws Exception {
    jones = new Contact("Jones", "21 Penn. Ave.");
}
  ⋮
@Test
 public void testAddContact() {
     MailingList ml = new MailingList();
     ml.addContact (jones);
       ⋮

3.4.1 Fixtures

3.4.2 Testing addContact - setup

Contact jones;
Contact baker;
Contact holmes;
Contact wolfe;
MailingList mlist;

@Before
public void setUp() throws Exception {
    jones = new Contact("Jones", "21 Penn. Ave.");
    baker = new Contact("Baker", "Drury Ln.");
    holmes = new Contact("Holmes",
                  "221B Baker St.");
    wolfe = new Contact("Wolfe",
                 "454 W. 35th St.");
    mlist = MailingList();
    mlist.addContact(baker);
    mlist.addContact (holmed);
    mlist.addContact (baker);
}

Create a series of contacts

3.4.3 Testing addContact - improved

@Test
public void testAddContact() {
    assertTrue (mlist.contains("Jones"));
    assertEquals ("Jones",
                  mlist.getContact("Jones").getName());
    assertEquals (4, ml.size());
}

Still seems a bit limited - we’ll address that later.

3.5 Writing Expressive Tests

One goal is writing tests is to make them both easy to read and to set them up so that the messages they issue upon failure are as helpful as possible.

3.5.1 Choose the most limited assertion.

These two assertions mean the same thing:

assertTrue(string1.equals(string2));
assertEquals(string1, string2);

I would argue that the second is easier to read. Moreover, if these assertions fail, the second gives a more informative message:

import static org.junit.Assert.*;

import org.junit.Test;

public class testMessages {

    String string1 = "abcdef";
    String string2 = "abcZef";

    @Test
    public void test1() {
       assertTrue (string1.equals(string2));
    }

    @Test
    public void test2() {
       assertEquals (string1, string2);
    }
}

results in the messages:

test1: java.lang.AssertionError
       at testMessages.java:14
test2: org.junit.ComparisonFailure: expected:<abc[d]ef> but was:<abc[Z]ef>
       at testMessages.java:19

The second message is clearly more helpful.

You can alleviate this a bit by adding a message

assertTrue ("string1 is not equal to string2",
            string1.equals(string2));

but, really, it’s still not as good.

Use assertTrue and assertFalse only when more specific assertions do not apply. Then consider adding messages to them explaining what they are actually checking for.

For example, if what we really wanted to assert was that one string contained the other, then we would be stuck with

assertTrue ("string1 does not contain " + string2",
            string1.contains(string2));

3.6 Matchers

Because the set of basic assertions is, of necessity, limited, a newer style of assertion has arisen that

  1. Tries to be more expressive for many common tests.
  2. Can be extended to new “kinds” of assertions fitted to custom ADTs.

The assertion style looks like

assertThat (object, matcher);

(Again, an optional message can be added to the front of the parameter list, but it is generally hopes that the new style reduces the need for this.)

In this new style,

assertThat (obj1, equalTo(obj2));

means the same thing as

assertEquals (obj1, obj2);

but uses the equalTo matcher instead.

3.6.1 Core matchers

The Hamcrest Matchers include a variety of primitives for checking one value:

3.6.2 Core matchers – modifiers

Other core matchers are used to modify the way that matchers are applied

With these, we can write assertions like:

assertThat(not(x.equalTo(y));
assertThat(anyOf(nullValue(x), not(x.equalTo(y)));

3.6.3 Additional Matchers

Also in the Hamcrest library, matchers for strings, numbers, iterable containers, etc.:

assertThat(x, lessThan(y));
assertThat(string1, isEmptyOrNullString());
assertThat(string1, startsWith("Hello"));
assertThat(myList, hasItem("Smith");
assertThat(myList, contains(smallerList);
assertThat(myList, containsInAnyOrder("Jones", "Doe", "Smith");

See the Matchers class API for a full list.

3.7 JUnit in Eclipse

The Eclipse IDE for Java includes a copy of the JUnit library.

4 C++ Unit Testing

4.1 Google Test

Google Test, a.k.a. gtest, provides a similar *Unit environment for C++

Example

This time, the C++ version of the mailing list.

  1. Source code is here.
    • Unpack into a convenient directory.
  2. With Eclipse create a C++ project in that directory.
  3. Compile
  4. Run the resulting executable as a local C++ application
  5. Run as a *unit test:
    • Right-click on the binary executable, select “Run as … Run configurations”.
    • Select C/C++ Unit, click “New” button
    • On the “C/C++ Testing” tab, select Tests Runner: Google Tests Runner
    • Click “Run”

Examining the ADT

mailinglist.h 
#ifndef MAILINGLIST_H
#define MAILINGLIST_H

#include <iostream>
#include <string>

#include "contact.h"
#include "mlIterator.h"

/**
   A collection of names and addresses
*/
class MailingList
{
public:

  typedef MLConstIterator iterator;
  typedef MLConstIterator const_iterator;

  MailingList();
  MailingList(const MailingList&);
  ~MailingList();

  const MailingList& operator= (const MailingList&);

  // Add a new contact to the list
  void addContact (const Contact& contact);

  // Does the list contain this person?
  bool contains (const Name&) const;

  // Find the contact
  const Contact& getContact (const Name& nm) const;
  //pre: contains(nm)

  // Remove one matching contact
  void removeContact (const Contact&);
  void removeContact (const Name&);

  // combine two mailing lists
  void merge (const MailingList& otherList);

  // How many contacts in list?
  int size() const;


  bool operator== (const MailingList& right) const;
  bool operator< (const MailingList& right) const;

  // Provides iterator over contacts
  const_iterator begin() const;
  const_iterator end() const;


private:

  struct ML_Node {
    Contact contact;
    ML_Node* next;

    ML_Node (const Contact& c, ML_Node* nxt)
      : contact(c), next(nxt)
    {}
  };

  ML_Node* first;
  ML_Node* last;
  int theSize;

  // helper functions
  void clear();
  void remove (ML_Node* previous, ML_Node* current);

  friend std::ostream& operator<< (std::ostream& out, const MailingList& addr);
};

// print list, sorted by Contact
std::ostream& operator<< (std::ostream& out, const MailingList& list);


#endif

Should look familiar after the Java version

Examining the Tests

MailingListTests.cpp 
#include "mailinglist.h"

#include <string>
#include <vector>

#include "gtest/gtest.h"

namespace {

using namespace std;

// The fixture for testing class MailingList.
class MailingListTests : public ::testing::Test {  ➀
 public:
  Contact jones;                                 ➃
  MailingList mlist;

  virtual void SetUp() {
    jones = Contact("Jones", "21 Penn. Ave.");     ➄
    mlist = MailingList();
    mlist.addContact (Contact ("Baker", "Drury Ln."));
    mlist.addContact (Contact ("Holmes", "221B Baker St."));
    mlist.addContact (Contact ("Wolfe", "454 W. 35th St."));
  }

  virtual void TearDown() {
  }

};

TEST_F (MailingListTests, constructor) {
  MailingList ml;
  EXPECT_EQ (0, ml.size());                ➁
  EXPECT_FALSE (ml.contains("Jones"));
}


TEST_F (MailingListTests, addContact) {
  mlist.addContact (jones);                      ➅
  EXPECT_TRUE (mlist.contains("Jones"));         ➂
  EXPECT_EQ ("Jones", mlist.getContact("Jones").getName());
  EXPECT_EQ (4, ml.size());
}




}  // namespace

Roughly similar to the JUnit tests

4.2 Boost Test Framework

Boost UTF

Boost is a well-respected collection of libraries for C++.

Boost Test

Example

Again, the C++ version of the mailing list.

  1. Source code is here.
    • Unpack into a convenient directory.
  2. With Eclipse create a C++ project in that directory.
    • Add path to Boost include directory
  3. Compile
  4. Run the resulting executable as a local C++ application
  5. Run as a *unit test:
    • Right-click on the binary executable, select “Run as … Run configurations”.
    • Select C/C++ Unit, click “New” button
    • On the “C/C++ Testing” tab, select Tests Runner: Boost Tests Runner
    • Click “Run”

Examining the Tests

MailingListTests2.cpp 
#define BOOST_TEST_MODULE MailingList test

#include "mailinglist.h"

#include <string>
#include <vector>

#include <boost/test/included/unit_test.hpp>

using namespace std;

// The fixture for testing mailing lists.
class MailingListTests  {                        ➀
 public:
  Contact jones;                                 ➁
  MailingList mlist;

  MailingListTests() {                           ➃
    jones = Contact("Jones", "21 Penn. Ave.");
    mlist.addContact (Contact ("Baker", "Drury Ln."));
    mlist.addContact (Contact ("Holmes", "221B Baker St."));
    mlist.addContact (Contact ("Wolfe", "454 W. 35th St."));
  }

  ~MailingListTests() {
  }

};

BOOST_AUTO_TEST_CASE ( constructor ) {         ➄
  MailingList ml;
  BOOST_CHECK_EQUAL (0, mlist.size());            ➅
  BOOST_CHECK (!mlist.contains("Jones"));
}


BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) { ➆
  mlist.addContact (jones);                         ➂
  BOOST_CHECK (mlist.contains("Jones"));           
  BOOST_CHECK_EQUAL ("Jones", mlist.getContact("Jones").getName());
  BOOST_CHECK_EQUAL (4, mlist.size());
}

4.3 CppUnitLite

CppUnitLite is my own C++ test framework, with features:

Future versions of this framework are intended to offer a mocking framework that is simpler and more intuitive than currently available elsewhere.

4.3.1 Example

MailingListTestsLite.cpp 
#include "unittest.h"     ➀
#include "mailinglist.h"

#include <string>
#include <vector>

using namespace std;

Contact jones;
MailingList mlist;

void setUp()
{
	jones = Contact("Jones", "21 Penn. Ave.");
	mlist = MailingList();
	mlist.addContact (Contact ("Baker", "Drury Ln."));
	mlist.addContact (Contact ("Holmes", "221B Baker St."));
	mlist.addContact (Contact ("Wolfe", "454 W. 35th St."));
}



UnitTest (constructor) {    ➁
	MailingList ml;
	assertThat (ml.size(), is(0));      ➂
	assertFalse (ml.contains("Jones"));
}


UnitTest (addContact) {
	setup();                     ➃
	mlist.addContact (jones);
	assertTrue (mlist.contains("Jones"));
	assertThat (mlist.getContact("Jones").getName(), is("Jones"));
	assertThat (ml.size(), is(4));
	assertThat (ml, hasItem(jones));  ➄
}
© 2015-2020, Old Dominion Univ.