Unit Testing Frameworks
Steven J Zeil
Unit testing frameworks streamline the process of writing test drivers with automated oracles.
1 JUnit
JUnit is a unit testing framework for Java.
Introduced in 2002 by Kent Beck and Erich Gamma, JUnit (and it’s xUnit relations) have become near-universal.
-
Reputedly, first version was prototyped during a Trans-Atlantic flight as a way for Beck to teach Java to Gamma.
-
Martin Fowler: “Never in the field of software development was so much owed by so many to so few lines of code.”
-
A 2015 survey shows how pervasive JUnit is become.
1.1 Concepts
Junit…
- Introduces a structure for a
- suite (separately executable program) of
- test cases (functions),
- each performing multiple self-checking tests (_assertions_)
- using a rich set of assertion operators
- also provides support for setup, cleanup, report generation
- readily integrated into IDEs
1.2 What Does a Unit Testing Framework Do?

Unit testing frameworks speed the process of
- Writing test drivers.
- They do not help with writing test stubs, but many are compatible with object mocking tools that address stubs.
- Automating the test oracle.
- The oracle is the procedure for determining whether a test has passed or failed.
1.3 JUnit by Example
We will be looking at JUnit 5, also known as JUnit Jupiter.
We’ll start with a simple example, a mailing list application. The MailingList
is a collection of Contact
s.
1.3.1 Set up the project.
package mailinglist;
/**
* A contact is a name and address.
* <p>
* For the purpose of this example, I have simplified matters
* a bit by making both of these components simple strings.
* In practice, we would expect Address, at least, to be a
* more structured type.
*
* @author zeil
*
*/
public class Contact
implements Cloneable, Comparable<Contact> {
private String theName;
private String theAddress;
/**
* Create a contact with empty name and address.
*
*/
public Contact ()
{
theName = "";
theAddress = "";
}
/**
* Create a contact
* @param nm name
* @param addr address
*/
public Contact (String nm, String addr)
{
theName = nm;
theAddress = addr;
}
/**
* Get the name of the contact
* @return the name
*/
public String getName()
{
return theName;
}
/**
* Change the name of the contact
* @param nm new name
*/
public void setName (String nm)
{
theName= nm;
}
/**
* Get the address of the contact
* @return the address
*/
public String getAddress()
{
return theAddress;
}
/**
* Change the address of the contact
* @param addr new address
*/
public void setAddress (String addr)
{
theAddress = addr;
}
/**
* True if the names and addresses are equal
*/
public boolean equals (Object right)
{
Contact r = (Contact)right;
return theName.equals(r.theName)
&& theAddress.equals(r.theAddress);
}
public int hashCode ()
{
return theName.hashCode() + 3 * theAddress.hashCode();
}
public String toString()
{
return theName + ": " + theAddress;
}
public Object clone()
{
return new Contact(theName, theAddress);
}
/**
* Compare this contact to another.
* Return value > 0 if this contact precedes the other,
* == 0 if the two are equal, and < 0 if this contact
* follows the other.
*/
public int compareTo (Contact c)
{
int nmcomp = theName.compareTo(c.theName);
if (nmcomp != 0)
return nmcomp;
else
return theAddress.compareTo(c.theAddress);
}
}
-
Set up an Eclipse project with these two files
- Make sure that these compile successfully with no errors.
-
In the Eclipse Build Path, add a library: JUnit,
- Choose JUnit5 from the list of versions
1.3.2 Set up the MailingList tests
- Right-click on the project. Select
New...Junit Test Case
- Make sure
JUnit Jupiter
is selected. Fill in the name (TestMailingList
) and the “Class Under Test” (mailinglist.MailingList
). ClickNext
. - For reasons we will explain later, we select the functions that initialize or make changes to the
MailingList
: the constructor,addContact
,removeContact
,merge
, andclone
. ClickFinish
.- These functions are known as the mutators of the class.
1.3.3 Structure of a JUnit test
We wind up with something like this:
package mailinglist;
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;
class TestMailingList ➀ {
@Test ➁
void testMailingList() {
fail("Not yet implemented");
}
@Test
void testAddContact() {
fail("Not yet implemented");
}
@Test
void testRemoveContactContact() {
fail("Not yet implemented");
}
@Test
void testRemoveContactString() {
fail("Not yet implemented");
}
@Test
void testMerge() {
fail("Not yet implemented");
}
@Test
void testClone() {
fail("Not yet implemented");
}
}
-
➀ A JUnit test is a class.
-
➁ It contains one or more “test cases” or “test functions” – member functions that return
void
, take no parameters, and are marked with@Test
. -
Each test case needs to
- Sets up any necessary test data.
- Runs the code to be tested on that data.
- Examines the results of the test via one or more assertions.
But, of course, right now none of that has been written yet.
-
A test case can
- Succeed if all assertions are true.
- Fail if any assertion is false (or if the
fail(...)
function is called explicitly). - Terminate with an error if the test case itself throws an exception.
1.3.4 Our first test case - the constructor
Let’s think about testing the MailingList
constructor:
MailingList() {
⋮
}
What do we think should be true when this has completed?
We go to our skeleton testMailingList()
and try to express those ideas:
@Test
void testMailingList() {
MailingList ml = new MailingList();
assertEquals (0, ml.size());
}
Is that enough?
- To be thorough, we should look at all of the
MailingList
functions that we did not select to create separate tests for. - These are the functions that examine the value of the
MailingList
without changing it:contains
,getContact
,size
,equals
,hashCode
, andtoString
.- These are known as the accessors of the class.
- For each of these, we ask what it should tell us about the newly created
MailingList
.
After construction…
contains
should return false for all names.getContact
should return null for all names.size
should be zero.equals
should return true when compared to another freshly-constructed list.hashCode
s should be equal wheneverequals
is true.- can’t really say anything about
toString
, as the documentation does not require a specific output format. (The primary role oftoString
in most classes is to provide debugging output.) But it should return some kind of string.
@Test
void testMailingList() {
MailingList ml = new MailingList(); ➀
assertEquals (0, ml.size()); ➁
assertFalse (ml.contains("Jones"));
assertNull (ml.getContact("Smith"));
MailingList ml2 = new MailingList();
assertEquals (ml2, ml);
assertEquals (ml2.hashCode(), ml.hashCode());
assertNotNull(ml.toString());
}
Every good JUnit test can be read as
- ➀ A demo of how to use the function being tested,
- ➁ followed by a description of that function’s effects.
Reading the unit tests is often one of the easiest ways to discover how to use an unfamiliar class.
1.3.5 Running the test
Now would be a good time to run the test.
In Eclipse, right-click on the TestMailinglist.java
file and select Run as...JUnit test
.
1.4 Assertions
We’ve now seen several examples of JUnit assertions.
These are fundamental, but not as descriptive as other options:
assertTrue(
condition)
: condition should be true.assertFalse(
condition)
: condition should be false.
More common basic assertions:
assertEquals(
x , y)
: x and y should be equal. (For objects, this is tested using theequals(...)
function. For primitives likeint
, the==
operator is used.)assertEquals(
x , y , delta)
: Floating point numbers x and y should be approximately equal, within plus or minus delta.assertNotEquals(
x , y)
: x and y should not be equal.assertSame(
obj1 , obj2)
: obj1 and obj2 should hold the same address.assertNotSame(
obj1 , obj2)
: obj1 and obj2 should not hold the same address.assertNull(
obj1)
: obj1 should be a null reference.assertNotNull(
obj1)
: obj1 should not be a null reference.
More specialized assertions:
assertArrayEquals(
array1 , array2)
: array1 and array2 should have the same length and corresponding elements in these arrays should be equal.assertLinesMatch(
expectedList , actualListOfStrings)
: compares a list of expected strings/regular expressions against a list of actual strings.assertIterablesEquals(
expectedContainer , actualContainer)
: compares two containers of objects to see if their contents are equal.
“Meta-” assertions:
fail(
message)
: always failsassertThrows(
exception, Executable)
: asserts that code will throw an exceptionassertTimeout(
timeLimit , Executable)
: asserts that code will finish within a stated time.assertTimeoutPreemptively(
timeLimit , Executable)
: asserts that code will finish within a stated time, and kills the code if it doesn’t.
In these, the Executable
is likely to be a Java lambda expression.
e.g.,
assertTimeout(Duration.ofSeconds(1),
() -> {longMailingList1.merge(longMailingList2);});
Each assertion can have an optional final argument of a string that will be printed when the assertion fails. For example,
Integer pos = search(arr, x);
assertNotNull (pos);
assertNotNull (pos, "Could not find x in arr");
1.5 Resuming the example…
Now let’s look at addContact
.
Remember the basic pattern:
- Setup the parameters we need to the call to the function we want to test.
- Call the function.
- Use assertions to examine the effects of that call.
We can carryout the first two steps pretty easily.
@Test
void testAddContact() {
Contact holmes = new Contact("Holmes", "21B Baker St.");
MailingList ml = new MailingList();
ml.addContact(holmes);
⋮
}
1.5.1 Examining the mailing list after adding a contact
Again we look at out list of accessors:
After adding holmes
…
contains
should return true for “Holmes” but false for other names.getContact
should return a contact equal toholmes
when given “Holmes”, null for all other names.size
should be 1.equals
should return false when compared to a freshly-constructed list, true when compared to one containing onlyholmes
.hashCode
s should be equal wheneverequals
is true.- “Holmes” should appear somewhere in the output of toString.
@Test
void testAddContact() {
Contact holmes = new Contact("Holmes", "21B Baker St.");
MailingList ml = new MailingList();
ml.addContact(holmes);
assertEquals(1, ml.size());
assertTrue(ml.contains("Holmes"));
assertFalse(ml.contains("Jones"));
assertEquals (holmes, ml.getContact("Holmes"));
assertNull (ml.getContact("Jones"));
assertNotEquals(ml, new MailingList());
MailingList ml2 = new MailingList();
ml2.addContact(holmes);
assertEquals (ml2, ml);
assertEquals (ml2.hashCode(), ml.hashCode());
assertTrue(ml.toString().contains("Holmes"));
}
Run this and see if we pass.
1.5.2 Testing for multiple adds
Of course, adding a single contact to an empty mailing list is not much of a test.
- It’s a special case for most data structures we might use, and we need to test the “mainstream” cases as well.
- We can’t explore the question of whether the mailing list is keeping the contacts in the desired order.
- We can’t check on what happens when we add duplicate contacts.
We could make our existing test case for addContact
longer, or we can add additional cases to explore these possibilities:
- ➀ We need to set up a bunch of contacts.
- ➁ This is the order in which we will add them to to our mailing list.
- ➂
ml0
sets up the “preserve-and-compare” idiom. - ➃ Each of the first
i
contacts should be in the mailing list.- ➄ It’s worth adding a specialized message on failure to help us know which iteration of the loop a failure occurs on.
- ➅ Each of the remaining contacts after the first
i
ones should not be in the mailing list.
1.5.3 Adding a duplicate contact
The documentation for addContact
says that duplicates are not added. We should test for that as well:
- ➀ Again, we set up a bunch of contacts and ➁ establish an order in which we will add them.
- ➂ It’s not until we get here that we actually invoke the case that we want to test.
- ➃ After which we basically assert that adding a duplicate does not change the mailing list in any way.
Look at how much of that test was devoted to the setup, much of which was identical to the setup of the previous testAddContact
case. Looking ahead, we can guess that we will need much of the same setup for test cases for removeContact
as well.
1.5.4 Collecting common setup code into a test fixture
A test fixture in unit testing is a reusable collection of data that provides support for running multiple tests.
In JUnit, we create test fixtures by
- Declaring the shared data values as members of the test class.
- Initializing them before each test case using a
@BeforeEach
function, and - If necessary, cleaning them up after each test using an
@AfterEach
function.
So, if we create this fixture near the top of the test class:
class TestMailingList {
Contact holmes;
Contact adams;
Contact baker;
Contact charles;
Contact dickens;
Contact adams2;
Contact[] inputs;
MailingList ml;
@BeforeEach
void setup() {
holmes = new Contact("Holmes", "21B baker St.");
adams = new Contact("Adams", "21 Pennsylvania Ave.");
baker = new Contact("Baker", "Drury Ln.");
charles = new Contact("Charles", "100 1st St.");
dickens = new Contact("Dickens", "200 2nd St.");
adams2 = new Contact("Adams", "32 Pennsylvania Ave.");
Contact[] order = {dickens, adams, charles, holmes, baker};
inputs = order;
ml = new MailingList();
}
Then we can simplify our tests:
@Test
void testMailingList() {
assertEquals (0, ml.size());
assertFalse (ml.contains("Jones"));
assertNull (ml.getContact("Smith"));
MailingList ml2 = new MailingList();
assertEquals (ml2, ml);
assertEquals (ml2.hashCode(), ml.hashCode());
assertNotNull(ml.toString());
}
@Test
void testAddContact() {
ml.addContact(holmes);
assertEquals (1, ml.size());
assertTrue(ml.contains("Holmes"));
assertFalse(ml.contains("Jones"));
assertEquals (holmes, ml.getContact("Holmes"));
assertNull (ml.getContact("Jones"));
assertNotEquals(ml, new MailingList());
MailingList ml2 = new MailingList();
ml2.addContact(holmes);
assertEquals (ml2, ml);
assertEquals (ml2.hashCode(), ml.hashCode());
assertTrue(ml.toString().contains("Holmes"));
}
@Test
void testAddContactMulti() {
for (int testSize = 1; testSize <= inputs.length; ++testSize) {
MailingList ml0 = (MailingList)ml.clone();
ml.addContact(inputs[testSize-1]);
assertEquals(testSize, ml.size());
for (int j = 0; j <= testSize-1; ++j) {
assertTrue (ml.contains(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
assertEquals (inputs[j], ml.getContact(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
assertTrue(ml.toString().contains(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
}
for (int j = testSize+1; j < inputs.length; ++j) {
assertFalse (ml.contains(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
assertNull (ml.getContact(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
assertFalse(ml.toString().contains(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
}
assertNotEquals(ml, ml0);
}
}
@Test
void testAddDuplicateContact() {
for (int i = 0; i < inputs.length; ++i) {
ml.addContact(inputs[i]);
}
MailingList ml0 = (MailingList)ml.clone();
// Add a duplicate
ml.addContact(baker);
assertEquals(ml0.size(), ml.size());
assertEquals(ml0, ml);
assertEquals(ml0.hashCode(), ml.hashCode());
assertTrue (ml.contains(baker.getName()));
assertEquals (baker, ml.getContact(baker.getName()));
assertTrue(ml.toString().contains(baker.getName()));
}
1.5.5 Are we keeping the contacts in the proper order?
The documentation of addContact says that we are supposed to keep the contacts in ascending order.
That’s not a problem for addTestContact
, because we only have one contact (so it can hardly be out of order!). But it is something that we should be checking in the other two test cases.
But, how can we check that? Take another look at the interface of MailingList
. There’s nothing in there that enables us to access all of the contacts. We can only get a Contact is we already know its name. Surely that’s not a good design!
It’s very common to discover flaws in a class’ interface when trying to write the unit tests. That’s because each unit test represents a kind of demonstration of how to use the class’ interface.
If you can’t write a desired unit test, you probably can’t do what you want with the class in the application either.
1.5.6 Providing access to the contacts
As a matter of Java style, when a class holds a collection of smaller items, we provide access to those by adding an iterator to provide access to the contained items and making the container class Iterable
.
So, let’s modify the MailingList
class accordingly:
public class MailingList implements Cloneable, Iterable<Contact> {
private LinkedList<Contact> theContacts;
/**
* Create an empty mailing list
*
*/
public MailingList() {
theContacts = new LinkedList<Contact>();
}
⋮
/**
* Provide access to the contacts.
*/
@Override
public Iterator<Contact> iterator() {
return theContacts.iterator();
}
}
Now, we can write a test to express the idea that the contacts are kept in order:
@Test
void testAddDuplicateContact() {
for (int i = 0; i < inputs.length; ++i) {
ml.addContact(inputs[i]);
}
MailingList ml0 = (MailingList)ml.clone();
// Add a duplicate
ml.addContact(baker);
assertEquals(ml0.size(), ml.size());
assertEquals(ml0, ml);
assertEquals(ml0.hashCode(), ml.hashCode());
assertTrue (ml.contains(baker.getName()));
assertEquals (baker, ml.getContact(baker.getName()));
assertTrue(ml.toString().contains(baker.getName()));
Contact[] expected = {adams, baker, charles, dickens, holmes};
int i = 0;
for (Contact contact: ml) {
assertEquals (expected[i], contact);
++i;
}
}
We’ll start with testAddDuplicateContact
because it’s a bit simpler.
- The
expected
array indicates the order in which we expect to see the various contacts. - The loop goes through the contacts in the mailing list one at a time, checking each one against the next element of
expected
.
Because this kind of comparison is so common, JUnit has an assertion specifically for that purpose:
@Test
void testAddDuplicateContact() {
for (int i = 0; i < inputs.length; ++i) {
ml.addContact(inputs[i]);
}
MailingList ml0 = (MailingList)ml.clone();
// Add a duplicate
ml.addContact(baker);
assertEquals(ml0.size(), ml.size());
assertEquals(ml0, ml);
assertEquals(ml0.hashCode(), ml.hashCode());
assertTrue (ml.contains(baker.getName()));
assertEquals (baker, ml.getContact(baker.getName()));
assertTrue(ml.toString().contains(baker.getName()));
Contact[] expected = {adams, baker, charles, dickens, holmes};
assertIterableEquals(Arrays.asList(expected), ml);
}
- The
Arrays.asList
call converts an ordinary array into somethingIterable
, which can then be compared directly against theIterable
variableml
.
1.5.7 testAddContactMulti - ordering test
With that, we can then add a similar ordering test to the more complicated testAddContactMulti
:
@Test
void testAddContactMulti() {
Contact[][] expected = {{dickens},
{adams, dickens},
{adams, charles, dickens},
{adams, charles, dickens, holmes},
{adams, baker, charles, dickens, holmes}
};
for (int testSize = 1; testSize <= inputs.length; ++testSize) {
MailingList ml0 = (MailingList)ml.clone();
ml.addContact(inputs[testSize-1]);
assertEquals(testSize, ml.size());
for (int j = 0; j <= testSize-1; ++j) {
assertTrue (ml.contains(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
assertEquals (inputs[j], ml.getContact(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
assertTrue(ml.toString().contains(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
}
for (int j = testSize+1; j < inputs.length; ++j) {
assertFalse (ml.contains(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
assertNull (ml.getContact(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
assertFalse(ml.toString().contains(inputs[j].getName()),
"failed after adding " + inputs[testSize-1].getName());
}
assertNotEquals(ml, ml0);
assertIterableEquals(Arrays.asList(expected[i]), ml);
}
}
- This time we have an array of expected arrays, and after adding
i
elements from theinput
, we check the mailing list againstexpected[i]
.
1.6 Parameterized Tests
That last test was a bit clunky because it was actually running a whole series of tests on mailingLists that had 1, 2, …, 5 contacts added to them. We can simplify this by using a facility called parameterized tests, which allow us to run a test repeatedly passing a different parameter to the test function each time:
@ParameterizedTest ➀
@ValueSource(ints = {1, 2, 3, 4, 5}) ➁
void testAddContactMulti(int testSize) { ➂
Contact[][] expected = {{dickens},
{adams, dickens},
{adams, charles, dickens},
{adams, charles, dickens, holmes},
{adams, baker, charles, dickens, holmes}
};
MailingList ml0 = (MailingList)ml.clone();
for (int i = 0; i < testSize; ++i) { ➃
ml.addContact(inputs[i]);
}
assertEquals(testSize, ml.size());
for (int j = 0; j <= testSize-1; ++j) {
assertTrue (ml.contains(inputs[j].getName()));
assertEquals (inputs[j], ml.getContact(inputs[j].getName()));
assertTrue(ml.toString().contains(inputs[j].getName()));
}
for (int j = testSize; j < inputs.length; ++j) {
assertFalse (ml.contains(inputs[j].getName()));
assertNull (ml.getContact(inputs[j].getName()));
assertFalse(ml.toString().contains(inputs[j].getName()));
}
assertNotEquals(ml, ml0);
assertIterableEquals(Arrays.asList(expected[testSize-1]), ml);
}
- ➀ Instead of
@Test
, we introduce this with@ParameterizedTest
. - ➁ A parameterized test must have one or more sources. The
@ValueSource
is probably the easiest one to use. It allows us to supply a stream of “basic” values (ints, strings, etc.). - ➂ The sequence of values specified in the source are passed to a function parameter in the test function. (Compare with
@Test
functions, which never take parameters.) - ➃ After that, our code is similar, but simpler because we no longer have the outer loop to explicitly iterate over different test sizes.
Parameterized tests can be useful in introducing boundary and special values testing into our test cases.
1.7 Finishing Up
1.7.1 testClone
Our test for the clone()
function starts much like you would expect it to:
@Test
void testClone() {
MailingList ml0 = new MailingList(); ➀
for (int i = 0; i < inputs.length; ++i) {
ml0.addContact(inputs[i]);
}
MailingList ml = (MailingList)ml0.clone(); ➁
assertThat (ml.size(), is(ml0.size())); ➂
assertThat (ml, equalTo(ml0));
assertThat (ml.hashCode(), equalTo(ml0.hashCode()));
assertIterableEquals(ml, ml0);
assertThat (ml.toString(), equalTo(ml0.toString()));
}
- ➀ Here we set up the parameter for the eventual
clone
call. - ➁ Here we call the function we are trying to test.
- ➂ Finally, a bunch of assertions indicating that the two mailing lists should be equal after the clone.
Checking for Deep Copy
But the documentation of the MailingList.clone
function indicates that it is supposed to carry out a “deep copy”. That means that an implementation like this:
⋮
/**
* Deep copy of the contacts.
*/
public Object clone() {
MailingList result = new MailingList();
result.contacts = contacts;
return result;
}
would not be acceptable, because any subsequent modifications of the clone would also alter the original, and vice-versa. So we should test to be sure that does not happen:
@Test
void testClone() {
MailingList ml0 = new MailingList();
for (int i = 0; i < inputs.length; ++i) {
ml0.addContact(inputs[i]);
}
MailingList ml = (MailingList)ml0.clone();
assertThat (ml.size(), is(ml0.size()));
assertThat (ml, equalTo(ml0));
assertThat (ml.hashCode(), equalTo(ml0.hashCode()));
assertIterableEquals(ml, ml0);
assertThat (ml.toString(), equalTo(ml0.toString()));
// clone() should be a deep copy, so changing one list should
// not affect the other.
int size = ml.size();
ml.removeContact(baker);
assertThat (ml0.size(), is(size));
assertThat (ml0, not(equalTo(ml)));
assertTrue (ml0.contains(baker.getName()));
assertThat (ml0.getContact(baker.getName()), equalTo(baker));
assertThat (ml0.toString(), not(equalTo(ml.toString())));
ml = (MailingList)ml0.clone();
ml0.removeContact(baker);
assertThat (ml.size(), is(size));
assertThat (ml, not(equalTo(ml0)));
assertTrue (ml.contains(baker.getName()));
assertThat (ml.getContact(baker.getName()), equalTo(baker));
assertThat (ml.toString(), not(equalTo(ml0.toString())));
}
- Here we modify the clone
ml
, and then check to be sure the originalml0
was unaffected. - Here we modify the original
ml0
, and then check to be sure that that cloneml
was unaffected.
1.7.2 removeContact & merge
We still need tests for removeContact
(both versions) and merge
. None of these require any ideas that we have not already examined, so I won’t go through them in detail. The full source code for this example is here:
package mailinglist;
/**
* A contact is a name and address.
* <p>
* For the purpose of this example, I have simplified matters
* a bit by making both of these components simple strings.
* In practice, we would expect Address, at least, to be a
* more structured type.
*
* @author zeil
*
*/
public class Contact
implements Cloneable, Comparable<Contact> {
private String theName;
private String theAddress;
/**
* Create a contact with empty name and address.
*
*/
public Contact ()
{
theName = "";
theAddress = "";
}
/**
* Create a contact
* @param nm name
* @param addr address
*/
public Contact (String nm, String addr)
{
theName = nm;
theAddress = addr;
}
/**
* Get the name of the contact
* @return the name
*/
public String getName()
{
return theName;
}
/**
* Change the name of the contact
* @param nm new name
*/
public void setName (String nm)
{
theName= nm;
}
/**
* Get the address of the contact
* @return the address
*/
public String getAddress()
{
return theAddress;
}
/**
* Change the address of the contact
* @param addr new address
*/
public void setAddress (String addr)
{
theAddress = addr;
}
/**
* True if the names and addresses are equal
*/
public boolean equals (Object right)
{
Contact r = (Contact)right;
return theName.equals(r.theName)
&& theAddress.equals(r.theAddress);
}
public int hashCode ()
{
return theName.hashCode() + 3 * theAddress.hashCode();
}
public String toString()
{
return theName + ": " + theAddress;
}
public Object clone()
{
return new Contact(theName, theAddress);
}
/**
* Compare this contact to another.
* Return value > 0 if this contact precedes the other,
* == 0 if the two are equal, and < 0 if this contact
* follows the other.
*/
public int compareTo (Contact c)
{
int nmcomp = theName.compareTo(c.theName);
if (nmcomp != 0)
return nmcomp;
else
return theAddress.compareTo(c.theAddress);
}
}
package mailinglist;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.ListIterator;
/**
* A collection of names and addresses
*/
public class MailingList implements Cloneable, Iterable<Contact> {
private LinkedList<Contact> theContacts;
/**
* Create an empty mailing list
*
*/
public MailingList() {
theContacts = new LinkedList<Contact>();
}
/**
* Add a new contact to the list. Contacts should be kept in ascending
* order with no duplicates.
*
* @param contact new contact to add
*/
public void addContact(Contact contact) {
ListIterator<Contact> it = theContacts.listIterator();
while (it.hasNext()) {
Contact c = it.next();
int compared = c.compareTo(contact);
if (compared > 0) {
it.previous();
it.add(contact);
return;
} else if (compared == 0) {
return;
}
}
theContacts.add(contact);
}
/**
* Remove one matching contact
* @param contact remove a contact equal to contact
*/
public void removeContact(Contact contact) {
ListIterator<Contact> it = theContacts.listIterator();
while (it.hasNext()) {
Contact c = it.next();
int compared = c.compareTo(contact);
if (compared > 0) {
return;
} else if (compared == 0) {
it.remove();
return;
}
}
}
/**
* Remove a contact with the indicated name
* @param name name of contact to remove
*/
public void removeContact(String name) {
ListIterator<Contact> it = theContacts.listIterator();
while (it.hasNext()) {
Contact c = it.next();
int compared = c.getName().compareTo(name);
if (compared > 0) {
return;
} else if (compared == 0) {
it.remove();
return;
}
}
}
/**
* Search for contacts
* @param name name to search for
* @return true if a contact with an equal name exists
*/
public boolean contains(String name) {
ListIterator<Contact> it = theContacts.listIterator();
while (it.hasNext()) {
Contact c = it.next();
int compared = c.getName().compareTo(name);
if (compared > 0) {
return false;
} else if (compared == 0) {
return true;
}
}
return false;
}
/**
* Search for contacts
* @param name name to search for
* @return contact with that name if found, null if not found
*/
public Contact getContact(String name) {
ListIterator<Contact> it = theContacts.listIterator();
while (it.hasNext()) {
Contact c = it.next();
int compared = c.getName().compareTo(name);
if (compared > 0) {
return null;
} else if (compared == 0) {
return c;
}
}
return null;
}
/**
* combine two mailing lists
*
*/
public void merge(MailingList anotherList) {
// For a quick merge, we will loop around, checking the
// first item in each list, and always copying the smaller
// of the two items into result
LinkedList<Contact> result = new LinkedList<Contact>();
ListIterator<Contact> thisPos = theContacts.listIterator();
ListIterator<Contact> otherPos = anotherList.theContacts.listIterator();
while (thisPos.hasNext() && otherPos.hasNext()) {
Contact thisContact = thisPos.next();
Contact otherContact = otherPos.next();
int comp = thisContact.compareTo(otherContact);
if (comp == 0) {
result.add(thisContact);
} else if (comp < 0) {
result.add(thisContact);
otherPos.previous();
} else {
result.add(otherContact);
thisPos.previous();
}
}
// Now, one of the two lists has been entirely copied.
// The other might still have stuff to copy. So we just copy
// any remaining items from the two lists. Note that one of these
// two loops will execute zero times.
while (thisPos.hasNext()) {
result.add(thisPos.next());
}
while (otherPos.hasNext()) {
result.add(otherPos.next());
}
// Now result contains the merged list. Transfer that into this list.
theContacts = result;
}
/**
* How many contacts in list?
*/
public int size() {
return theContacts.size();
}
/**
* Return true if mailing lists contain equal contacts
*/
public boolean equals(Object anotherList) {
if (!(anotherList instanceof MailingList)) {
return false;
}
MailingList right = (MailingList) anotherList;
if (size() != right.size()) { // (easy test first!)
return false;
}
ListIterator<Contact> thisPos = theContacts.listIterator();
ListIterator<Contact> otherPos = right.theContacts.listIterator();
while (thisPos.hasNext() && otherPos.hasNext()) {
Contact thisContact = thisPos.next();
Contact otherContact = otherPos.next();
if (!thisContact.equals(otherContact)) {
return false;
}
}
return true;
}
public int hashCode() {
return theContacts.hashCode();
}
public String toString() {
StringBuffer buf = new StringBuffer("{");
boolean first = true;
for (Contact c: theContacts) {
if (!first) {
buf.append(", ");
}
first = false;
buf.append(c.toString());
}
buf.append("}");
return buf.toString();
}
/**
* Deep copy of contacts
*/
public Object clone() {
MailingList result = new MailingList();
for (Contact c: theContacts) {
result.theContacts.add((Contact)c.clone());
}
return result;
}
/**
* Provide access to the contacts.
*/
@Override
public Iterator<Contact> iterator() {
return theContacts.iterator();
}
}
package mailinglist;
import static org.junit.jupiter.api.Assertions.*;
import java.time.Duration;
import java.util.Arrays;
import org.hamcrest.core.IsNull;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;
import static org.junit.jupiter.api.Assertions.*;
class TestMailingList {
Contact holmes;
Contact adams;
Contact baker;
Contact charles;
Contact dickens;
Contact adams2;
Contact[] inputs;
MailingList ml;
@BeforeEach
void setup() {
holmes = new Contact("Holmes", "21B baker St.");
adams = new Contact("Adams", "21 Pennsylvania Ave.");
baker = new Contact("Baker", "Drury Ln.");
charles = new Contact("Charles", "100 1st St.");
dickens = new Contact("Dickens", "200 2nd St.");
adams2 = new Contact("Adams", "32 Pennsylvania Ave.");
Contact[] order = {dickens, adams, charles, holmes, baker};
inputs = order;
ml = new MailingList();
}
@Test
void testMailingList() {
assertEquals (0, ml.size());
assertFalse (ml.contains("Jones"));
assertNull (ml.getContact("Smith"));
MailingList ml2 = new MailingList();
assertEquals (ml2, ml);
assertEquals (ml2.hashCode(), ml.hashCode());
assertNotNull(ml.toString());
}
@Test
void testAddContact() {
ml.addContact(holmes);
assertEquals (1, ml.size());
assertTrue(ml.contains("Holmes"));
assertFalse(ml.contains("Jones"));
assertEquals (holmes, ml.getContact("Holmes"));
assertNull (ml.getContact("Jones"));
assertNotEquals(ml, new MailingList());
MailingList ml2 = new MailingList();
ml2.addContact(holmes);
assertEquals (ml2, ml);
assertEquals (ml2.hashCode(), ml.hashCode());
assertTrue(ml.toString().contains("Holmes"));
}
@ParameterizedTest
@ValueSource(ints = {1, 2, 3, 4, 5})
void testAddContactMulti(int testSize) {
Contact[][] expected = {{dickens},
{adams, dickens},
{adams, charles, dickens},
{adams, charles, dickens, holmes},
{adams, baker, charles, dickens, holmes}
};
MailingList ml0 = (MailingList)ml.clone();
for (int i = 0; i < testSize; ++i) {
ml.addContact(inputs[i]);
}
assertEquals(testSize, ml.size());
for (int j = 0; j <= testSize-1; ++j) {
assertTrue (ml.contains(inputs[j].getName()));
assertEquals (inputs[j], ml.getContact(inputs[j].getName()));
assertTrue(ml.toString().contains(inputs[j].getName()));
}
for (int j = testSize; j < inputs.length; ++j) {
assertFalse (ml.contains(inputs[j].getName()));
assertNull (ml.getContact(inputs[j].getName()));
assertFalse(ml.toString().contains(inputs[j].getName()));
}
assertNotEquals(ml, ml0);
assertIterableEquals(Arrays.asList(expected[testSize-1]), ml);
}
@Test
void testAddDuplicateContact() {
for (int i = 0; i < inputs.length; ++i) {
ml.addContact(inputs[i]);
}
MailingList ml0 = (MailingList)ml.clone();
// Add a duplicate
ml.addContact(baker);
assertThat(ml.size(), equalTo(ml0.size()));
assertThat(ml, is(ml0)); // is() == equalTo()
assertThat(ml.hashCode(), equalTo(ml0.hashCode()));
assertTrue (ml.contains(baker.getName()));
assertThat (ml.getContact(baker.getName()), equalTo(baker));
assertTrue(ml.toString().contains(baker.getName()));
Contact[] expected = {adams, baker, charles, dickens, holmes};
assertThat(ml, contains(expected));
}
@Test
void testRemoveContactContact() {
for (int i = 0; i < inputs.length; ++i) {
ml.addContact(inputs[i]);
}
MailingList ml0 = (MailingList)ml.clone();
ml.removeContact(charles);
Contact[] expected = {adams, baker, dickens, holmes};
assertThat (ml.size(), is(ml0.size()-1));
assertThat (ml, not(equalTo(ml0)));
assertFalse (ml.contains(charles.getName()));
assertNull (ml.getContact(charles.getName()));
assertThat (ml, contains(expected));
}
@Test
void testRemoveContactString() {
for (int i = 0; i < inputs.length; ++i) {
ml.addContact(inputs[i]);
}
MailingList ml0 = (MailingList)ml.clone();
ml.removeContact(dickens.getName());
Contact[] expected = {adams, baker, charles, holmes};
assertThat (ml.size(), is(ml0.size()-1));
assertThat (ml, not(equalTo(ml0)));
assertFalse (ml.contains(dickens.getName()));
assertNull (ml.getContact(dickens.getName()));
assertThat (ml, contains(expected));
}
@Test
void testMerge() {
for (int i = 0; i < inputs.length; ++i) {
ml.addContact(inputs[i]);
}
int divider = inputs.length / 2;
MailingList ml1 = new MailingList();
for (int i = 0; i <= divider; ++i) {
ml1.addContact(inputs[i]);
}
MailingList ml2 = new MailingList();
for (int i = divider; i < inputs.length; ++i) {
ml2.addContact(inputs[i]);
}
MailingList ml0 = (MailingList)ml2.clone();
ml1.merge(ml2);
assertThat (ml1.size(), is(ml.size()));
assertThat (ml1, equalTo(ml));
assertThat (ml1.hashCode(), equalTo(ml.hashCode()));
assertIterableEquals(ml1, ml);
assertThat (ml1.toString(), equalTo(ml.toString()));
assertThat (ml2.size(), is(ml0.size()));
assertThat (ml2, equalTo(ml0));
assertIterableEquals(ml2, ml0);
for (int i = 0; i < inputs.length; ++i) {
assertTrue (ml1.contains(inputs[i].getName()));
assertThat (ml1.getContact(inputs[i].getName()), is(inputs[i]));
}
}
@Test
void testClone() {
MailingList ml0 = new MailingList();
for (int i = 0; i < inputs.length; ++i) {
ml0.addContact(inputs[i]);
}
MailingList ml = (MailingList)ml0.clone();
assertThat (ml.size(), is(ml0.size()));
assertThat (ml, equalTo(ml0));
assertThat (ml.hashCode(), equalTo(ml0.hashCode()));
assertIterableEquals(ml, ml0);
assertThat (ml.toString(), equalTo(ml0.toString()));
// clone() should be a deep copy, so changing one list should
// not affect the other.
int size = ml.size();
ml.removeContact(baker);
assertThat (ml0.size(), is(size));
assertThat (ml0, not(equalTo(ml)));
assertTrue (ml0.contains(baker.getName()));
assertThat (ml0.getContact(baker.getName()), equalTo(baker));
assertThat (ml0.toString(), not(equalTo(ml.toString())));
ml = (MailingList)ml0.clone();
ml0.removeContact(baker);
assertThat (ml.size(), is(size));
assertThat (ml, not(equalTo(ml0)));
assertTrue (ml.contains(baker.getName()));
assertThat (ml.getContact(baker.getName()), equalTo(baker));
assertThat (ml.toString(), not(equalTo(ml0.toString())));
}
}
1.8 Is this Overkill?
We’ve illustrated here the mutator/accessor coverage introduced earlier.
Many, perhaps most, ADTs provide a small number of operations that manipulate the data in a non-trivial fashion, and a large number of get/set attribute functions that, conceptually at least, simply store and retrieve a private data member.
For example, suppose we wanted to test the following ADT:
package mailinglist;
/**
* A contact is a name and address.
* <p>
* For the purpose of this example, I have simplified matters
* a bit by making both of these components simple strings.
* In practice, we would expect Address, at least, to be a
* more structured type.
*
* @author zeil
*
*/
public class Address implements Cloneable {
private String name;
private String streetAddress;
private String city;
private String state;
private String zipCode;
/**
* Create an address with all empty fields.
*
*/
public Address ()
{
name = "";
streetAddress = "";
city = "";
state = "";
zipCode = "";
}
/**
* Create an address.
*/
public Address (String nm, String streetAddr, String city,
String state, String zip)
{
name = nm;
streetAddress = streetAddr;
this.city = city;
this.state = state;
zipCode = zip;
}
/**
* @return the theName
*/
public String getName() {
return name;
}
/**
* @param theName the theName to set
*/
public void setName(String theName) {
this.name = theName;
}
/**
* @return the streetAddress
*/
public String getStreetAddress() {
return streetAddress;
}
/**
* @param streetAddress the streetAddress to set
*/
public void setStreetAddress(String streetAddress) {
this.streetAddress = streetAddress;
}
/**
* @return the city
*/
public String getCity() {
return city;
}
/**
* @param city the city to set
*/
public void setCity(String city) {
this.city = city;
}
/**
* @return the state
*/
public String getState() {
return state;
}
/**
* @param state the state to set
*/
public void setState(String state) {
this.state = state;
}
/**
* @return the zipCode
*/
public String getZipCode() {
return zipCode;
}
/**
* @param zipCode the zipCode to set
*/
public void setZipCode(String zipCode) {
this.zipCode = zipCode;
}
/**
* True if the names and addresses are equal
*/
public boolean equals (Object right)
{
Address r = (Address)right;
return name.equals(r.name)
&& streetAddress.equals(r.streetAddress)
&& city.equals(r.city)
&& state.equals(r.state)
&& zipCode.equals(r.zipCode);
}
public int hashCode ()
{
return name.hashCode() + 3 * streetAddress.hashCode()
+ 5 * city.hashCode()
+ 7 * state.hashCode()
+ 11 * zipCode.hashCode();
}
public String toString()
{
return name + ": " + streetAddress + ": "
+ city + ", " + state + " " + zipCode;
}
public Object clone()
{
return new Address(name, streetAddress, city,
state, zipCode);
}
}
You can see that, in this case, the bulk of the operations are simple gets and sets. There are a few operations that work, in some sense, on the whole ADT, mainly for the purpose of output and comparisons.
If we were to test this, we would identify the mutators (the two constructors, the five set
… functions, and the clone
function) and accessors (the five get
… functions and the toString
, equals
, and hashCode
functions). Then we would create a test function for each mutator. For example, for the setCity
function, we might write:
public class TestAddress {
final private String name0 = "John Doe";
final private String street0 = "221B Baker St.";
final private String city0 = "Podunk";
final private String state0 = "IL";
final private String zip0 = "01010";
⋮
@Test
public final void testSetCity() {
String city1 = "Norfolk"; ➀
Address addr0 = new Address(name0, street0, city0, state0, zip0);
Address addr1 = new Address(name0, street0, city0, state0, zip0);
addr1.setCity(city1); ➁
assertEquals (name0, addr1.getName()); ➂
assertEquals (street0, addr1.getStreetAddress());
assertEquals (city1, addr1.getCity());
assertEquals (state0, addr1.getState());
assertEquals (zip0, addr1.getZipCode());
assertFalse (addr1.equals(addr0));
assertTrue (addr1.toString().contains(city1));
}
This follows a pattern that should be increasingly familiar:
- ➀ Set up the data values that we will need.
- ➁ Invoke the mutator being tested.
- ➂ Evaluate and test each of the accessor functions on the mutated ADT value.
You can see that four out of the first five assertions actually assert that this value was unaffected by the mutator. Some students look at this and wonder why we bother. Isn’t that a lot of wasted code? It doesn’t seem to really be relevant to the mutator function (setCity
) that we are testing in this function. And when you consider that there will be similar “waste” in each of the other functions for testing the other mutator functions, this can seem excessive.
But there are a number of reasons why these “does not change” assertions are worth performing:
-
Our intuition is that a working implementation of
setCity
won’t have any effect on the street address, zip code, etc.. But, if we knew that the code was working, we wouldn’t be testing it!True story: I often write an ADT like this by typing out the first pair of get and set functions (e.g.,
getName
andsetName
), then copying and pasting those multiple times, and then using my editor’s search-and-replace function to alter the attribute names in the subsequent pairs.On more than one occasion, I have forgotten to change the name of the data member being retrieved by a get/set pair, winding up with a bug something like like this:
public String getCity() { return name; } public void setCity(String city) { this.name = name; }
This bug was detected by the "
getName()
does not change whensetCity
is called) assertion in thetestSetCity
test function. -
Our intuition that the behavior of “
getCity
” has no bearing on the testing of “setCity
” is based on a misunderstanding of what the test functions are doing. Although we allocate one test function per mutator, each such function is not merely testing its associated mutator. It is testing both that mutator and all the accessors.The collected tests of all of accessor functions are distributed over the entire set of test functions. If we eliminated all checks of an accessor
getX
but the one in thetestSetX
function, that accessor would wind up being woefully under-tested.On the other hand, what would be excessive would be to devote separate test functions to both mutators and accessors (depeite the fact that this is what Eclipse volunteers to do if you use its “New … Junit Test Case” helper). The only way to test an accessor is to use some mutator to set the ADT value, and we’re already doing that the the test functions for mutators.
%endif
- Our intuition that the behavior of a
getX
accessor is independent of asetY
mutator for a different attribute is colored by our assumption that these get/set functions are doing nothing but retrieving simple data members.But that’s not always the case, and, even if that’s how the ADT is implemented now, it might not be implemented that way in the future.
-
If the program is distributed, so that address values are actually being transferred over the network, we will likely receive and sometimes store addresses in a serialized form, essentially a single string.
In that case, we would need to tease out the individual field values like cities, states, etc., and the possibility of “cross-talk” among the fields due to buggy code becomes significant.
-
Similar serialized forms are common if we have a program that mixes code written in different programming langauges.
-
A similar problem can arise if the actual Address values need to be stored in and retrieved from a database. Perhaps the database is set up to combine, for example, the city and state in a single field.
These are Black-box tests, and we want them to remain valid even if the implementation changes.
-
In general, it’s always a bit dangerous to argue that tests are unnecessary based on our intuition about how the code will behave when it runs correctly. Our choice of tests really have to be more informed by the possibilities of how the code might misbehave when it has bugs.
Furthermore, these “extra” assertions are not really all that oppressive. Most of them, written once, can be copied and pasted into the next test function. For example, the testSetState
function could be:
@Test
public final void testSetState() {
String state1 = "VA";
Address addr0 = new Address(name0, street0, city0, state0, zip0);
Address addr1 = new Address(name0, street0, city0, state0, zip0);
addr1.setState(state1);
assertEquals (name0, addr1.getName());
assertEquals (street0, addr1.getStreetAddress());
assertEquals (city0, addr1.getCity());
assertEquals (state1, addr1.getState());
assertEquals (zip0, addr1.getZipCode());
assertFalse (addr1.equals(addr0));
assertTrue (addr1.toString().contains(state1));
}
The highlighted portions are the only changes from the earlier testSetCity
function.
A final comment on the “Is this excessive?” question:
- These tests are not, by any measure, a perfect or full set of tests that will detect all bugs. Having done this first pass, we would still want to look at the tests from the perspective of good black-box testing.
2 Writing Expressive Tests
One goal is writing tests is to make them both easy to read and to set them up so that the messages they issue upon failure are as helpful as possible.
2.1 Choose the most limited assertion.
These two assertions mean the same thing:
assertTrue(string1.equals(string2));
assertEquals(string1, string2);
I would argue that the second is easier to read. Moreover, if these assertions fail, the second gives a more informative message:
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;
public class testMessages {
String string1 = "abcdef";
String string2 = "abcZef";
@Test
public void test1() {
assertTrue (string1.equals(string2));
}
@Test
public void test2() {
assertEquals (string1, string2);
}
}
results in the messages:
test1: java.lang.AssertionError
at testMessages.java:14
test2: org.junit.ComparisonFailure: expected:<abc[d]ef> but was:<abc[Z]ef>
at testMessages.java:19
The second message is clearly more helpful.
You can alleviate this a bit by adding a message
assertTrue (string1.equals(string2),
"string1 is not equal to string2");
but, really, it’s still not as good.
Use
assertTrue
andassertFalse
only when more specific assertions do not apply. Then consider adding messages to them explaining what they are actually checking for.
For example, if what we really wanted to assert was that one string contained the other, then we would be stuck with
assertTrue (string1.contains(string2),
"string1 does not contain " + string2");
3 Matchers (Hamcrest)
Because the set of basic assertions is, of necessity, limited, a newer style of assertion has arisen that
- Tries to be more expressive for many common tests.
- Can be extended to new “kinds” of assertions fitted to custom ADTs.
The assertion style looks like
assertThat (object, matcher);
(Again, an optional message can be added to the front of the parameter list, but it is generally hopes that the new style reduces the need for this.)
In this new style,
assertThat (obj1, equalTo(obj2));
means the same thing as
assertEquals (obj1, obj2);
but uses the equalTo
matcher instead.
- Many people find the Hamcrest matcher style easier to read and more expressive.
- It also can yield more descriptive messages when a test does fail.
- It is also extensible – programmers can write their own matchers if they nothing suitable is at hand.
3.1 Core matchers
The Hamcrest Matchers include a variety of primitives for checking one value, inlcuding
-
x
, equalTo(
y)
: x should be equal to y.-
Sometimes abbreviated as x
.is(
y)
.
-
-
x
, instanceOf(
class)
: x should be an instance of a particular class- Sometimes abbreviated as x
.isA(
class)
.
- Sometimes abbreviated as x
-
x
, nullValue()
: x should be a null reference. - x
, notNullValue()
: x should not be a null reference. -
x
, sameInstance(
y)
: x and y should hold the same address.
3.2 Core matchers – modifiers
Other core matchers are used to modify the way that matchers are applied
-
not(
matcher)
: The matcher should return false. -
allOf(
matcher1, matcher2, …)
; all of the matchers should return true. -
anyOf(
matcher1, matcher2, …)
; at least one of the matchers should return true. -
anything()
: this matcher always returns true.
With these, we can write assertions like:
assertThat(x, not(equalTo(y));
assertThat(x, anyOf(nullValue(), not(equalTo(y)));
3.3 Additional Matchers
Also in the Hamcrest library, matchers for strings, numbers, iterable containers, etc.:
assertThat(x, lessThan(y));
assertThat(string1, isEmptyOrNullString());
assertThat(string1, startsWith("Hello"));
assertThat(myList, hasItem("Smith");
assertThat(myList, contains(smallerList);
assertThat(myList, containsInAnyOrder("Jones", "Doe", "Smith");
3.3.1 An Example in the Hamcrest Style
@Test
void testAddDuplicateContact_Hamcrest_Style() {
for (int i = 0; i < inputs.length; ++i) {
ml.addContact(inputs[i]);
}
MailingList ml0 = (MailingList)ml.clone();
// Add a duplicate
ml.addContact(baker);
assertThat(ml.size(), equalTo(ml0.size()));
assertThat(ml, is(ml0)); // is() == equalTo()
assertThat(ml.hashCode(), equalTo(ml0.hashCode()));
assertTrue (ml.contains(baker.getName()));
assertThat (ml.getContact(baker.getName()), equalTo(baker));
assertTrue(ml.toString().contains(baker.getName()));
Contact[] expected = {adams, baker, charles, dickens, holmes};
assertThat(ml, equalTo(Arrays.asList(expected)));
}
Some personal preferences:
- I still prefer
assertTrue(condition)
overassertThat(condition, is(true))
andassertFalse(condition)
overassertThat(condition, is(false))
. - I usually prefer
equalTo
rather thanis
. I will normally only useis
with constants.- That’s particularly true when I want to negate it. I just think
assertThat(x, not(equalTo(y)))
reads better thanassertThat(x, not(is(y)))
.
- That’s particularly true when I want to negate it. I just think
4 Running JUnit Tests
4.1 Eclipse
The Eclipse IDE for Java includes a copy of the JUnit library and a stripped-down (no matchers) version of Hamcrest.
- You need to add Junit to your project settings.
- Right-click on your Java project, select
Build Path
,Add Libraries...
,JUnit
, and selectJUnit5
.
If you are using Hamcrest matchers, you will need to
-
Add it to your Eclipse build path:
-
Right-click on your project and select
Build path -> Configure build path...
. -
On the
Libraries
tab, selectClasspath
and clickAdd External Jars...
.Follow the on-screen prompts to located you downloaded Hamcrest jar file.
Click
Apply
. -
On the
Order and Export
tab, select you Hamcrest jar and, if necessary move itUp
aboveJUnit 5
.
-
You should be able to run your unit tests by right-clicking on the Java file and selecting Run as -> JUnit Test
. You might need to enter the Run as -> Run Configurations
menu to confirm that you are using JUnit 5.
5 Command-Line
You will need to download the JUnit and Hamcrest libraries as .jar
files and save them somewhere convenient.
- Download the latest release version (avoid
M
andRC
versions) of the following. Click on the version number you want and then look for the “jar” link.- Junit Jupiter API
- Junit Jupiter Engine
- Junit Platform Console Standalone
- Junit Jupiter Params (if you are using any parameterized tests)
- Download the latest
hamcrest-all-*.jar
.
These four jars need to be added to your CLASSPATH
(the set of locations where Java looks for code, source or compiled). You can do this before issuing any commands by modifying your $CLASSPATH
environment variable, or in the javac
and java
commands via the -cp
option.
For the purpose of example, I am assuming you put them into into the same directory from which you will be running your javac
and `java
commands.
To compile, you would do something like this:
javac -cp junit-jupiter-api-5.6.2.jar:junit-jupiter-engine-5.6.2.jar:junit-platform-runner-1.6.2.jar:junit-jupiter-params-5.6.2.jar:hamcrest-all-1.3.jar mailinglist/*.java
If you have your jars in a different location, you will need to provide the correct path to each one.
To execute your tests, do
java -jar junit-platform-console-standalone-1.6.2.jar -cp .:junit-jupiter-api-5.6.2.jar:junit-jupiter-engine-5.6.2.jar:junit-jupiter-params-5.6.2.jar:hamcrest-all-1.3.jar -c mailinglist.TestMailingList
The -c
at the end names the test class you want to execute.
Admittedly, this is all a mess. Junit5 really seems to be designed for use inside IDEs and from inside build managers.
This will all be much easier when we have covered modern build managers.
6 C++ Unit Testing
6.1 Google Test
Google Test, a.k.a. gtest, provides a similar *Unit environment for C++
-
Download & follow instructions to prepare library
- gtest will be added to your project as source code
- easiest is to copy the files from fused-src/gtest/ files into a subdirectory gtest within your project.
-
For Eclipse support, add the Eclipse CDT C/C++ Tests Runner plugin
Example
This time, the C++ version of the mailing list.
- Source code is here.
- Unpack into a convenient directory.
- With Eclipse create a C++ project in that directory.
- Compile
- Run the resulting executable as a local C++ application
- Run as a *unit test:
- Right-click on the binary executable, select “Run as … Run configurations”.
- Select C/C++ Unit, click “New” button
- On the “C/C++ Testing” tab, select Tests Runner: Google Tests Runner
- Click “Run”
Examining the ADT
#ifndef MAILINGLIST_H
#define MAILINGLIST_H
#include <iostream>
#include <string>
#include "contact.h"
#include "mlIterator.h"
/**
A collection of names and addresses
*/
class MailingList
{
public:
typedef MLConstIterator iterator;
typedef MLConstIterator const_iterator;
MailingList();
MailingList(const MailingList&);
~MailingList();
const MailingList& operator= (const MailingList&);
// Add a new contact to the list
void addContact (const Contact& contact);
// Does the list contain this person?
bool contains (const Name&) const;
// Find the contact
const Contact& getContact (const Name& nm) const;
//pre: contains(nm)
// Remove one matching contact
void removeContact (const Contact&);
void removeContact (const Name&);
// combine two mailing lists
void merge (const MailingList& otherList);
// How many contacts in list?
int size() const;
bool operator== (const MailingList& right) const;
bool operator< (const MailingList& right) const;
// Provides iterator over contacts
const_iterator begin() const;
const_iterator end() const;
private:
struct ML_Node {
Contact contact;
ML_Node* next;
ML_Node (const Contact& c, ML_Node* nxt)
: contact(c), next(nxt)
{}
};
ML_Node* first;
ML_Node* last;
int theSize;
// helper functions
void clear();
void remove (ML_Node* previous, ML_Node* current);
friend std::ostream& operator<< (std::ostream& out, const MailingList& addr);
};
// print list, sorted by Contact
std::ostream& operator<< (std::ostream& out, const MailingList& list);
#endif
Should look familiar after the Java version
Examining the Tests
#include "mailinglist.h"
#include <string>
#include <vector>
#include "gtest/gtest.h"
namespace {
using namespace std;
// The fixture for testing class MailingList.
class MailingListTests : public ::testing::Test { ➀
public:
Contact jones; ➃
MailingList mlist;
virtual void SetUp() {
jones = Contact("Jones", "21 Penn. Ave."); ➄
mlist = MailingList();
mlist.addContact (Contact ("Baker", "Drury Ln."));
mlist.addContact (Contact ("Holmes", "221B Baker St."));
mlist.addContact (Contact ("Wolfe", "454 W. 35th St."));
}
virtual void TearDown() {
}
};
TEST_F (MailingListTests, constructor) {
MailingList ml;
EXPECT_EQ (0, ml.size()); ➁
EXPECT_FALSE (ml.contains("Jones"));
}
TEST_F (MailingListTests, addContact) {
mlist.addContact (jones); ➅
EXPECT_TRUE (mlist.contains("Jones")); ➂
EXPECT_EQ ("Jones", mlist.getContact("Jones").getName());
EXPECT_EQ (4, ml.size());
}
} // namespace
Roughly similar to the JUnit tests
-
➀ The class provides a fixture where data can be shared among test cases
- This is optional, but common
-
➁ Test cases are introduced with
TEST_F
- First argument identifies the suite (same as fixture name)
- Second argument names the test case
- The combination must be unique
- Use
TEST
if you don’t provide a fixture class
-
➂ The test cases feature assertions similar to those seen in JUnit
EXPECT
assertions allow testing to continue after failureASSERT
variations also exist that shut down testing on failure
-
➃ Public members of the fixture class will be visible to all test cases
- ➄ and are assigned the values during SetUp, run before each test case
- ➅ You can see the fixture members used here
6.2 Boost Test Framework
Boost UTF
Boost is a well-respected collection of libraries for C++.
-
Many of the new library components of C++11 were distributed in Boost for “beta test”.
-
Other, more specialized libraries will remain separately distributed.
- This include another popular *Unit framework for C++, the Boost Unit Test Framework (UTF).
-
Basic principles are similar to Google Test
- Also has some support for building trees of test suites.
Boost Test
-
The UTF can be added as a single header
#include
or, for greater efficiency when dealing with multiple suites, compiled to form a static or dynamic library. -
Easiest to start with the single header approach. Download and unpack the Boost library.
- Add the Boost
include
directory to your C++ project’s search path for system headers (#include < ... >
)- If you re using a makefile, add the
-I
compiler option, e.g.,-I /home/zeil/src/boost/include
- In Eclipse, this is
Project ... Properties ... C/C++ Build ... Settings ... GCC C++ compiler ... Includes
.
- If you re using a makefile, add the
- Add the Boost
-
For Eclipse support, use the same Eclipse CDT C/C++ Tests Runner plugin
Example
Again, the C++ version of the mailing list.
- Source code is here.
- Unpack into a convenient directory.
- With Eclipse create a C++ project in that directory.
- Add path to Boost include directory
- Compile
- Run the resulting executable as a local C++ application
- Run as a *unit test:
- Right-click on the binary executable, select “Run as … Run configurations”.
- Select C/C++ Unit, click “New” button
- On the “C/C++ Testing” tab, select Tests Runner: Boost Tests Runner
- Click “Run”
Examining the Tests
#define BOOST_TEST_MODULE MailingList test
#include "mailinglist.h"
#include <string>
#include <vector>
#include <boost/test/included/unit_test.hpp>
using namespace std;
// The fixture for testing mailing lists.
class MailingListTests { ➀
public:
Contact jones; ➁
MailingList mlist;
MailingListTests() { ➃
jones = Contact("Jones", "21 Penn. Ave.");
mlist.addContact (Contact ("Baker", "Drury Ln."));
mlist.addContact (Contact ("Holmes", "221B Baker St."));
mlist.addContact (Contact ("Wolfe", "454 W. 35th St."));
}
~MailingListTests() {
}
};
BOOST_AUTO_TEST_CASE ( constructor ) { ➄
MailingList ml;
BOOST_CHECK_EQUAL (0, mlist.size()); ➅
BOOST_CHECK (!mlist.contains("Jones"));
}
BOOST_FIXTURE_TEST_CASE (addContact, MailingListTests) { ➆
mlist.addContact (jones); ➂
BOOST_CHECK (mlist.contains("Jones"));
BOOST_CHECK_EQUAL ("Jones", mlist.getContact("Jones").getName());
BOOST_CHECK_EQUAL (4, mlist.size());
}
-
➀ This names the suite.
-
➁ The class provides a fixture where data can be shared ➂ among test cases
- Optional, but common
- Simpler in Boost than in Google
- Can be any class.
- Initialization is done in the class constructor instead of in a special function. ➃
-
➄ Test cases that don’t need anything from a fixture are introduced with
BOOST\_AUTO\_TEST\_CASE
- Argument names the test case
-
➅ The assertions have different names but are similar in function and variety to JUnit and GTest
- A full list is here.
-
➆ Test cases that need a fixture are introduced with
BOOST_FIXTURE_TEST_CASE
- Second argument identifies the fixture class
- Public members of the fixture class will be visible to test cases ➂
6.3 CppUnitLite
CppUnitLite is my own C++ test framework, with features:
- Supports the newer “matcher” style of JUnit/Hamcrest
- Lightweight – requires adding one
.h
and one.cpp
file to a project. - Works with Eclipse
- Portable: works in Linux, MacOs, and Windows with CygWin or MingW
- Unit tests are time-limited by default (Linux/MacOs only)
- But timing functions are turned off when running in a debugger.
- When running in a debugger, a breakpoint is generated automatically on test assertion failure.
Future versions of this framework are intended to offer a mocking framework that is simpler and more intuitive than currently available elsewhere.
6.3.1 Example
#include "unittest.h" ➀
#include "mailinglist.h"
#include <string>
#include <vector>
using namespace std;
Contact jones;
MailingList mlist;
void setUp()
{
jones = Contact("Jones", "21 Penn. Ave.");
mlist = MailingList();
mlist.addContact (Contact ("Baker", "Drury Ln."));
mlist.addContact (Contact ("Holmes", "221B Baker St."));
mlist.addContact (Contact ("Wolfe", "454 W. 35th St."));
}
UnitTest (constructor) { ➁
MailingList ml;
assertThat (ml.size(), is(0)); ➂
assertFalse (ml.contains("Jones"));
}
UnitTest (addContact) {
setup(); ➃
mlist.addContact (jones);
assertTrue (mlist.contains("Jones"));
assertThat (mlist.getContact("Jones").getName(), is("Jones"));
assertThat (ml.size(), is(4));
assertThat (ml, hasItem(jones)); ➄
}
- ➀ Imports the unit test framework.
- ➁ New test cases are introduced by
UnitTest
orUnitTestTimed
.All tests are time-limited by default (not supported in Windows) but
UnitTestTimed
overrides the default. -
➂ A typical assertion, in the newer “matcher” style favored by JUnit.
- ➃ In general,
UnitTestLite
aovoids creating special constructs for things that can easily be done by normal programming, such as running a setup function at the start of a test. - ➄ An example of testing a collection.
hasItem
searches any data structure that provides iterators.This could also be written as
assertThat (jones, isIn(ml));