Stubs and Mocking
Steven J Zeil
Abstract
The *Unit frameworks that we have looked at provide powerful support for writing test drivers. But we know that scaffolding comes in two forms: drivers and stubs.
In this lesson, we look at a modern approach to stubs, mock objects. Suppose that we are testing class A
and that A
calls functions from a class B
. A mock object for B
is interface-compatible with B
, but combines a simple (possibly automatically-generated) implementation together with test instrumentation to help us determine if A
is calling upon B
properly during testing.
1 Stubs
The Lowly Stub
-
Most of what *Unit does is to provide a standard, convenient framework for writing test drivers.
-
But unit testing often requires stubs as well
- supplying values/operations to the module under test
- processing outputs from the module under test
-
We can avoid stubs by doing bottom-up integration
- reduces independence of test suites
- restricts development options
Where ADTs and Stubs Meet
In some ways, ADTs and OOP make it easier to do stubs.
A mock object is an object whose interface emulates a “real” class for testing purposes.
Mock objects may …
-
be simpler than the real thing
-
stand-in for classes not yet implemented
-
decouple test suites
-
offer mechanisms for data collection, thus improving testability
Example: Mailing List
In our MailingList tests, we relied on a Contact class.
Here is the real interface for Contact:
#ifndef CONTACT_H
#define CONTACT_H
#include <iostream>
#include <string>
#include "name.h"
#include "address.h"
class Contact {
Name theName;
Address theAddress;
public:
Contact (Name nm, Address addr)
: theName(nm), theAddress(addr)
{}
Name getName() const {return theName;}
void setName (Name nm) {theName= nm;}
Address getAddress() const {return theAddress;}
void setAddress (Address addr) {theAddress = addr;}
bool operator== (const Contact& right) const
{
return theName == right.theName
&& theAddress == right.theAddress;
}
bool operator< (const Contact& right) const
{
return (theName < right.theName)
|| (theName == right.theName
&& theAddress < right.theAddress);
}
};
inline
std::ostream& operator<< (std::ostream& out, const Contact& c)
{
out << c.getName() << " @ " << c.getAddress();
return out;
}
#endif
#ifndef NAME_H
#define NAME_H
#include <iostream>
#include <string>
struct Name {
std::string last;
std::string first;
std::string middle;
Name (std::string lastName,
std::string firstName,
std::string middleName)
: last(lastName), first(firstName),
middle(middleName)
{}
bool operator== (const Name& right) const;
bool operator< (const Name& right) const;
};
std::ostream& operator<< (std::ostream& out, const Name& addr);
#endif
#ifndef ADDRESS_H
#define ADDRESS_H
#include <iostream>
#include <string>
struct Address {
std::string street1;
std::string street2;
std::string city;
std::string state;
std::string zipcode;
Address (std::string str1,
std::string str2,
std::string cty,
std::string stte,
std::string zip)
: street1(str1), street2(str2),
city(cty), state(stte), zipcode(zip)
{}
bool operator== (const Address& right) const;
bool operator< (const Address& right) const;
};
std::ostream& operator<< (std::ostream& out, const Address& addr);
#endif
What Do We Need for the MailingList Test?
But in our tests, the only things we needed to do with contacts was
-
create them with known names
-
copy/assign them
-
getName()
-
compare them, ordering by name
A Mock Contact
So we made do with a simpler interface that
-
made it easier to create and check contacts during testing
-
but would still compile with the existing MailingList code
#ifndef CONTACT_H
#define CONTACT_H
#include <iostream>
#include <string>
typedef std::string Name;
typedef std::string Address;
class Contact {
Name theName;
Address theAddress;
public:
Contact() {}
Contact (Name nm, Address addr)
: theName(nm), theAddress(addr)
{}
Name getName() const {return theName;}
void setName (Name nm) {theName= nm;}
Address getAddress() const {return theAddress;}
void setAddress (Address addr) {theAddress = addr;}
bool operator== (const Contact& right) const
{
return theName == right.theName
&& theAddress == right.theAddress;
}
bool operator< (const Contact& right) const
{
return (theName < right.theName)
|| (theName == right.theName
&& theAddress < right.theAddress);
}
};
inline
std::ostream& operator<< (std::ostream& out, const Contact& c)
{
out << c.getName() << " @ " << c.getAddress();
return out;
}
#endif
Replacing the Name and Address classes by simple strings.
2 Mock Objects
Object-oriented languages make it easier to create some mock objects
-
Declare the mock as a subclass of the real interface
-
Override the functions as desired
Example: for an application that drew graphics using the Java java.awt.Graphics API, I created the following as a mock class:
package Pictures;
import java.awt.Color;
import java.awt.Font;
import java.awt.FontMetrics;
import java.awt.Graphics;
import java.awt.Image;
import java.awt.Rectangle;
import java.awt.Shape;
import java.awt.image.ImageObserver;
import java.text.AttributedCharacterIterator;
/**
* Special version of Graphics for testing that simply writes all graphics requests to
* a string.
*
* @author zeil
*
*/
public class GraphicsStub extends Graphics {
private StringBuffer log;
public GraphicsStub() {
log = new StringBuffer();
}
public void clearRect(int arg0, int arg1, int arg2, int arg3) {
entering("clearRect",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
}
public void clipRect(int arg0, int arg1, int arg2, int arg3) {
entering("clipRect",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
}
public void copyArea(int arg0, int arg1, int arg2, int arg3, int arg4,
int arg5) {
entering("copyArea",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4 + ":" + arg5);
}
public Graphics create() {
entering("create", "");
return new GraphicsStub();
}
public void dispose() {
entering("dispose", "");
}
public void drawArc(int arg0, int arg1, int arg2, int arg3, int arg4,
int arg5) {
entering("drawArc",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4 + ":" + arg5);
}
public boolean drawImage(Image arg0, int arg1, int arg2, ImageObserver arg3) {
entering("drawImage",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
return true;
}
public boolean drawImage(Image arg0, int arg1, int arg2, Color arg3,
ImageObserver arg4) {
entering("drawImage",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
return true;
}
public boolean drawImage(Image arg0, int arg1, int arg2, int arg3,
int arg4, ImageObserver arg5) {
entering("drawImage",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4);
return true;
}
public boolean drawImage(Image arg0, int arg1, int arg2, int arg3,
int arg4, Color arg5, ImageObserver arg6) {
entering("drawImage",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4 + ":" + arg5);
return true;
}
public boolean drawImage(Image arg0, int arg1, int arg2, int arg3,
int arg4, int arg5, int arg6, int arg7, int arg8, ImageObserver arg9) {
entering("drawImage",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4 + ":" + arg5
+ arg6 + ":" + arg7 + ":" + arg8);
return true;
}
public boolean drawImage(Image arg0, int arg1, int arg2, int arg3,
int arg4, int arg5, int arg6, int arg7, int arg8, Color arg9,
ImageObserver arg10) {
entering("drawImage",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4 + ":" + arg5
+ arg6 + ":" + arg7 + ":" + arg8 + ":" + arg9);
return true;
}
public void drawLine(int arg0, int arg1, int arg2, int arg3) {
entering("drawLine",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
}
public void drawOval(int arg0, int arg1, int arg2, int arg3) {
entering("drawOval",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
}
public void drawPolygon(int[] arg0, int[] arg1, int arg2) {
entering("drawPolygon",
"" + arg0 + ":" + arg1 + ":" + arg2);
}
public void drawPolyline(int[] arg0, int[] arg1, int arg2) {
entering("drawPolyline",
"" + arg0 + ":" + arg1 + ":" + arg2);
}
public void drawRoundRect(int arg0, int arg1, int arg2, int arg3, int arg4,
int arg5) {
entering("drawRoundRect",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4 + ":" + arg5);
}
public void drawString(String arg0, int arg1, int arg2) {
entering("drawString",
"" + arg0 + ":" + arg1 + ":" + arg2);
}
public void drawString(AttributedCharacterIterator arg0, int arg1, int arg2) {
entering("drawString",
"" + arg0 + ":" + arg1 + ":" + arg2);
}
public void fillArc(int arg0, int arg1, int arg2, int arg3, int arg4,
int arg5) {
entering("fillArc",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4 + ":" + arg5);
}
public void fillOval(int arg0, int arg1, int arg2, int arg3) {
entering("fillOval",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
}
public void fillPolygon(int[] arg0, int[] arg1, int arg2) {
entering("fillPolygon",
"" + arg0 + ":" + arg1 + ":" + arg2);
}
public void fillRect(int arg0, int arg1, int arg2, int arg3) {
entering("fillRect",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
}
public void fillRoundRect(int arg0, int arg1, int arg2, int arg3, int arg4,
int arg5) {
entering("fillRoundRect",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3 + ":" + arg4 + ":" + arg5);
}
private Shape clip = null;
public Shape getClip() {
return clip;
}
public Rectangle getClipBounds() {
return null;
}
private Color color;
public Color getColor() {
return color;
}
public Font getFont() {
return null;
}
public FontMetrics getFontMetrics(Font arg0) {
return null;
}
public void setClip(Shape arg0) {
entering("setClip",
"" + arg0);
clip = arg0;
}
public void setClip(int arg0, int arg1, int arg2, int arg3) {
entering("setClip",
"" + arg0 + ":" + arg1 + ":" + arg2 + ":" + arg3);
}
public void setColor(Color arg0) {
entering("setColor",
"" + arg0);
color = arg0;
}
public void setFont(Font arg0) {
entering("setFont",
"" + arg0);
}
public void setPaintMode() {
entering("setPaintMode", "");
}
public void setXORMode(Color arg0) {
entering("setXORMode",
"" + arg0);
}
public void translate(int arg0, int arg1) {
entering("translate",
"" + arg0 + ":" + arg1);
}
public String toString()
{
return log.toString();
}
public void clear()
{
log = new StringBuffer();
}
private void entering(String functionName, String args)
{
log.append(functionName);
log.append(": ");
log.append(args);
log.append("\n");
}
}
It basically writes each successive graphics call into a string, that can later be examined by unit test cases.
OO Mock Example
Here is a test using that mock:
package PictureTests;
import java.awt.Color;
import java.awt.Point;
import java.lang.reflect.Method;
import java.util.Scanner;
import junit.framework.*;
import junit.textui.TestRunner;
import Pictures.GraphicsStub;
import Pictures.LineSegment;
import Pictures.Shape;
/**
* Test of the LineSegment class
*/
public class LineSegmentTest extends TestCase {
⋮
public void testPlot()
{
Point p1 = new Point(22, 341);
Point p2 = new Point(104, 106);
LineSegment ls = new LineSegment(p1, p2, Color.blue, Color.red);
GraphicsStub g = new GraphicsStub();
GraphicsStub gblue = new GraphicsStub();
gblue.setColor(Color.blue);
GraphicsStub gline1 = new GraphicsStub();
gline1.drawLine(22, 341, 104, 106);
GraphicsStub gline2 = new GraphicsStub();
gline2.drawLine(104, 106, 22, 341);
ls.plot(g);
assertTrue (g.toString().contains(gblue.toString()));
assertTrue (g.toString().contains(gline1.toString())
|| g.toString().contains(gline2.toString()));
}
⋮
-
The (main) mock graphics object is created here.
-
The function being tested in called here.
-
Then, assertions like this one can test the info recrded in the mock object.
Mock Frameworks
Increasingly, unit test frameworks are including special support for mock object creation.
2.1 Google Mock (C++)
-
For example, the Google Mock Framework provides support for creating mock objects that automatically keep a log of calls made to them.
-
And adds special functions and test assertions to both test that the expected calls were made and controlling what results will be returned from those calls.
**Mocking Up an Example **
Suppose that we are writing an application that uses this class:
class Turtle {
⋮
virtual ~Turtle() {}
virtual void PenUp() = 0;
virtual void PenDown() = 0;
virtual void Forward(int distance) = 0;
virtual void Turn(int degrees) = 0;
virtual void GoTo(int x, int y) = 0;
virtual int GetX() const = 0;
virtual int GetY() const = 0;
};
To set up the test of the application code that uses this, we derive a mock class that has the same interface…
A Mock Class
#include "gmock/gmock.h"
class MockTurtle : public Turtle {
public:
...
MOCK_METHOD0(PenUp, void());
MOCK_METHOD0(PenDown, void());
MOCK_METHOD1(Forward, void(int distance));
MOCK_METHOD1(Turn, void(int degrees));
MOCK_METHOD2(GoTo, void(int x, int y));
MOCK_CONST_METHOD0(GetX, int());
MOCK_CONST_METHOD0(GetY, int());
};
- Note that both the macro names and the function signatures are related to the original interface.
- There are scripts that can sometimes generate this automatically.
Using the Mock
Then in unit tests for the application class, we
- use instances of the mock class
- Write assertions that describe how we expect the stub to be exercised.
#include "mock-turtle.h"
#include "gmock/gmock.h"
#include "gtest/gtest.h"
using ::testing::AtLeast;
TEST(DrawACircle, CanDrawSomething) {
MockTurtle turtle;
EXPECT_CALL(turtle, PenDown())
.Times(AtLeast(1));
Painter painter(&turtle);
EXPECT_TRUE(painter.DrawCircle(0, 0, 10));
}
So, if we call painter.drawCircle(...)
, we
- expect it to return
true
- expect it to call
tutle.PenDown()
at least once.
Stubs can Generate Output
EXPECT_CALL(g, foo(_))
.TIMES(3)
.WillRepeatedly(Return(150));
EXPECT_CALL(g, foo(1))
.WillOnce(Return(100));
EXPECT_CALL(g, foo(2))
.WillOnce(Return(200));
This assertion says
- a function
g.foo(x)
will be called 5 times,- with x being 1 and 2, one time each
- and an unspecified value on the other three calls
- It also specifies what value should be returned on each call.
-
If the function is called 6 times, or if one or more of the 5 expected calls are not made before the test case ends, the
EXPECT_CALL
assertion fails, just like other test assertions.
2.2 JMockit (Java)
There are a number of popular mocking frameworks for Java: JMock, EasyMock, Mockito
JMockit is one of newer ones.
- IMO, it seems one of the more elegant.
- Although recent trends suggest that Mockito has become more popular.
2.2.1 The Record-Replay-Verify Model
A test with JMockit mock objects typically follows a structure of
- Record
- We establish our expectations of what calls will be made on our mock objects by separately executing those calls in a recording mode.
In this phase we also record any return/output values we want our mocks to return from those calls.
- Replay
- We actually perform the test, calling upon some functions of the ADT under test that, in the run, we expect will make those calls on the mocks. The actual calls made are compared against the “recording” we made earlier.
We fail the test if we do not see the expected (recorded) calls.
- Verify
- More detailed checks are made to see if the test matched criteria other than simply having called the expected functions in the expected order.
We may check the parameters supplied to those calls and/or check to see that other mock functions were not called.
3 Case Studies
3.1 Example 1 (Spreadsheet CLI)
In our spreadsheet example, we had these stories in the first increment:
- As a calculation author I would like to load a spreadsheet via the CLI.
- As a calculation author I would like to generate a value report via the CLI.
I decided to merge these as, together, they represented the minimum testable behavior on the CLI.
However, during this first project increment, the spreadsheet itself has not been implemented.
I had, for an earlier story, taken a first pass at designing the spreadsheet API:
package edu.odu.cs.espreadsheet;
⋮
public class Spreadsheet implements Iterable<CellName> {
public Spreadsheet()
{
// TODO
}
⋮
/**
* Loads a spreadsheet from a file or other input source. Input format is
* one assignment per line.
*
* @param input an input source
* @throws IOException if assignments cannot be parsed form the input.
*/
public void load (Reader input) throws IOException
{
// TODO Auto-generated method stub
}
⋮
/**
* Write out the value report for the spreadsheet, giving the current value of
* each non-empty cell.
*
* @param output output device
*/
public void report (Writer output)
{
// TODO Auto-generated method stub
}
}
The CLI
I expected the CLI to look something like:
package edu.odu.cs.espreadsheet.cli;
⋮
import edu.odu.cs.espreadsheet.Spreadsheet;
public class Run {
⋮
public Run(String filename)
{
// TODO
}
/**
* Load the indicated spreadsheet and generate its value report.
* @throws IOException if the spreadsheet cannot be loaded
*/
public void doIt() throws IOException
{
⋮
}
public static void main (String[] args) throws FileNotFoundException, IOException {
if (args.length != 1) {
System.err.println("Usage: must supply a path to a spreadsheet file as a command line parameter.");
System.exit(-1);
}
new Run(args[0]).doIt();
}
}
because this is a pretty standard pattern for me.
Testing With a Spreadsheet
Now, if I had been working bottom-up, then by the time I looked at the CLI, I would have a working spreadsheet and could write some “real” tests of the CLI:
package edu.odu.cs.espreadsheet.cli;
import static org.junit.Assert.assertTrue;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.PrintStream;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import org.junit.BeforeClass;
import org.junit.Test;
public final class ITestCLI {
static Path testDataDir = Paths.get("target/itest-data");
@BeforeClass
public static void setupDir()
{
File dir = testDataDir.toFile();
dir.mkdir();
}
@Test
public void testRunNonExistentFile() throws FileNotFoundException, IOException {
Run running = new Run("target/testdata/nonExistent.ss");
boolean thrown = false;
try {
running.doIt();
} catch (FileNotFoundException ex) {
thrown = true;
}
assertTrue (thrown);
}
@Test
public void testBadFile() throws FileNotFoundException, IOException {
Path badDataPath = testDataDir.resolve("badData.ss");
Files.write(badDataPath,
Arrays.asList("a12", "ab2=1"),
Charset.forName("US-ASCII"));
Run running = new Run(badDataPath.toString());
boolean thrown = false;
try {
running.doIt();
} catch (IOException ex) {
thrown = true;
}
assertTrue (thrown);
}
@Test
public void testSimpleAssignments() throws FileNotFoundException, IOException {
Path simpleDataPath = testDataDir.resolve("simpleData.ss");
Files.write(simpleDataPath,
Arrays.asList("a12=12", "ab2=1"),
Charset.forName("US-ASCII"));
// Capture System.out
ByteArrayOutputStream baos = new ByteArrayOutputStream();
PrintStream stringStream = new PrintStream(baos);
PrintStream savedOut = System.out;
System.setOut (stringStream);
try {
Run running = new Run(simpleDataPath.toString());
running.doIt();
} finally {
//Restore System.out
System.out.flush();
System.setOut(savedOut);
}
String reportOutput = baos.toString();
assertTrue (reportOutput.startsWith("a12=12"));
assertTrue (reportOutput.contains("ab2=1"));
}
@Test
public void testEvaluationPerformed() throws FileNotFoundException, IOException {
Path simpleDataPath = testDataDir.resolve("evalData.ss");
Files.write(simpleDataPath,
Arrays.asList("a12=10*ab2", "ab2=2"),
Charset.forName("US-ASCII"));
// Capture System.out
ByteArrayOutputStream baos = new ByteArrayOutputStream();
PrintStream stringStream = new PrintStream(baos);
PrintStream savedOut = System.out;
System.setOut (stringStream);
try {
Run running = new Run(simpleDataPath.toString());
running.doIt();
} finally {
//Restore System.out
System.out.flush();
System.setOut(savedOut);
}
String reportOutput = baos.toString();
assertTrue (reportOutput.startsWith("a12=20"));
assertTrue (reportOutput.contains("ab2=2"));
}
}
This has tests for the cases:
- Input file does not exist
- Input file exists but has parsing errors
- Input file OK, contains simple numeric assignments
- Input file OK, actually requires some expression evaluation and references to other cells.
- I’m not attempting here to thoroughly test all value kinds and expression operators.
- That would take place i nthe tests of those other classes (
Value
,Expression
, etc.)
Problems with that Test
- I won’t be able to pass it until we are near the end of the overall implementation.
- It could be passed by other means (e.g., if the CLI used the Spreadsheet
save
operation instead ofreport
, butsave
was buggy and listed values instead of expressions). - It is heavily dependent on exact I/O formatting and other details that might change when we consider the relevant classes in more detail.
- i.e., it is dependent on the
Spreadsheet
and other classes.
- i.e., it is dependent on the
This is actually an integration or system test, not a unit test.
Unit Testing Without a Spreadsheet
For a unit test of the CLI, I can mock the spreadsheet:
package edu.odu.cs.espreadsheet.cli;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.Reader;
import java.io.Writer;
import mockit.Expectations;
import mockit.Mocked;
import org.junit.Test;
import edu.odu.cs.espreadsheet.Spreadsheet;
public final class TestCLI {
@Mocked Spreadsheet ss; ➀
@Test
public void testRun() throws FileNotFoundException, IOException {
// Record phase: expectations on mocks are recorded; empty if there is nothing to record.
new Expectations() {{ ➁
ss.load(withAny((Reader)null)); ➂
ss.report(withAny((Writer)null)); ➃
}};
// Replay phase: invocations on mocks are "replayed"; here the code under test is exercised.
Run running = new Run("ivy.xml"); ➄
running.doIt();
// Verify phase: expectations on mocks are verified; empty if there is nothing to verify.
➅
}
}
-
➀ Here I announce that I am mocking the spreadsheet class.
- JMockit does not mock just this variable – it mocks all Spreadsheet objects in this test.
- A different approach than used in other mocking frameworks.
- JMockit does not mock just this variable – it mocks all Spreadsheet objects in this test.
-
➁ The
Expectations
block starts the recording.- There are different varieties of Expectations. This one indicates that we expect to see the replayed function calls in the order we list them here. Other varieties allow the calls to occur in arbitrary order.
-
➂ First we expect to see a
load
call made on the spreadsheet. Because JavaReader
classes can’t be compared for equality, the test says that anyReader
value is acceptable. -
➃ Later we expect to see a
report
call made on the spreadsheet. -
➄ Now we do the actual test.
-
➅ We don’t need any additional verification at the end.
Are We OK With This?
You might argue that this test is
-
Simple
-
Not necessarily a bad thing. It serves as a pretty clear specification of what we expect the CLI to do.
-
-
Not very thorough
- It does not check to be sure that there aren’t defects in the
Spreadsheet
- But that’s the job of the future
Spreadsheet
unit tests, not of the CLI unit test. - It does not check for unexpected bad interactions between the CLI and the Spreadsheet.
- But interactions between classes are the province of integration tests, not unit tests.
- And we don’t throw out that first test. It’s an integration test for use later when the
Spreadsheet
is actually implemented.
- It does not check to be sure that there aren’t defects in the
3.2 Example 2: Adding Numeric Literals to the Spreadsheet
Shortly after the prior example, I had the story:
As an API user I would like to add a numeric literal to a cell in a spreadsheet.
The task list I came up with for this was:
- Check Spreadsheet API
- Design Cell API
- Design Expression API
- Add Unit test to Spreadsheet
- Add Integration test to Spreadsheet
- Add Unit test to Expression
- Add Unit test to Cell
- Implement Expression parse for numeric literals
- Implement Cell storage of expressions
- Implement Spreadsheet expression load
- Check in changes.
That may look like a lot, but it boils down to a common pattern:
- Make changes to the API to support the new functionality
- Write Unit tests for the new functionality
- If any of those tests used stubs or mocks, write integration or system tests for the behavior.
- Implement the new functionality
- Wrap things up
We expect that, at the end of step 4, we will pass all the new units tests devised in step 2.
- We may not be able to pass the new integration & system tests until other parts of the system have been finished.
3.2.1 Adding to the APIs
I made a conscious decision not to expose the internal Expression
class as part of the API:
old
package edu.odu.cs.espreadsheet;
⋮
public class Spreadsheet implements Iterable<CellName> {
⋮
/**
* Get the formula stored in the indicated cell.
*
* @param cell a cell in the spreadsheet
* @return the formula in that cell, or null if no formula has been placed there
*/
public Expression getFormula (CellName cell)
{
⋮
}
/**
* Assigns a formula to the indicated cell.
*
* @param cell a cell in the spreadsheet
* @param newFormula an expression for future evaluation (may be null)
* @throws ExpressionParseError if assignments cannot be parsed from the input.
*/
public void assignFormula (CellName cell, Expression newFormula) throws ExpressionParseError
{
⋮
}
⋮
}
new
package edu.odu.cs.espreadsheet;
⋮
public class Spreadsheet implements Iterable<CellName> {
⋮
/**
* Get the formula stored in the indicated cell.
*
* @param cell a cell in the spreadsheet
* @return the formula in that cell, or null if no formula has been placed there
*/
public String getFormula (CellName cell)
{
⋮
}
/**
* Assigns a formula to the indicated cell.
*
* @param cell a cell in the spreadsheet
* @param newFormula an expression for future evaluation (may be null)
* @throws ExpressionParseError if assignments cannot be parsed from the input.
*/
public void assignFormula (CellName cell, String newFormula) throws ExpressionParseError
{
⋮
}
⋮
}
A welcome side effect of this change is to make independent testing of these classes even easier.
Cells contain Expressions
package edu.odu.cs.espreadsheet;
import edu.odu.cs.espreadsheet.expressions.Expression;
import edu.odu.cs.espreadsheet.values.Value;
/**
* A single cell withing a spreadsheet.
*
* @author zeil
*
*/
public
class Cell
{
private Spreadsheet ssheet;
private CellName name;
private Expression formula;
private Value value;
/**
* Create a new cell
* @param sheet the spreadsheet containing this cell
* @param name the name of this cell
*/
public Cell (Spreadsheet ssheet, CellName name)
{
this.ssheet = ssheet;
this.name = name;
this.formula = null;
this.value = null;
}
public CellName getName()
{
return name;
}
/**
* Get the formula associated with this cell.
*
* @return an expression or null if the cell is empty
*/
public Expression getFormula()
{
return formula;
}
public void putFormula(Expression e)
{
formula = e;
value = null;
}
public Value getValue()
{
if (value == null && formula != null)
value = formula.evaluate(ssheet);
return value;
}
public String toString()
{
return name.toString() + "::" + formula + "::" + value;
}
public int hashCode()
{
return name.hashCode();
}
public boolean equals(Object obj)
{
if (obj instanceof Cell) {
Cell other = (Cell)obj;
return name.equals(other.name);
} else
return false;
}
}
Expressions can be Evaluated
package edu.odu.cs.espreadsheet.expressions;
import java.io.StringReader;
import edu.odu.cs.espreadsheet.ExpressionParseError;
import edu.odu.cs.espreadsheet.Spreadsheet;
import edu.odu.cs.espreadsheet.values.Value;
/**
* Expressions can be thought of as trees. Each non-leaf node of the tree
* contains an operator, and the children of that node are the subexpressions
* (operands) that the operator operates upon. Constants, cell references,
* and the like form the leaves of the tree.
*
* For example, the expression (a2 + 2) * c26 is equivalent to the tree:
*
* *
* / \
* + c26
* / \
* a2 2
*
* @author zeil
*
*/
public abstract class Expression implements Cloneable
{
/**
* How many operands does this expression node have?
*
* @return # of operands required by this operator
*/
public abstract int arity();
/**
* Get the k_th operand
* @param k operand number
* @return k_th operand if 0 < k < arity()
* @throws IndexOutOfBoundsException if k is outside of those bounds
*/
public abstract Expression operand(int k) throws IndexOutOfBoundsException;
/**
* Evaluate this expression, using the provided spreadsheet to resolve
* any cell referneces.
*
* @param usingSheet spreadsheet form which to obtain values of
* cells referenced by this expression
*
* @return value of the expression or null if the cell is empty
*/
public abstract Value evaluate(Spreadsheet usingSheet);
/**
* Copy this expression (deep copy), altering any cell references
* by the indicated offsets except where the row or column is "fixed"
* by a preceding $. E.g., if e is 2*D4+C$2/$A$1, then
* e.copy(1,2) is 2*E6+D$2/$A$1, e.copy(-1,4) is 2*C8+B$2/$A$1
*
* @param colOffset number of columns to offset this copy
* @param rowOffset number of rows to offset this copy
* @return a copy of this expression, suitable for placing into
* a cell (ColOffSet,rowOffset) away from its current position.
*
*/
public abstract Expression clone (int colOffset, int rowOffset);
/**
* Copy this expression.
*/
@Override
public Expression clone ()
{
return clone(0,0);
}
/**
* Attempt to convert the given string into an expression.
* @param in
* @return
*/
public static Expression parse (String in) throws ExpressionParseError
{
try {
parser p = new parser(new ExpressionScanner(new StringReader(in)));
Expression expr = (Expression)p.parse().value;
return expr;
} catch (Exception ex) {
throw new ExpressionParseError("Cannnot parse " + in);
}
}
@Override
public String toString ()
{
⋮
}
@Override
public abstract boolean equals (Object obj);
@Override
public abstract int hashCode ();
// The following control how the expression gets printed by
// the default implementation of toString
/**
* If true, print in inline form.
* If false, print as functionName(comma-separated-list).
*
* @return indication of whether to print in inline form.
*
*/
public abstract boolean isInline();
/**
* Parentheses are placed around an expression whenever its precedence
* is lower than the precedence of an operator (expression) applied to it.
* E.g., * has higher precedence than +, so we print 3*(a1+1) but not
* (3*a1)+1
*
* @return precedence of this operator
*/
public abstract int precedence();
/**
* Returns the name of the operator for printing purposes.
* For constants/literals, this is the string version of the constant value.
*
* @return the name of the operator for printing purposes.
*/
public abstract String getOperator();
}
A lot of this design was lifted from an earlier project of mine.
- Expression is an abstract class – we never expect to see any actual objects of this type.
-
But we will have lots of interesting subclasses (one per operator) that will have “real” instances.
-
Numeric Literals
This story requires just enough expression handling to deal with numeric literals. So we’ll need an API for numeric literals. We’ve already covered that, however.
3.2.2 Writing the Unit tests
NumericLiteral
NumericLiteral
needs a good set of unit test cases. We’ve looked at those tests earlier.
The NumericLiteral test is a good example of unit testing, but did not use stubs or mocks.
- It is, in effect, a bottom-level class whose implementation does not require that we write any additional classes.
Cells
Although I had a task to create a unit test for Cell
, currently it really only has encapsulated data member get/set functions, so I gave that a pass.
Spreadsheet
An existing unit test for spreadsheets was augmented to cover this story of inserting numeric literals.
/**
*
*/
package edu.odu.cs.espreadsheet;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertTrue;
import mockit.Expectations;
import mockit.Mocked;
import org.junit.Test;
import edu.odu.cs.espreadsheet.expressions.Expression;
import edu.odu.cs.espreadsheet.expressions.NumericLiteral;
/**
* @author zeil
*
*/
public class MockTestSpreadSheet {
@Mocked Expression expr;
@Test
public final void testConstructor() {
Spreadsheet s = new Spreadsheet();
assertEquals (0, s.getDimension());
assertNull (s.getFormula(new CellName("a1")));
assertFalse (s.iterator().hasNext());
}
@Test
public final void testNumericLiteralInsertion() throws ExpressionParseError {
final Spreadsheet s = new Spreadsheet();
final String input = "1.234";
new Expectations() {{
Expression.parse(input); result = new NumericLiteral(input);
expr.toString(); result = "1.234";
}};
CellName cn = new CellName("a1");
assertNull (s.getFormula(cn));
assertNull (s.getValue(cn));
s.assignFormula(cn, input);
assertEquals (input, s.getFormula(cn));
}
@Test(expected = ExpressionParseError.class)
public final void testParsingError() throws ExpressionParseError {
final Spreadsheet s = new Spreadsheet();
final String input = "1.23.4";
new Expectations() {{
Expression.parse(input); result = new ExpressionParseError();
}};
CellName cn = new CellName("b2");
assertNull (s.getFormula(cn));
assertNull (s.getValue(cn));
s.assignFormula(cn, input);
assertTrue(false);
}
}
-
Two new test cases added to an existing test for spreadsheets
-
Inserting any formula/expression is done by passing a string to the spreadsheet, which must parse that string to yield the actual expression.
- Parsing would not be implemented yet in this story.
- Calls for a Mock!
New Expression test case 1
Let’s break this testcase down:
public final void testNumericLiteralInsertion() throws ExpressionParseError {
final Spreadsheet s = new Spreadsheet();
final String input = "1.234";
new Expectations() {{ ➀
Expression.parse(input); result = new NumericLiteral(input);
expr.toString(); result = "1.234";
}};
CellName cn = new CellName("a1");
assertNull (s.getFormula(cn));
assertNull (s.getValue(cn));
s.assignFormula(cn, input); ➁
assertEquals (input, s.getFormula(cn)); ➂
}
-
➀ The
Expectations
block is our recording phase. -
We expect that, once we try to assign a formula (a string) to a cell (➁ ) that the
Expression.parse
function will be called to convert the string into an expression.-
That
parse
function isn’t going to be implemented yet. But we expect that the result of parsing theinput
string for this test should simply be aNumericLiteral
.The
result =
establishes what return value should be supplied when we “play back” this recording.
-
- Back to ➀ , again – we expect that when we call
getFormula
(➂ ), this process will need ot be reversed and theExpression
toString()
function will be called to convert an expression back into string form.
New Expression test case 2
The other new test case starts with a similar structure:
@Test(expected = ExpressionParseError.class)
public final void testParsingError() throws ExpressionParseError {
final Spreadsheet s = new Spreadsheet();
final String input = "1.23.4";
new Expectations() {{
Expression.parse(input); result = new ExpressionParseError();
}};
CellName cn = new CellName("b2");
assertNull (s.getFormula(cn));
assertNull (s.getValue(cn));
s.assignFormula(cn, input);
assertTrue(false);
}
But in this case we are supplying an invalid input:
-
Here we tell JMockIt that we expect the spreadhseet to invoke the parse operation, but that we want the simulated parser to throw an exception.
-
Here we are telling JUnit that the expected outcome of this test case is that an exception will be thrown (and not caught).
3.2.3 Integration Tests
Because one of our new unit tests used mocks, we should add an integration to system test now that we will fail now but will pass some time in the future when the missing modules have been implemented:
package edu.odu.cs.espreadsheet;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertTrue;
import java.io.IOException;
import java.io.StringReader;
import org.junit.Test;
import edu.odu.cs.espreadsheet.values.NumericValue;
import edu.odu.cs.espreadsheet.values.Value;
/**
* @author zeil
*
*/
public class ITestSpreadSheet {
@Test
public final void testConstructor() {
Spreadsheet s = new Spreadsheet();
assertEquals (0, s.getDimension());
assertNull (s.getFormula(new CellName("a1")));
assertFalse (s.iterator().hasNext());
}
@Test
public final void testNumericLiteralInsertion() throws ExpressionParseError {
final Spreadsheet s = new Spreadsheet();
CellName cn = new CellName("a1");
assertNull (s.getFormula(cn));
assertNull (s.getValue(cn));
s.assignFormula(cn, "1.234");
assertEquals ("1.234", s.getFormula(cn));
Value v = s.getValue(cn);
assertNotNull(v);
assertTrue (v instanceof NumericValue);
NumericValue nv = (NumericValue)v;
assertEquals (1.234, nv.getDouble(), 0.0001);
}
@Test
public final void testParsingError() {
final Spreadsheet s = new Spreadsheet();
CellName cn = new CellName("b2");
assertNull (s.getFormula(cn));
assertNull (s.getValue(cn));
boolean thrown = false;
try {
s.assignFormula(cn, "1.23.4");
} catch (ExpressionParseError e) {
thrown = true;
}
assertTrue (thrown);
assertNull (s.getFormula(cn));
Value v = s.getValue(cn);
assertNull(v);
}
}
- Basically, all we did here was to strip out the mocking portions of the unit tests!
4 Are Mock Frameworks Useful?
-
My experience with them is still somewhat limited.
-
When they work, they seem wonderful.
-
many professionals swear by them
-
-
Some mock frameworks impose strong limits on the design of the classes that can be mocked
-
Mock proponents argue that this is a virtue, enforcing what they see as preferred design styles.
-
I’m not convinced.
- One reason I prefer JMockit to other Java mock frameworks is that it imposes far fewer limitations on the design of the classes to be mocked.
-
-
Most of the published examples and tutorials focus on primitive types for function parameters and return values.
- My experience suggests that functions taking ADTs as parameter values or returning ADTs can be handled. It’s a little more complicated, but not terribly so.
4.1 Maybe Stubs aren’t so Bad?

If we are taking a “vertical” incremental approach to implementation, we may not need stubs as scaffolding…
- If we only need a few functions out of a class to support our current story, we might just go ahead and write those.
- Then we won’t need stubs for that during testing.
- We may choose to supply
- “functional” stubs that fake behavior for demonstration purposes, or
- quick and dirty algorithms that give a correct answer but might be slow or memory-hogs.
Not Really Unit Testing, Is it?
With this incremental approach, we’re really doing integration testing, not unit testing.
-
But I’m OK with that, and from what I can see, most practitioners don’t even think about the difference.
-
I still like mocks for testing CLIs & user interfaces, but I generally only use them if they actually save me time and effort rather than costing me extra time.