Automating the Testing Oracle

Steven J Zeil

Last modified: Oct 20, 2020

Contents:

Earlier we introduced this model of the basic testing process.

In this lesson, we turn our attention to the oracle, the process for determining whether a test has failed.

Why Automate the Oracle?

Most of the effort in large test suites is evaluating correctness of outputs
The oracle is a decision procedure for deciding whether the output of a particular test is correct.
Humans (“eyeball oracles”) are notoriously unreliable
- better to have tests check themselves.
Regression test suites can be huge.
Modern development methods emphasize rapid, repeated unit tests
- Test-driven development: Develop the tests first, then write the code.