1.1. Testing

Testing

  • Testing is the act of executing a program with selected data to uncover bugs.

    • As opposed to debugging, which is the process of finding the faulty code responsible for failed tests.

Definitions

  • A failure is an execution of a program on which incorrect output was obtained.

  • A fault is a defect in the code that could lead to failures.

  • An error refers to either

    • A defect in the output produced by a program

    • A mistake made by the software developer that resulted in a fault.

Test Cases and Suites

  • A test case is a description of a set of test inputs developed for a particular objective.

    • Could be so detailed as to give the actual input data, but often simply describes the desired input data.

  • A test suite is a collection of test cases.

Choosing Test Data

Testing traditionally can be conducted in three styles

  • Black-Box testing

    • Try to choose "smart" tests based on the requirements, without looking at the code.

  • White-Box testing

    • Try to choose "smart" tests based on the structure of the code, with minimal reference to the requirements.

  • Random testing

    • Try to use directed random selection to choose tests that are "representative" of how the program will be used.

We'll discuss the first of these today and white-box later this semester.