Getting Started & Examples
The testing environment for initial unit, system and integration tests are usually performed in the development area, unless special equipment must be used, such as large machines or air cooling systems.
By the end of this semester we will be conducting acceptance tests. They will be demonstrated and staged to match the domain environment situations or conditions. Things to consider:
- Where will the final testing take place?
- In what room(s)?
- How is the room configured?
- What furniture is available?
- How many people will be involved?
- Where will the equipment be placed?
- What are the physical limitations of the room, such as power sources and permanent structures?
- Who should be positioned where?
Section 3.6 of Lab 3 is titled “Test Environment”. The content should answer these questions. It is typical to include a diagram of the room. An example from a previous semester is provided, see below.
In industry, acceptance testing, or any testing, can be an expensive and laborious effort. Planning how to prepare the room can be critical for success. It would be terrible to arrive to find that only one electrical outlet is available, and during the process, the laptops all run out of battery charge. A simple multi-plug device could have saved thousands of dollars.
2 Roles and Responsibilities
For acceptance testing at the end of the semester, it is obvious who the attendees will be: the professor, the mentor, and the team. But, is that really enough? Do we need make sure that a technical expert be present in case of an equipment failure that we do not provide? Do we want a control-room operator for example?
Knowing who will be attending is important. Knowing who cannot attend, is even more important. The team members who can be in attendance (hopefully ALL) need to prepare for their activities during the testing. They must be aware of what they are expected to do, and say, and come prepared.
Considering the use of scenarios – preplanned testing scripts the mimic occurrences typical in the problem domain. You may, for example, wish for a team member to play the role of Administrator. This person may be seated in the first row of the room, with a laptop that is connected to one of the display screens (see Environment notes). Their actions during the demonstration would all involve executing Administrator privileges to tell the story, or show the results of other actions.
Another team member may play the role of an end user. Sitting in another location of the room, with another laptop, and other responsibilities. When their portion of the script is completed, they could be replaced by a team member playing the role of a different type of user, using the same laptop and seat in the room. Students have been known to bring props, such as hats, name tags, place cards, etc…..
Section 3.7 defines the roles and responsibilities. It is expected that your team will collaborate to determine the players, their roles and responsibilities, resulting in a table. Example below.
|Mary Washington||Test Manager||Main presenter, handle questions|
|William Tell||Administrator||Demonstrate Administrator screens, Change content when needed for particular test cases, show back-end aspects when needed to prove correct actions taken|
|Jack Sprat||End User||Demonstrate guest access features, become a registered user, demonstrate registered user aspects/capabilities|
Timing is everything! You will be playing the role of an actor (see Roles and Responsibilities).
To effectively impress your customer/mentor, considerations of the order in which to present functionality are important. It is assumed that time is limited. Actually – you have 60 minutes.
Your mentor already knows the background of your problem and your approach, so re-presenting CS 410 aspects are not necessary, unless significant changes have been made.
The first activity might be to explain the testing approach: who, what, when, where, and how. This could be expected to take 5 minutes. That leaves 55 minutes for the rest. The mentor and instructor will be allowed to interrupt the demonstration at any time for clarifications or questions. So, Q & A must be built into your schedule.
It is the role of a team member to keep track of time!
Section 3.3 defines the test schedule. You are expected to determine the order in which your test categories will be demonstrated. You defined these categories in Section 3.2 last week.
Things to consider:
- Start time – typically 0:00
- Duration in minutes – some portion of the 60 minutes allowed.
- Description or Test category – as found in Section 3.2 or as per scenario
- Test cases covered – also from Section 3.2
Here is an example from the Traffic Wizard project. They did not include all considerations, nor does their table match the example in the Lab 3 requirements. Yet, their team is able to understand what is to be done, when, and for how long.
|Start Time (minutes)||Duration (minutes)||Description||Test Cases Covered|
|0:10||5||Database Demo (Driver Profile and Virtual Checkpoint)||1.1|
|0:15||10||Algorithm Unit Tests (via Simulation Console)||2.1- 2.6|
|0:25||10||Integration Simulation||3.1 – 3.5, 5.1 – 5.7|
|0:35||10||Smartphone Application Demo||4.1 – 4.6|
It is always necessary to utilize resources in testing. It might be printed documents, hardware, power cords, test data files, or……
Section 3.5 of Lab 3 defines these necessary elements. Things to consider:
- Support staff
- Test data files or materials
Each team should collaborate to identify everything that needs to be in the room to facilitate testing. Create an itemized list in any format and post on your website.
When writing Section 3.5, indicate the item and its expected use.
5 Requirements Traceability
“If you are not going to identify the defects, the customer certainly will!”
Traceability between requirements and testing is required to ensure test and validation process is complete and comprehensive
The process of documenting the links between the requirements for the system you are building and the test cases and procedures developed to implement and verify those requirements.
If a test case fails, the source of the problem could be:
- The software (coding or design)
- The hardware
- The test case
- The requirement
For example, assume Test Case 4 fails. The first step would be to examine the requirements that it is validating. In this example Test Case 4 can be traced back to Requirements 1 and 3. The next step might be to verify that the test case and the requirements are adequately written and are correct. Eventually the actual code would be evaluated.
Section 5 of Lab 3 defines this aspect. Each team will provide a paragraph a text to introduce the section. A matrix/table is the common form of documentation. The requirements (#s) make up the row headings, and the test cases (#s) make up the column headings. Wherever there is a relationship between the requirement and the test case, the intersecting cell is noted.
Every requirement must be evaluated by at least one test case. And every test case must validate at least one requirement. Meaning, every row must have an “X”, and the same for every column.