Lecture 9. Test Execution: New Feature Testing -> Quick Intro -> Test Estimates -> Entry/Exit Criteria -> Test Plan -> Aggressive Testing From Jason Fisher
The test plan is the master document about the activities regarding the testing of a certain feature (or another possible subject of testing, e.g., how the system handles a load).
Let’s forget about software for a minute.
Below are two situations:
1. You have to drive to the nearest supermarket to buy some milk.
2. You have a wedding coming soon.
Does it make sense to sit and write down a plan…
– …in the first case? No, it’s absurd. Just go out and get the milk.
– …in the second case? Absolutely, because a wedding involves a large number of activities and people, and it has a significant meaning. (Of course, I’m talking about a traditional wedding, not a “let’s get married NOW” thing in Las Vegas.)
The two situations above are different from a planning perspective because they provide different answers to the following questions:
– How quickly must the project be completed?
– How many people and activities are involved in the project?
– What is the significance of the project?
Let’s get back to our start-up business. Consider the following:
– New features in start-ups are usually developed (and tested) very quickly (taking just days or weeks).
– There is usually one PM, one developer, and one tester per each feature.
– Any feature can be changed, removed, or deprecated overnight.
So, as a rule, in the start-up environment there is no need to create a test plan.
BTW
Sometimes a tester is asked to write a test plan by his manager when the manager doesn’t know what a test plan is. If you have a manager like this, ask him: “Do you want me to create a formal test plan, or do you want me to generate test cases?”
To be fair, there are situations when a separate test plan is needed; e.g., for long-term projects, or for projects that include integration with a vendor’s software.
Test plans come in many shapes and forms, but the majority of them have their roots in the test plan document introduced by ANSI/IEEE Standard 829-1983. For my projects I use a simplified version of this. You can find it in Downloads on QATutor.com. Let’s look at the main sections.
General info
Introduction
Schedule
Feature documentation
Test documentation
Things to be tested
Things not to be tested
Entry/Exit criteria
Suspension/Resumption criteria
Other things
GENERAL INFO
Author | <Name (-s) of test plan author (-s)> |
Version | <Version of test plan, e.g., 1.0> |
Last updated | <Date and time of ANY last change inside this document> |
Status (Draft/Finished) | <“Draft” is while writing the test plan. “Finished” is after the test plan is finished and ready to be used. > |
To-do list | <Things to do to finish the test plan. This section should be blank when the status is “Finished”.> |
INTRODUCTION
This is a short intro to the feature that is going to be tested*.
*For simplicity, I assume that we are testing a certain feature, but you should remember that we can also test other things – site performance, for example.
SCHEDULE
Here you specify detailed info about things that should happen, when they should happen, and who is responsible.
Example:
Date | Deliverable | Responsible person | Team |
09/12/08 | Coding is finished | Billy | Dev |
09/12/08 | Test cases are finished | George | QA |
09/26/08 | NFT and RT are finished | George | QA |
10/01/08 | Go/No Go | Linda, Billy, George | PM, Dev, QA |
FEATURE DOCUMENTATION
Here you list and provide links to all relevant documents that specify how a feature should work. Usually those documents are specs.
Example:
Title | Notes |
Spec #1288 “Redesign and Reachitecture of Checkout | Link |
“Rules on credit card transactions in North America” by Visa Corp. | Link |
TEST DOCUMENTATION
Here you list provide links to the test documentation needed to test this feature. Usually these documents are test suites.
Example:
Title | Notes |
Test Suite #4837 “Redesign and Reachitecture of Checkout | Link |
THINGS TO BE TESTED
Here you describe what you are going to test and how.
Example:
Subject of testing | Testing approach | Needs automation helper? (leave blank for “no”) |
Checkout with Visa | 1. Component testing: a. Positive testing: card valid + enough balance b. Negative testing: card is invalid/not enough balance. 2. Integration testing: a. Integration with Shopping Cart 3. System testing: Search->View book->Shopping cart->Checkout | Credit Card Generator. This tool has already been written. |
Checkout with MasterCard | see approach for Visa | see approach for Visa |
Checkout with AmEx | see approach for Visa | see approach for Visa |
THINGS NOT TO BE TESTED
Here you specify which things are not going to be tested.
Example:
Subject of testing | Why we don’t test it |
All features other then “Checkout” | We are going to test Checkout only. Other features will be tested separately during NFT and/or RT. |
Load/Performance/Realiability and security | For this release we’ll focus on feature testing only. |
ENTRY/EXIT CRITERIA
Here we specify the entry/exit criteria for the NFT of the feature.
Example:
Entry Criteria | Exit Criteria |
Code is frozen; test cases for NFT are ready to be executed | NFT and RT are done; no open P1 and P2 bugs |
SUSPENSION/RESUMPTION CRITERIA
Suspension criteria are certain conditions upon which testing must be suspended.
Resumption criteria are certain conditions upon which testing must be resumed after suspension.
Example:
Suspension Criteria | Resumption Criteria |
7 or more open P1 bugs | 6 or less open P1 bugs |
Testing is blocked | Testing is unblocked |
OTHER THINGS
Here you specify whatever else you want included in your test plan (e.g., training, hardware/software requirements, contact info, sign-off procedure, etc.) Next ->
Lecture 9. Test Execution: New Feature Testing -> Quick Intro -> Test Estimates -> Entry/Exit Criteria -> Test Plan -> Aggressive Testing From Jason Fisher