Lecture 9. Test Execution: New Feature Testing -> Quick Intro -> Test EstimatesTest execution consists of two parts:
1. New feature testing (NFT)
2. Regression testing (RT)
After the code for the coming release is ready for testing, the release engineer pushes it to test environment, where testers perform a smoke test (also called a sanity test or a confidence test) to see if the main flows of the main functionalities still work. In the case of ShareLane, we try to register once and we try to checkout once. That’s it – nothing fancy like negative testing. Please note that during a smoke test we concentrate on finding blocking issues.
Example
The smoke test fails if we cannot register a new user or if checkout cannot be completed.
The smoke test does NOT fail if a user can be registered, but the email with the registration confirmation has the wrong content, or the checkout is working, but the wrong discount is given, depending on the number of books in the shopping cart.
So, if you see non-blocking bugs during the smoke test, take a quick note, finish the smoke test, and log those bugs. The idea of a smoke test is to check the testability of the software; i.e., whether the software is ready to be tested or not.
Depending on the complexity of the system, a smoke test usually takes 5 to 30 minutes.
Usually there is no need in formal documentation for a smoke test; testers just do a quick evaluation based on their experience with the Web site. If documentation is needed (e.g., your manager wants it), you can do it in form of a checklist.
Naturally, if a blocking bug is found during a smoke test, the bug should be immediately communicated personally (following logging it into the BTS), so the developers and release engineers can start figuring out what’s wrong. Once the problem is fixed, the code is pushed to the test environment again, and another round of smoke tests is performed, and so on. Once smoke tests pass, the release engineer freezes the code and the testers start the NFT. As one of my friends says, “Testers wait for freshly written code like a pride of hungry lions waits for an antelope. Both want to tear something apart.”
BTW
Here is what I recommend that you do right before, or at the beginning of, the NFT for a major release: call a QA team meeting where each tester does a quick introduction to the features that he will be testing in the coming release. “Quick” means up to 5 minutes per person. The benefit here is that each and every tester will have a general idea about what’s coming. This improves each tester’s overall understanding of the system and may prevent “Not a bug” situations.
For example, George, who tests functionality A, finds a bug in functionality B, tested by Anitha. But George doesn’t know that it’s actually not a bug, but a legitimate functional change for that release. If there had been a team meeting, there would have been a chance that George would learn about that change and, thus, wouldn’t waste his and the developer’s time.
During the NFT we execute our test cases, log bugs into the BTS, communicate bugs to the developers, and verify bug fixes. There is nothing much to it, but let me share several quick things that you’ll find useful.
Test Estimates
Entry/Exit Criteria
Test Plan
Aggressive testing from Jason Fisher Next ->
Lecture 9. Test Execution: New Feature Testing -> Quick Intro -> Test Estimates
