Software Testing; Introduction

Software Testing

Introduction

When Testing should occur:

Testing is sometimes incorrectly thought as an after-the –fact activity, performed after programming is done for a product. Instead, testing should be performed at every development stage of the product. Test data sets must be derived and their correctness and consistency should be monitored through out the development process. If we divide the lifecycle of software development into “requirement analysis”, “design”, “programming”, and “operation and maintenance”, then testing should accompany each of the above phases. If testing is isolated as a single phase in the cycle, errors in the problem statement or design may incur exorbitant costs. Not only must the original error be corrected, but the entire structure built upon it must also be changed. Therefore, testing should be involved through out the SDLC in order to bring a quality product. Testing activities in each phase: Requirement analysis: Determine correctness Generate functional test data Design Determine correctness and consistency Generate structural and functional test data Programming Determine correctness and consistency Generate structural and functional test data Apply test data Refine test data Operation and maintenance Retest When should testing stop “When to stop testing” is one of the most difficult questions to a test engineer.

Some of the common test stop criteria are:

All the high priority bugs are fixed. The rate at which bugs are found is too small. The testing budget is exhausted. The project duration is complete. The risk in the project is under acceptable limit.

Practically, we feel that the decision of stopping testing is based on the level of the risk acceptable to the management. As testing is a never ending process we can never assume that 100% testing has been done, we can only minimize the risk of shipping the product to client with testing done. The risk can be measured by risk analysis but for small duration/ low budget/ low resources project, risk can be deduced by simply:-

Measuring test coverage. Number of test cycles. Number of high priority bugs.

How much to test

Testing is never ending process Every testing has some associated cost Each planned testing should be evaluated for “testing cost” Vs “cost of defect”

Test Strategy

Before starting any test activities, the team lead will have to think a lot and arrive at a strategy. This will describe the approach, which is to be adopted for carrying out test activities including the planning activities. This is a formal document and the very first document regarding the testing area and is prepared at a very early stage in SDLC. This document must provide generic test approach as well as specific details regarding the project. The following areas are addressed in the test strategy document.

Test Plan

The test plan, containing the details of the testing process, has to be prepared during the project planning stage itself. This plan should contain all the details of required resources, the testing approaches to be followed, the testing methodologies, the test cases etc. The test strategy identifies multiple test levels, which are going to be performed for a project. Activities at each level must be planned well in advance and it has to be formally documented. Based on individual plans only, the individual test levels are carried out.

The plans are to be prepared by experienced people only. In all test plans the EVTX (Entry task validation exit) criteria are to be mentioned. Entry means the entry point to that phase. For example for unit testing, the coding must be complete and then only one can start unit testing. Task is the activity that is performed. Validation is the way in which the progress and correctness and compliance are verified for that phase. Exit tells the completion criteria of that phase after validation is done.

EVTX is a modeling technique for developing worldly and atomic level models. It sands for entry, task, validation and exit. It is a task based model where the details of each task are explicitly defined in a specification table against each phase.

Test Plan Format

Before taking up the testing, a test plan has to be prepared. It is preferable if the test plan is prepared during the planning stage itself and revised just before the testing process starts. A test plan format is given:

Test Plan Project Name: Estimated start date for testing: Estimated end date for testing: Actual start date for testing: Actual end date for testing: Estimated effort in person hours: Test set up (including the hardware, software environment), other peripherals required: Test personnel and their responsibility: Type of testing to be carried out: For each type of testing, the test cases are to be specified: Testing tools to be used: Test schedule for each type of test: Test cases to be executed for regression testing: