Test Director; Introduction

Test Director Overview

Introduction

Now a day’s very organization is giving a lot of importance to process oriented software development. To deliver quality software, the testing process has to be very well defined and managed. Mercury’s Test Director is an excellent tool for managing the testing process effectively. As it is a web based tool, it is very easy to use. Also, even if the development team and testing team are located at different places, the testing process can be managed effectively. The important features of Test Director are listed below:

It is a web based tool and hence it facilitates distributing testing. As testing the software is linked to the requirements of the software, it provides the feature of linking the software requirements to the testing plan. It provides the features to document the testing procedures. It provides the feature of scheduling the manual and automated test- the testing can be done during nighttimes or when the system load is less. It provides the feature of setting groups of machines to carry out testing. It keeps a history of all the test runs. The audit trail feature allows keeping track of changes in the tests and test run. It keeps a log of all defects found and the status of each bug can be changed by authorized person only. It provides feature of creating different users with different privileges. It generates test reports and analysis for the QA manager to decide when the software can be released in to the market.

Testing Management Process

Test Director Overview

Testing Management Process

While using the Test Director, the testing management process can be defined using the following four steps:

Testing requirements Design and develop test Run the test in manual mode or automatic mode Analyze the defect Accordingly, Test Director Use can be divided in to four phases. Test requirement management Test planning Test execution Test result analysis

Test requirement management

Requirement manager is used to link the requirements with the test to be carried out. Each requirement in the SRS has to be tested at least once. In the SRS, the functional requirements and performance requirements are specified. Functional requirements are generated from use case scenarios. Performance requirements are dependent on the application.

Test Planning

In test planning, the QA manger does a detailed planning and addresses the following issues:

Hardware and software platforms on which the testing has to be carried out. The various test to be performed (functional/ regression testing, performance testing etc.) Time schedule for conducting the test Roles and responsibilities of the person associated with the project. Procedure for running the tests. (manual or automatic) Various test cases to be generated. Procedure for tracking the progress of testing. Documents to be generated during the testing process. Criteria for completion of testing.

During the test planning stage, test design is done which involves defining the sequence of steps to execute a test in manual testing. This is the most challenging task for test engineers as the test cases have to be created intelligently to uncover the possible bugs. The test engineers also identify the common test scripts that can be reused to test different modules and map the workflow between tests.

The test plan is communicated to all the test engineers and also the development team.

Test Execution

The actual testing is carried out based on the test cases generated, either manually or automatically. In the case of automated testing, the test scheduling is done as per the plan. A history of all test runs is maintained and audit trail, to trace the history of tests and test runs, is also maintained.

During this phase, test sets are created. A test set is a set of test cases. For example, a login test set is the set of test cases to test the login process. In addition, execution logic is also set. The logic specifies what to do when a test fails in a series of tests. Consider the sequence of tests- login to a database, update a record and logout. Suppose during testing, the login test itself fails. One alternative is to stop the testing completely, the other alternative is to go ahead with the testing, but report that login test has failed.

Test Result Analysis

In this phase, the test results are analyzed which tests passed and which test failed. For the tests, an analysis has to be carried out as to why they failed. Also, each bug is classified based on its severity. A simple way of classification is:

Critical Major Minor A more detailed way of classification is: Cosmetic or GUI related Inconsistent performance of the application Loss of functionality System crash Loss of data Security violation

When a bug is reported to the developer, it is not enough if you inform that there is a bug. You need to give additional information such as what is the problem, what is the system configuration on which the test was run, what is the version of the software, which testing tool is used, what is the step by step procedure to reproduce the problem.

The bug report was stored in a database. The privileges to read write and update database need to be decided by QA manager. For example, the test engineer will update the database to indicate the new bug and give the status as “new”. The developer may remove the bug and update the status as “cleared”. The QA manager may change the status to “approve” or “to be fixed”. It is also possible to assign priorities to the bugs, critical bugs have high priority as compare to cosmetic bugs.

Based on the bug tracking and analysis tools, the QA manager and the project manager can take the decision whether the software can be released to the customer or still more testing is required.

Managing the Testing process using Test Director

Test Director Overview

Managing the Testing process using Test Director

In this section we will study how to use test director to manage the entire testing process. We will study how to manage and control various phases of software testing process. We will discuss the three phases:

Planning the tests: In this phase, we specify the testing goals, define what has to be tested, define the tests and define the detailed step by step procedure for each test. Running the test: In this phase, we will run or execute the test as planned in earlier phase. Tracking the defects: in this phase, we will analyze the test results, identify the defects and study the cause of defects.

Creating a new project

You can create a new project using the “project administrator” utility for retrieving and storing the information related to your tests. To create new project, follow the following steps below:

Open the “project administrator” utility. START>PROGRAMS>TEST DIRECTOR>PROJECT ADMINISTRATOR. Enter login name and password, then click OK button to login. Create a new project PROJECT>NEW PROJECT On selecting the new project option, a dialog box is displayed which prompts you to enter the name of project. Enter the project name, select the location and select the Database type.

Click the “create” button to create the new project with the specified details. Once the project is successfully created, it displays the message “create new project finished successfully”. Click the OK button.

Creating test plan

We will now create a test plan and store their corresponding results.

Open Test Director Start>programs> test director>test director This will open a dialog box. Select the required project from the project combo box. Enter the user name as admin. If you login in as admin, there is no need of password. Click Ok button to login. When you will click ok button, the initial screen of test director will open. Select the plan test tab. Creating the user. To create the users and to assign the privileges to each user, follow the steps: Open the administration window. Administration>set up users. When you select this option, a dialog box opens. Select the users tab in it. By default two users are created: “admin” and “guest” with privileges. To create new users click on the “new user” button. When you click on the “new” button, it displays the dialog box prompting you to enter the login (username), full name and the password. Enter the login, full name and password. Click ok button to add the new user to the user list in the set up users. To assign privileges to new user i.e. john: From the user table, select the user “john”. By default the user will be listed with viewer privilege. Select the “viewer” from the “member of” list and click the arrow button to remove the default privilege. Select the “project manager” from “not member of” list and click the back arrow button to assign the “project manager” privilege to the user “john”. Click the close button to close the “set up users” window. The user is now assigned “project manager” privilege. Creating the test plan. Select the “plan tests” tab from the “test director” window. Add the subject folders to the tree. To create a folder, click the “new” button in the “folder” group box of test director window. On click the new button, the dialog box is displayed prompting you to enter the folder name. Enter the name of the folder and click ok button. When you will click on OK button, the subject will be added to the tree structure. Create the test plan for the subject defined in the tree structure. Select the “plan tests” tab. Select the test from the tree. Click the new button from the test group box to create the new test. When you click on new button, create new test dialog box appears. Select the test type. Click OK button to add the test to the tree hierarchy. Select the details tab and enter the description of the test case. Select the design steps tab to define the steps of the test case. Click the new button. It opens a design step editor window. Insert a new record into the database. Enter the description and the expected results. Insert all the steps required to insert a record into the application. For example, we are using employer details as the application. Test script generation. Select the test script tab. Click the launch button to open Win Runner. Open the application. Start recording. Select the record- context sensitive mode from the create menu. elect new from file menu to insert a new record into database. Enter the employer name, salary and designations field. Click Add button to insert the record. Stop recording. Save the test script and close the test script window. The test script file, just created/ recorded in Win Runner will be loaded in to the view window. Analyze the test plan by generating a report. Report>standard. On selecting the standard option, the new dialog box opens. Select the design steps and portrait option and click the ok button. A report of selected details will be displayed in selected format. Create the test plans for update and delete operations in the database. Steps for updating a record: Run the employer Db application. Open the existing record by clicking the open option in file menu. Select the Employer by no. option. Enter the employer no. Click OK to display the retrieved record. Make the required changes. Click the modify button to update the changes. Steps for deleting a record: Run the employer Db application. Open the existing record by clicking the open option in file menu. Select the Employer by no. option. Enter the employer no. Click OK to display the retrieved record Click “delete” button to delete the record. S confirmation message is displayed. Click yes to delete the record. Select “standard” from report menu and select the “complete information” button to view the complete details of the test plans. Click OK button to see the results.

Running the automated test

Test Director Overview

Running the automated test

When you run the test script in win runner, it displays the result. To run the automated test plan, follow the steps given below:

Select the run tests tab from the test director window. Select “test set builder” from the run menu. Run>Test set builder. When you select the “test set builder” option, it displays the dialog box. By default the “test set information” is set to “default”. If not, select the “default” from the combo box and select the category as subject. Select the “DB trans” folder and explore it. It will display the test plans that are created under the category. Select the test plan that is to be executed. Click the Close button to close the test set builder. The name of selected test in the test set builder dialog will be displayed in test director window. Select the add employer test and click the automated button from the run tests. It will display the run scheduler dialog. Select the “Add Employer” test and click arrow button to run the selected test. Once the execution of the test plan is completed, Test Director displays the result. If it is successful, then “passed” is displayed in green color in the “status” column. If the test fails, then “failed” is displayed in red color in status column. Generating test run reports. Select the “Run Tests” tab. Select the “standard” option from the “report” menu Report>Standard It displays a dialog box prompting you to select the type of information for which the report has to be generated and the format of the report. Click the “complete information” button and select the layout as “portrait”. Click OK button to view the report.

Tracking defects

It is very important to track the defects in the application so that they can be rectified to make the application more effective. You need to first identify the defects, identify the effects of the defect on the application and report the details of the defect to the developer so that he can rectify it as soon as possible.

To identify the defects in the application, follow the steps given below:

Select the “test defects” tab in the application. Click the Add button to add a new defect. The “new defect” dialog appears prompting you to enter the details of the detected defect. Enter the following details. Summary: Brief description of the defect. Detected by: The name of the user who detected the defect. Detected on: The date on which the defect was detected. Assigned to: The name of the person to whom the job has been assigned. Status: The status of the defect. Project: The name of the project in which the defect has been detected. Severity: the degree to which it affects the application. Priority: Indicator of the importance of the defect. Whether the defect has to be repaired immediately or whether the repair can be postponed. Enter or select the details of the defect. Click OK button to create the defect. Once the defect is created successfully it will display message “added successfully”. Click OK button. When you click the ok button, it adds the newly created defect to the “defect list” in the Test Director dialog. To view/ modify the details of defect, select the defect and click the details button. The defect details dialog box appears. Change the required properties of defect. Mail the defect to project team member. Select the defect. Click the mail button. The compose dialog box is displayed. Select the name of the user to whom the defect has to be mailed. Click OK button to send the mail. Report generation. To generate the report: Select the track defect tab, if not selected. Select the defect for which the report has to be generated. Select the standard option from the report menu. The dialog box will be displayed. Click the complete information button and select the layout as portrait. Click the filter button to select the defects based on particular condition. Click ok button to see the generated report. Defect tracking with the help of progress chart. Select the defect. Select the defect progress graph from the analysis menu. Analysis>defect progress graph. The defect progress graph is displayed. This graph is displays the progress of the graph from the day it was created till today. Defect tracking with the help of summary graph. Select the defect. Select defect summary graph from the analysis menu. The graph will be displayed. The X axis represents the person to whom the defect has been assigned. The Y axis represents the no. of defects.

Running the automated test

Test Director Overview

Running the automated test

When you run the test script in win runner, it displays the result. To run the automated test plan, follow the steps given below:

Select the run tests tab from the test director window. Select “test set builder” from the run menu. Run>Test set builder. When you select the “test set builder” option, it displays the dialog box. By default the “test set information” is set to “default”. If not, select the “default” from the combo box and select the category as subject. Select the “DB trans” folder and explore it. It will display the test plans that are created under the category. Select the test plan that is to be executed. Click the Close button to close the test set builder. The name of selected test in the test set builder dialog will be displayed in test director window. Select the add employer test and click the automated button from the run tests. It will display the run scheduler dialog. Select the “Add Employer” test and click arrow button to run the selected test. Once the execution of the test plan is completed, Test Director displays the result. If it is successful, then “passed” is displayed in green color in the “status” column. If the test fails, then “failed” is displayed in red color in status column. Generating test run reports. Select the “Run Tests” tab. Select the “standard” option from the “report” menu Report>Standard It displays a dialog box prompting you to select the type of information for which the report has to be generated and the format of the report. Click the “complete information” button and select the layout as “portrait”. Click OK button to view the report.

Tracking defects

It is very important to track the defects in the application so that they can be rectified to make the application more effective. You need to first identify the defects, identify the effects of the defect on the application and report the details of the defect to the developer so that he can rectify it as soon as possible.

To identify the defects in the application, follow the steps given below:

Select the “test defects” tab in the application. Click the Add button to add a new defect. The “new defect” dialog appears prompting you to enter the details of the detected defect. Enter the following details. Summary: Brief description of the defect. Detected by: The name of the user who detected the defect. Detected on: The date on which the defect was detected. Assigned to: The name of the person to whom the job has been assigned. Status: The status of the defect. Project: The name of the project in which the defect has been detected. Severity: the degree to which it affects the application. Priority: Indicator of the importance of the defect. Whether the defect has to be repaired immediately or whether the repair can be postponed. Enter or select the details of the defect. Click OK button to create the defect. Once the defect is created successfully it will display message “added successfully”. Click OK button. When you click the ok button, it adds the newly created defect to the “defect list” in the Test Director dialog. To view/ modify the details of defect, select the defect and click the details button. The defect details dialog box appears. Change the required properties of defect. Mail the defect to project team member. Select the defect. Click the mail button. The compose dialog box is displayed. Select the name of the user to whom the defect has to be mailed. Click OK button to send the mail. Report generation. To generate the report: Select the track defect tab, if not selected. Select the defect for which the report has to be generated. Select the standard option from the report menu. The dialog box will be displayed. Click the complete information button and select the layout as portrait. Click the filter button to select the defects based on particular condition. Click ok button to see the generated report. Defect tracking with the help of progress chart. Select the defect. Select the defect progress graph from the analysis menu. Analysis>defect progress graph. The defect progress graph is displayed. This graph is displays the progress of the graph from the day it was created till today. Defect tracking with the help of summary graph. Select the defect. Select defect summary graph from the analysis menu. The graph will be displayed. The X axis represents the person to whom the defect has been assigned. The Y axis represents the no. of defects.