7/27/2019 Test Strategy Template Instructions_Automation
1/24
Test Automation Strategy
Author:
7/27/2019 Test Strategy Template Instructions_Automation
2/24
Master Test Plan Version 1.0- 10/14/2009
Page 2
Document Status: Final Template Version: 1.1
Version History
Changes to the Test Automation Strategy must follow the update process to ensure that the most
current version(s) are being used by all project team members. This section outlines how theseupdates to the Test Automation Strategy will be published and approved. The project teams
versioning standards should be adhered to for numbering convention.
Circumstances requiring approvals for changes to the Test Automation Strategy could include
any of the following:
Changes in automation scope or requirements resulting in corresponding updates to theAutomation Test Strategy
Risks, Constraints, Dependencies and Assumptions as and when identified as far asautomation is concerned.
Following any significant revision, the Test Automation Strategy will be distributed by email forreview and sign off by individuals identified as approvers in section 1.2. Voting button options
will include approve, approve with change(note in email response) orreject. Smaller or lesssignificant changes do not require project team approval.
Review Log
Version Date Section(s) Changed By Approval
0.1 Original Release QA Lead
7/27/2019 Test Strategy Template Instructions_Automation
3/24
Master Test Plan Version 1.0- 10/14/2009
Page 3
Document Status: Final Template Version: 1.1
This Test Automation Strategy template reflects the standard processes and
procedures for the test automation-planning phase. The format of this template is
designed to facilitate the creation of a Test Automation Strategy by providing theessential sections that should be included in a comprehensive test automation plan,
and by suggesting subject matter for each section. The author should be familiarwith the Principal Financial Group development processes and related automation
testing procedures.
Text displayed in blue italics (style=InfoBlue) is included to provide guidance to
the author and should be deleted before publishing the document.
To customize automatic fields in Microsoft Word (which display a gray
background when selected), select File>Properties and replace the Title, Subject,
Project, and Version fields with the appropriate information for this document.After closing the dialog, automatic fields may be updated throughout the document
by selecting Edit>Select All (or Ctrl-A) and pressing F9, or simply click on the
field and press F9. This must be done separately for Headers and Footers. Alt-F9will toggle between displaying the field names and the field contents. See Word
help for more information on working with fields.
All sections of this Test Automation Strategy document are to be preserved to
maintain uniformity and indicate areas of deliberate omissions. The author shouldnot deliberately omitsections with the notation NA.
Each section of this template provides guiding information for the author and or
actual text that the author may use as examples of the content of each section.
Action Item: Any text in blue serves as a placeholder and/or sample data.Substitute it with real information or delete it.
Action Item: Please make sure to update all fields and tables in this document.(The key sequence is then )
7/27/2019 Test Strategy Template Instructions_Automation
4/24
Master Test Plan Version 1.0- 10/14/2009
Page 4
Document Status: Final Template Version: 1.1
Table of Contents
1.0 INTRODUCTION ......................................................................................................................................... 6
1.1 PURPOSE...................................................................................................................................................... 6
1.2 TARGET AUDIENCE ..................................................................................................................................... 6
2.0 ESTABLISHING THE ASSIGNMENT ...................................................................................................... 7
2.1 IDENTIFY THE SCOPE ................................................................................................................................... 7
2.1.1 In scope .................................................................................................................................................. 7
2.1.2 Out of scope ........................................................................................................................................... 7
2.2 IDENTIFY THE PRECONDITIONS.................................................................................................................... 82.2.1 Dependencies/Constraints/Considerations ............................................................................................. 8
2.3 IDENTIFY THE ASSUMPTIONS....................................................................................................................... 9
2.4 IDENTIFY THE RISKS .................................................................................................................................. 10
3.0 DETERMINING THE STRATEGY ......................................................................................................... 13
3.1 TEST LEVELS............................................................................................................................................. 13
3.1.1 System Test .......................................................................................................................................... 13
3.1.2 Regression Testing .......... .......... ........... .......... ........... .......... ........... .......... ........... .......... ........... .......... .. 13
3.2 RISK TABLE ............................................................................................................................................... 13
4.0 HIGH LEVEL MILESTONES .................................................................................................................. 14
4.1 HIGH LEVEL MILESTONE ESTIMATED DATES ............................................................................................ 14
5.0 ENTRANCE / EXIT CRITERIA ............................................................................................................... 15
6.0 SUSPENSION CRITERIA ......................................................................................................................... 15
7.0 DEFINE THE TEST PRODUCTS ............................................................................................................ 16
7.1 TEST STRATEGY ........................................................................................................................................ 16
7.2 TEST SCRIPTS ............................................................................................................................................ 16
7.3 TEST DATA................................................................................................................................................ 167.4 PROGRESS REPORTS................................................................................................................................... 16
7.5 TEST EVALUATION REPORTS..................................................................................................................... 16
8.0 DEFINE THE ORGANIZATION ............................................................................................................. 17
8.1 ROLES &RESPONSIBILITIES ...................................................................................................................... 17
8.2 TRAINING NEEDS ....................................................................................................................................... 18
7/27/2019 Test Strategy Template Instructions_Automation
5/24
Master Test Plan Version 1.0- 10/14/2009
Page 5
Document Status: Final Template Version: 1.1
9.0 COMMUNICATION PLAN ...................................................................................................................... 19
9.1 MEETINGS AND REPORTS ........................................................................................................................... 19
9.2 TELECONFERENCE INFORMATION .............................................................................................................. 21
9.3 WEB CONFERENCE INFORMATION............................................................................................................. 21
9.4 PROJECT DISTRIBUTION LISTS .................................................................................................................... 21
9.5 OUTLOOKPUBLIC FOLDERS ...................................................................................................................... 21
9.6 DOCUMENT STORAGE ............................................................................................................................... 21
9.7 QUERY LOG............................................................................................................................................... 21
10.0 DEFINE THE INFRASTRUCTURE ........................................................................................................ 22
10.1 APPLICATION SETUP IN SYSTEM TEST ENVIRONMENT .............................................................................. 22
10.2 TEST USERS SETUP.................................................................................................................................... 22
10.3 SYSTEM SETUP .......................................................................................................................................... 22
11.0 ORGANIZE TEST MANAGEMENT ....................................................................................................... 23
11.1 TEST PROCESS MANAGEMENT .................................................................................................................. 23
11.2 TEST CASE COVERAGE RATIOS .................................................................................................................. 24
11.3 INFRASTRUCTURE MANAGEMENT ............................................................................................................. 24
11.3.1 Meetings .......................................................................................................................................... 24
11.3.2 Environment, Tools and Workplaces .............................................................................................. 24
11.4 DEFECT MANAGEMENT............................................................................................................................. 24
7/27/2019 Test Strategy Template Instructions_Automation
6/24
Page 6
Document Status: Final Template Version: 1.1
1.0 INTRODUCTION
This Test Automation Strategy communicates the approach to implementing the automation
testing effort using a test automation tool.
This document serves as both a communication tool for the automation team and a governingdocument for the test automation initiative. It will be maintained to reflect all significant
changes to the scope, approach, test methods or entry/exit criteria defined in this strategy. In
addition, the only authorized deviations from standard practice controls and deliverables are
those noted in the approved plan. All activities and their deliverables will be conducted inaccordance with standard policy and procedures and are subject to a compliance audit.
1.1 PURPOSE
The purpose of this Test Automation Strategy is to communicate to the project
stakeholders and to the testing team how the Strategy will be implemented and to
establish authorization to proceed with the test initiative using the referenced approach,
standards and controls. This is a governing document for the test activities therefore thedeliverables and key controls identified in this Strategy are subject to review for
compliance with standard policies and procedures.
1.2 TARGET AUDIENCE
Indicate who will approve document. Sign-off is only required for approvers.
The target audiences for this document are those that represent the following roles on thisproject:
Role Name Approver /
Reviewer
Sign-off
Date
QA LeadTest Manager(As per
the RUP template)
Test Analyst -Automation
Business Analyst
7/27/2019 Test Strategy Template Instructions_Automation
7/24
Page 7
Document Status: Final Template Version: 1.1
2.0 ESTABLISHING THE ASSIGNMENT
2.1 IDENTIFY THE SCOPE
2.1.1 In scope
This Test Strategy for the testing initiativeincludes the following areas of scope:
In this section, identify the scope of automation testing activities planned. Includethe following topics:
Explain about the need/goals of automation for the project and how it would help
in overall testing.
What types of testing are covered under this automation strategy, such as
functional, regression, smoke or other tests?
Why a particular test tool is selected.
How the proof of concept was carried out, how much coverage is expected throughautomation, which types of test cases are to be automated.
Define dependencies, such as, impact of changes in development plan, features,
etc. Make recommendations on the time-line for automation.
Provide a high level design that is scalable and allows future addition of testscripts.
Provide a plan to manage and maintain the Automation scripts (post-delivery)
What components of the overall application are to be tested?
What is the relationship of the application to be automated to the overall system /
environment?
2.1.2 Out of scope
The following functions, systems, processes are specifically out of scope for the testing initiative:
In this section, identify specific items that are outside the scope of testing
What functional business areas will not be automated?
Mention that high complex functions will not be automated.
Mention that tests requiring large calculation or more user interference will not beautomated. Files containing multiple records would not be tested.
Mention if tests involving backend validations are not done
Mention that non repetitive Test Cases (They may be automated but is not feasible)
Mention that tests requiring longer time to automate but manual testing time would
be too less, would not be automated.
7/27/2019 Test Strategy Template Instructions_Automation
8/24
Page 8
Document Status: Final Template Version: 1.1
Mention that tests involving Graphics might not be automated.
Mention that test cases involving testing of printing features can not be tested
Mention that test cases involving the Mozilla browser can not be tested
2.2 IDENTIFY THE PRECONDITIONS
2.2.1 Dependencies/Constraints/Considerations
List any dependencies/constraints/considerations that may impact the design,
development or implementation of automation testing; e.g., dependencies on other
organizations (such as DEV, Environment group, Users, and or SMEs) and orschedules, resource dependencies, etc. Some samples have been provided below.
Indicate other projects that may affect this project.
Description Type
Dependency, Constraint, Offshore
consideration, XBU consideration,
Vendor consideration, other
Impact/Consideration
Manual test cases not ready Dependency Automation may not start as
the framework maynot be
created without knowing allthe test cases that needs to be
automated.
Contractors contract expires12/01/2010
Constraint Resource availability
There is a company freezebeginning 12/15/2009
Constraint Time frame/ Schedule
PFG will be assessed a fine if
change does not occur by12/31/2009
Constraint Regulatory
SharePoint Access Offshore consideration Who will need access toSharePoint
Timing Issues related to cycle Offshore consideration Time difference
Communication across
business units
XBU consideration How will communication be
handled?
Determine which QualityCenter Domain & Project will
be used
XBU consideration Access/Security to QualityCenter Domain & Project
Defect Resolution Vendor consideration How will defects becommunicated to vendor and
how will vendor communicate
when defect has been
7/27/2019 Test Strategy Template Instructions_Automation
9/24
Page 9
Document Status: Final Template Version: 1.1
resolved?
2.3 IDENTIFY THE ASSUMPTIONS
Identify the significant assumptions made in developing this plan, which if false,
may impact the design, development or implementation of testing
The Business Analysts for this project will be available to answer questionswhen necessary.
The environment will be available andstable.
Manual testing will be completed before automation begins. The results willbe available to the project team.
If there is a slippage in the project timeline or scope, automation test scriptcreation will also be impacted.
Existing production issues will remain unless specifically addressed by the
project. QA resources assigned to the project will have previous knowledge of
creating test scripts using the test automation tool.
All assigned QA resources must be given a background of the project andunderstand those aspects prior to the start of script creation.
Test execution will not start until signoffs are received for all QA artifacts thatrequire signoff.
All the security access needed will be given to the test automators prior to testscript creation.
Test cases will be written based on business and functional requirements andtechnical designs. If any of these documents are not completed, test scripts
will not be in accordance and there will be a risk of having gaps and rework.
7/27/2019 Test Strategy Template Instructions_Automation
10/24
Page 10
Document Status: Final Template Version: 1.1
2.4 IDENTIFY THE RISKS
The table below outlines project or process risks to testing. These types of risk generally
affect the test teams ability to work efficiently or meet project milestones.
The content of the table below are examples only -
Estimates of project Impact and Probability of occurrence are listed in the table as
L=Low, M=Medium, or H=High. The Risk Measure is a calculation of[Impact x Probability] with a scale of 1-9. Values are assigned as follows:
L = 1
M = 2
H=3
Description Impact Probability
Risk
Measure Mitigation
7/27/2019 Test Strategy Template Instructions_Automation
11/24
Page 11
Document Status: Final Template Version: 1.1
Description Impact Probability
Risk
Measure Mitigation
The completion of
automation scripts woulddepend on the Manual
Testing completion
H H 9 As and when the manual test case
creation and execution iscomplete, one can start
automation, need not wait for theall the test cases to be complete.
Certain test cases will not
be automated due to high
complexity. Henceautomation will be
incomplete for those area
H H 9 This might not be decided at the
time of planning but as the work
progresses in automation, one mayfind such a scenario which needs
to be discussed and addressed.
Requirements will changewhile the project is in
development due to
changing regulatoryrequirements, businesschanges, etc.
H H 9 Project Management will reviewany changes, assess impact, risk,
and cost and present these to the
steering committee for review.This will include date changediscussions.
Testers unfamiliar with newenvironments.
H H 9 PM will provide point of contactfor environments to the QA Lead.
If any environmental issues arise,
QA will raise any issues orconcerns immediately to the PM.
Security access and profile
timelines.
M L 2 PM has negotiated the dates for
Security definition
Testing resources assigned
to project are unable to
commit time for testing.
M M 4 Work with testing resources to set
up dedicated testing time. Work
with Project Manager to resolveany resourcing issues.
Delay in code delivery. H L 3 Additional QA effort will need tobe identified to overcome the
timeline gap. This will happen if
the timeline is not adjusted for the
delay.
7/27/2019 Test Strategy Template Instructions_Automation
12/24
Page 12
Document Status: Final Template Version: 1.1
7/27/2019 Test Strategy Template Instructions_Automation
13/24
Page 13
Document Status: Final Template Version: 1.1
3.0 DETERMINING THE STRATEGY
The test strategy is based on the product risk analysis, and the decision on which
quality characteristic must be tested per test level, and to what depth.
3.1 TEST LEVELS
The following test levels will be executed:
3.1.1 System Test
A system test is a test carried out by the test team, with the aim of
demonstrating that the developed system, or parts of the system, meets with thefunctional specifications and the technical design. (This includes regression
testing of existing functionality and interfaces with other systems.)
The automation scripts would be created as per the test cases and would be
executed end to end to complete the System Testing test cases.
3.1.2 Regression Testing
Regression Testing will be to demonstrate rerunning previously run scripts andchecking whether previously fixed faults have re-emerged.
3.2 RISK TABLE
Determine which test phase characteristics are tested in; System, Performance,
Regression. Rate them by High. Medium, Low
The manual test cases would cater to these characteristics but in case you want to
emphasize on the part creation of test scripts, you can enter the product riskanalysis and the decision on which quality characteristic the test scripts must be
created per test level, and to what depth. The decision totally depends on the
feasibility of creation of automation scripts and support of the test tool.
System Performance Regression
Connectivity
Continuity
Data
controllability
EffectivelyEfficiency
Flexibility
Functionality
Suitability of
Infrastructure
Maintainability
Manageability
7/27/2019 Test Strategy Template Instructions_Automation
14/24
Page 14
Document Status: Final Template Version: 1.1
Performance
Portability
Reusability
Security
Suitability
TestabilityUser-
Friendliness
4.0 HIGH LEVEL MILESTONES
4.1 HIGH LEVEL MILESTONE ESTIMATED DATES
Below are high-level milestone as currently defined. For more detailed schedule
and milestone information reference the project Strategy on the project share drive.
Activity Estimated Start Date Estimated End Date
Create Test Automation Strategy
Create Automation Framework
Identifying the reusablecomponents and preparation
Creating the OR
Test Data Preparation
Script Creation
Review
Rework
Individual Execution
Batch Execution
Report Analysis
Downtime/Issue Log
Documentation
7/27/2019 Test Strategy Template Instructions_Automation
15/24
Page 15
Document Status: Final Template Version: 1.1
5.0 ENTRANCE /EXIT CRITERIA
System Test (performed by automation Tester role)Criteria Description
Entry Criteria Coding is complete and unit integration testing has been reviewed and
accepted by the test organization
No outstanding critical defects open
The defined System Test environment is prepared and stable
System Test scripts have been written and reviewed for coverage and
completeness
System Test data is available for test execution
Change Request and Version Control process is in place
Exit Criteria Agreed upon coverage ratios have been met and all scripts have been
executed.
Application has been sufficiently tested to meet requirements (by
execution of scripts)
No outstanding critical defects remain open
Regression Test (performed by automation Tester roles)
Criteria Description
Entry Criteria Approval to deploy application has been received from business
Application has been deployed to production
Exit Criteria All test scripts have been executed
No outstanding critical defects remain open
6.0 SUSPENSION CRITERIA
This section will identify and address any situation where the automation test script
creation effort might have to be temporarily suspended.
6.1 Test environment becomes unavailable.
6.2 Test cases are blocked by a documented defect.
7/27/2019 Test Strategy Template Instructions_Automation
16/24
Page 16
Document Status: Final Template Version: 1.1
7.0 DEFINE THE TEST PRODUCTSDefine the artifacts that will be produced for this project. Examples have been includedbelow.
7.1 TEST STRATEGY
System Test Plan
Regression Test Plan
7.2 TEST SCRIPTSTest scripts, test data and documentation and other xls, vbs files attached to the
script are housed in Quality center:
Domain=Project=
7.3 TEST DATADocumentation of data to be used for execution of test scripts, both in QC and in
the xls data sheet.
7.4 PROGRESS REPORTS
Test execution status reportsweekly reports of test execution status andapplication quality
7.5 TEST EVALUATION REPORTS
Issues after Release Advice
Gives insight into the way the test script creation and execution process hasgone
Provides results from the lessons learned session
7/27/2019 Test Strategy Template Instructions_Automation
17/24
Page 17
Document Status: Final Template Version: 1.1
8.0 DEFINE THE ORGANIZATIONDefine the Roles & Responsibilities that will be utilized with this project. If you will notbe using a specific role, indicate Not Applicable.
8.1 ROLES &RESPONSIBILITIES
Role Assigned To Responsibilities
QA Lead/
AutomationAssignment Manager
Prepare Test Automation Strategy
Schedules and facilitates automation testrelated meetings
Provides oversight of automation relatedDefect Management process
Assists in mentoring and training of projecttesters when appropriate
Provides testing status updates throughout theautomation testing phase
Provides resource estimatesTesters/ Test
Automationdeveloper(as per the RUPtemplate)
Create test scripts
Execute test scripts within the timeframeallowed
Log defects and retest as necessary
Communicate status of testing to QA Lead
Assignment Manager
Partner with QA Lead to ensure successfulexecution and completion of automationtesting related activities on business unit
projects.
Business Owner/SME Work with Business Analyst to clarify
requirements. Collaborate with QA to get clarification while
creating test scripts in order to ensure propercoverage of all business requirements andfunctional specs., that are already dealt with inthe test case.
Help identify new areas for automationtesting, and define validation responsibilities.
Project Manager Ensure the project scope is finalized prior tothe team beginning test script creation and
execution.
Ensure adequate project timelines support thetime needed for test preparation andautomation script execution.
Assist the QA Lead, as a resource, inidentifying the reporting needs, specific to the
project.
7/27/2019 Test Strategy Template Instructions_Automation
18/24
Page 18
Document Status: Final Template Version: 1.1
Test Environment
Support
Migrate code
Test environment issue resolution.
Configuring test environment.
Participate in Automation Testing StatusMeetings as needed.
8.2 TRAINING NEEDS
Describe what training, if any, is needed for the test team to come up to speed.
Examples may include automation tool training, Quality Center training orapplication knowledge.
7/27/2019 Test Strategy Template Instructions_Automation
19/24
Page 19
Document Status: Final Template Version: 1.1
9.0 COMMUNICATION PLAN
9.1 MEETINGS AND REPORTS
Describe how often and in which way the automation test team will report withinand outside the team. Make sure this is in accordance with the customers wishes.
Describe how often and with whom a meeting shall take place
This section should address all forms of communication; meetings, reporting, etc.,list exceptions to the standards.
Event Objective Method
(Report
/Meeting)
Chair Attendees /
Audience
Frequency
Testing Kickoff Review timeline
and introduce theTest Automation
Strategy document
to QA resources.
QA Lead Project Manager
QA LeadAutomation
developer
Technical Lead
Project Sponsor
Once
Test Status
Review
Review general
health of
Automation Testing
Initiative.
Operational Issues
QA Lead Test team
QA Lead
Weekly
General
Project
Meeting(s)
Describe any
general project
meetings known at
the time of the Test
Strategy that
members of the test
team will be
attending.
Project
Manager
Technical Lead
Business AnalystProject Sponsor
QA Lead
Weekly
Weekly Status Shows status of
automation script
creation and
execution.
Report QA Lead Project Manager
QA Lead
Project Sponsor
Weekly
Defect Review Review status of
open defects.
Identify any newissues. Release
status.
QA Lead QA Lead
Project Manager
Technical Lead
Weekly or As
needed
Lessons
Learned
Discuss project
results and ID
lessons learned for
future
improvement.
Project
Manager
QA Lead
Project Manager
Technical Lead
Automation
developer
Once
7/27/2019 Test Strategy Template Instructions_Automation
20/24
Page 20
Document Status: Final Template Version: 1.1
Business Analyst
7/27/2019 Test Strategy Template Instructions_Automation
21/24
Page 21
Document Status: Final Template Version: 1.1
9.2 TELECONFERENCE INFORMATION
Update items in red with your specific information.
US & Canada: Dial-In Number: XXX-XXX-XXXXInternational: Dial-In Number: XXX-XXX-XXXX
Enter conference Code: XXX-XXX-XXXX#
Leader enter *; then PIN: XXXX#, then 1
System will ask you to state your namemay do so, or leave blank bypressing #
System will sound a tone then connect the call
9.3 WEB CONFERENCE INFORMATION
Update items in red with your specific information.Sign in by clicking on the URL keying all information below:
Meeting URL:https://www.livemeeting.com/cc/principal/joinMeeting ID: XXXXXX
Presenter Key: XXXXXX
Attendee Key: XXXXXXXX
9.4 PROJECT DISTRIBUTION LISTS
Include email distribution lists built for this project.
9.5 OUTLOOKPUBLIC FOLDERS
The following public folders will be used by this project for testing:
Add Outlook path here
9.6 DOCUMENT STORAGE
Indicate where your documentation is stored and provide the path or a link.
SharePoint
Project folder
9.7 QUERY LOGIf your project is not utilizing PGS resources, the Query Log is Not Applicable.
The Query Log within SharePoint will be utilized to communicate general
questions and answers regarding terminology, resources and knowledge gaps
encountered while creating automation test scripts.
https://www.livemeeting.com/cc/principal/joinhttps://www.livemeeting.com/cc/principal/joinhttps://www.livemeeting.com/cc/principal/joinhttps://www.livemeeting.com/cc/principal/join7/27/2019 Test Strategy Template Instructions_Automation
22/24
Page 22
Document Status: Final Template Version: 1.1
10.0 DEFINE THE INFRASTRUCTURE
10.1 APPLICATION SETUP IN SYSTEM TEST ENVIRONMENT
The system test environment needs to mimic the production environment. The
environment, including the software and data, must be a controlled environment.Development will be delivering new builds to the System environment on an agreed upon
schedule. When each new build is delivered to QA, the following application setup needs
to be completed:
Provide additional information as needed for your project.
10.2 TEST USERS SETUP
Specify any special user setup. Consider the needs for test accounts with various
access levels.
10.3 SYSTEM SETUP
The following applications must be controlled and available in the System environmentto support the System Testing effort.(This should include any hardware setup that needs
to be done for the automation tool to run smoothly.)
System Included in test? Notes
Yes No Note special data requirements, etc.
Yes No
Yes No
Yes No
Yes No
Yes No
Yes No
Yes No
Yes No
7/27/2019 Test Strategy Template Instructions_Automation
23/24
Page 23
Document Status: Final Template Version: 1.1
11.0 ORGANIZE TEST MANAGEMENT
Describe the manner in which the management of the test process, infrastructure, test
products and defects is organized
11.1 TEST PROCESS MANAGEMENT
This should be in line with the manual test process management. The defects detected bythe execution of the automation scripts should be assigned and tracked. Some in-process,
production readiness metrics can be calculated from the data gathered in defect
management tool and the test management tool. Some of these real-time metrics help
predict whether the quality of the software will be acceptable by the scheduled release.Others provide status on the completeness of the testing phases, which can be compared
to the project plan. Typically, these metrics are calculated and reviewed as part of project
status.
Note: In the metrics below defects include defects only. Defects do not include anyenhancements that may be logged. Each test level may report the metrics for that level in
their weekly status.
Metric: Indicates/Reflects on:
# open defects by severity Status of the software towards production ready.
List of open defects used to evaluate the current status,
severity and priority of open defects.
# new defects found by week Whether the software is approaching production ready.
This is viewed graphically as a time series chart.
If defect arrival rate does not quickly decrease, teamleaders and management should promptly assess the
situation.
# new defects found by week by
severity
Whether the software is approaching production ready.
This is viewed graphically as a time series chart.
Critical and major defects should decrease very quickly.
# defects resolved by week Whether the software is approaching production ready.This is viewed graphically as a time series chart.
If defect resolution rate is not greater than the # of newdefects found by week, team leaders and management
should promptly assess the situation.
# test cases by execution status
pass/fail/no run
% test cases executed
Whether the software is converging on production
ready.
If the # test scripts passed per week does not quicklyincrease, team leaders and management should promptly
7/27/2019 Test Strategy Template Instructions_Automation
24/24
Page 24
Metric: Indicates/Reflects on:
assess the situation.
11.2 TEST CASE COVERAGE RATIOS
80% is the recommended overall test case coverage ratio for test automation. Thismay be adjusted up or down based on project risks, priorities and impacts.
The test coverage ratio is the comparison of the total number of test cases
automated to the total number of test cases written.
11.3 INFRASTRUCTURE MANAGEMENT
11.3.1 Meetings
The Automation Test Manager & QA Lead will meet on an agreedupon scheduled time to discuss what more needs to be automation and
its feasibility.
11.3.2 Environment, Tools and Workplaces
Describe any additional items that are specific to your project. Examples havebeen provided.
Test automation validation will occur in Text Pad and/or Excel asneeded
Test Environment Location:
Path/Logical location information needed to access Test Environment
Security required for access to the Test Environment
Script creation and execution will occur at automation testers desks
11.4 DEFECT MANAGEMENT
Defects will be tracked within the following Quality Center Domain & Project.
< QC Project>
Justify any deviations from this process.