Automation Concepts & QTP
Demo
Name : Dheenan Ganesan
2
Please mute your mobile phones
Do not come late OR while the session is ON
Help each other in learning – as learning is a continuous process
Participate actively to make the session interactive
No side conversations
Do not beat the dead horse
Ground Rules
3
Agenda
Automation Concept
1. Test Automation – Why?
2. Test Automation – Qualifying Criteria
3. Test Automation - Prerequisites
4. Automation Readiness Check
5. Challenges in Automation
6. Phases of Automation
7. Automation Framework
8. Skills required for scripting
9. Availability of Automation tools
Automation Tool - QTP
1. Tool Selection Process
2. Costs involved in automation
3. ROI
4. Automation Benefits
5. Automation Drawbacks
6. Case Study
7. Test
QTP Demo
3
Test Automation – Why?
Reduced Test Cycle Time
Earlier defect reporting
Reduction in Test Execution Effort
Remove Tester fatigue
Earlier informed decision making
Detailed Test Log Archives
Reduced criticality of Functional/Domain expertise
4
Test Automation – Qualifying Criteria
Repetitive nature of tests:
If the tests are repetitive in nature like regression tests, portability tests,
compatibility tests etc, then automation can be considered.
Regression tests – Tests that are run for every build/release
Portability tests – Tests that are run on different OS or webservers
Compatibility Tests – Tests that are run on different browsers or software
components
Tests run on production:
The smoke tests( without data change) have to be completed within a very
short duration of time to minimize the downtime of the production servers.
5
Test Automation – Prerequisites
Application stability
Stable GUI, stable functionality
Tool readinessTool decided, tool compatibility
established, availability of licenses
People readiness
Automation team available with
required skills
Project Readiness
The project has reached a stage when test automation is estimated,
The approval of project management team for going ahead
with test automation.
Process Readiness
The right automation framework is identified and communicated to the
automation team.
6
Automation Readiness Check
• Includes Test Calls, Test data arrangement, scripting standardsFramework should be
functional
• automation specific tasks like GUI objects maintenance and synching, local to shared staging, reviewing, configuration control etc.
Tasks allocation is understood
• Individual Roles and responsibilities are understoodRoles and Responsibilities
are identified
• Distribution lists, communication matrix established. Escalation mechanism understood.
Communication and Escalation channels defined
• Environment readiness checklist is doneAutomation Environment is
ready
• Formal – Automation plan signoff, kick off meeting; informal – stand up meeting
Project Management Team informed
7
Test Methodology - Automated
Top 5 Best Practices for Automation Success
Focus on the methodology, not the tool Choose extensible test tools Separate test design and test automation Lower costs Jumpstart with a pre-trained team
Assess
Environment
Identify
Gaps
Establi
sh
Framew
ork
Tool
Evaluatio
n, PoC
Script
Develop
ment
Stabilizati
on
Application
Architecture,
Stability
Test Pack Analysis
Financial
Resources
Personnel
Time-line
Dependencies
Completed
Environmental
Assessment
Determine priorities
Identify Strategic
Outlook
Prioritize Test Pack
Test Strategy
Test Pack Re-
engineering
Research, Evaluate,
Score test tool
candidates
Object Identification
Proof of Concept /
Prototyping
Approval
Test Scripts Stds
Learning, Metrics from
PoC re-used
Automation Framework
Design
Test Scripting
Script Integration,
Reviews, Testing, Re-
work
Dry Run
Documentation
Training
Metrics (% Automation
achieved)
Process Improvements
Maintenance Phase
1 2 3 4 5 6, 7
Key Considerations for automation
Application/functionality stability, release plans Application architecture Test pack analysis for automatable test conditions/cases Interfacing application dependency Analysis of automation ROI
Early Involvement
Define Performance Testing Requirements
Structured and Organized Test Case Repository
Software Testing Life Cycle (STLC)
Requirement Analysis
Testing Activities
Manual / Automation/ Performance Test Automation
Specific Activities
Test Strategy
Test Planning
Test Scenarios/
Conditions
Identify features which can/ cannot be automated
High level Automation Design / Framework
Automation Strategy
Design & Development of Automation Scripts
Test Script Execution
Test Script Maintenance
Design & Coding
Testing
Software Development Life Cycle (
SDLC )
Performance Testing
Specific Activities
Test Cases
Traceability
Test Environment
Test Data
Test execution
Defect Tracking & closure
Test Summary Report
Defect Summary Report
Design & Development of Performance Test Scripts
Performance Testing
Analysis and Fine Tuning
Performance Test Reports
•Identify business
scenarios for
performance testing
Framework Driven Automation
Modularized
Automation
Test Suite
Automation Framework
Manual Test Case Repository Manual Test Procedures
Automation
Design Considerations
Test
Automation
Standards /
Guidelines
Business
Scenarios
Automation
Test Scripts
Re-usable
Library
Automation
Design
Document
Test Automation
Results /
Metrics
Test Strategy
Test
Automation
Configuration
Control
Application Test Automation Tool Test Management Tool
Test Environment
Ap
plicatio
n M
ainten
ance
Reg
ression
Testing
En
han
cemen
t
Automation Frameworks in Practice
Modularity Driven
Data Driven
Keyword Driven
Hybrid
Types of Automation Framework
The Test Automation Life Cycle
Customer Speaks…..
Automated test generation reduces
manual effort up to 90%
Test planning time reduced by up to
50%
Test Execution Cycle reduced by
30%
Reduced QA effort by 25%
Modularity and Reusability - The same
tests can be reused on different
versions of an application, even if the
user interface changes
Process Expertise
Client References spanning across
geographies
Cost Effective solutions enriched by
stable processes
World class methodologies & tools
Standards Complying with IEEE, SW-
CMM/CMMI, ISO 9000:2000
Certified testing consultants & lead
assessors with domain expertise
Back
Approach to Test Automation
• Analyze business functionalities, application architecture • Economic feasibility – ROI analysis• Identify features which can/
cannot be automated
Test Planning & Design2
• Automation Effort Estimation• Test Strategy • Test Framework • Prepare Test Conditions/Cases• Prepare Traceability Matrix
Development of Scripts 3
• Test Scripts • Identify & place re-usable components in reuse library • Add checkpoints, synchronization points& functional validations• Dry Run
Decision to Automate1 Maintenance of Scripts
• Periodic Updation of scripts with changes to existing functionalities / new requirements• Label and maintain the changes • Versioning scripts & other related Testware
5
Test Execution4
• Test Script Execution • Reporting Results • Defect Management • Test Automation metrics• Update scripts • Prepare Operations Manual
Value Proposition on Test Automation
Fast – Runs Tests significantly faster with reduced time of testing lifecycle
Reliable – Tests perform precisely the same operations each time they are run
Repeatable – Repeat the same set of execution as many times desired.
Programmable – Program sophisticated tests that bring out hidden information.
Comprehensive – Build a suite of comprehensive tests that covers every feature
Reusable – Reuse test cases created in various situations
Maintainable – Easy maintainability and minimizes effort of maintenance
Long Term Savings – Phased ROI based Automation - in testing effort & cycle time
Improved Planning and Focused testing effort
Enhanced Coverage
Minimal Manual Intervention and coding and hence reliable
Regression Pack for Future Product Releases
Reduces Go-To-Market time for an application
Reduced Defect Density
Challenges in Automation - Process
15
Challenges in Automation – Application
Automation to keep pace with the rapidly changing application
Standards not followed during development.
Operations cannot be performed on some objects.
Batch processes – wait time and processing time
Environment – Shared environment, unstable
Test Data – Test data issues, modification of test data
16
Challenges in Automation - Tool
Objects not recognizable
by tool.
Backward compatibility
of tool versions
Difficult to use, non-
user friendly
Memory cache size
Non-availability of
support
17
Challenges in Automation - Skill
18
Test Automation – Framework Architecture
QUICK TEST
PROFESSIONAL
TERMINAL
`
TERMINAL
VPN
``TERMINAL
QUICK TEST PROFESSIONAL
TEST
RESULTS
SCRIPT
OBJECT
REPOSITORY
REUSABLE
ACTIONSLIBRARY FILES
QUALITY CENTER
DATA FILESRECOVERY
MANAGER
APPLICATION UNDER TEST
Tools Availability
20
•HP-QTP
•HP-WinRunner
•Rational Robot
•Compuware TestPartner
•Segue Silktest
Selenium
Floyd
Watir
Frankenstein SWTbot
JMeter
About QTP
QuickTest Professional enables you to
test
Standard Windows applications
Web objects, ActiveX controls
Visual Basic applications
Note:
Add-ins for special environments such as Java, Oracle, SAP
solutions, .NET Windows and Web Forms, Siebel,
PeopleSoft, Web services, and Terminal emulator
applications.
QTP Versions
QTP 10.0
QTP 9.5
QTP 9.2
QTP 9.0
QTP 8.2
QTP 8.1
QTP 8.0
QTP 6.5
QTP 6.0
QTP 5.6
QTP – Add in Manager
Keyword & Expert View
Keyword View:
Enables to create and view the steps of the test in a keyword-
driven.
Table-like view, in which each step is a separate row in the
table, and each column represents different parts of the steps.
Expert View:
Enables to create scripts in the form of VBscript for each
operation performed on the application.
Script editor and for each object and method in an Expert
View statement, a corresponding row exists in the Keyword
View
Keyword View
Expert View
Creating QTP Script
Record a session
Build an object repository
Insert checkpoints into your test
Broaden the scope of your test or component by replacing fixed
values with parameters.
Enhance your test and/or add programming statements
Test Results
QTP Recording Modes
Recording Modes:
Standard Recording Mode
Records the objects in your application and
the operations performed on them.
Low-Level Recording Mode
Records mouse clicks and keyboard input
with the exact x- and y-coordinate
QTP Default Mode : Standard Recording Mode
Synchronizing your Tests
When you run tests, your application may not always respond
with the same speed. For example, it might take a few seconds:
for a progress bar to reach 100%
for a status message to appear
for a button to become enabled
for a window or pop-up message to open
You can handle these anticipated timing problems by
synchronizing your test to ensure that QuickTest waits until your
application is ready before performing a certain step.
Synchronization Methods
Synchronization point
Exist or Wait statements
Increase the default timeout settings
Synchronization Point
A synchronization point is a
line in the test script that
instructs QTP to wait for a
certain response from the
application during playback.
Insert ->Synchronization Point
Synchronization point, which instructs QuickTest to pause the test
until an object property achieves the value you specify.
When you insert a synchronization point into your test, QuickTest
WaitProperty step is displayed in the Tree View with the icon for
the selected object. And also it generates the WaitProperty
statement in the Expert View.
Usage of Comments
Comments - A comment is a free text entry that spans an entire row.
Object Repository
When you create a test by recording, QuickTest learns
the properties of test objects and are stored in Object
repository. When the test runs, QuickTest searches for
the object that matches the description it learned.
Object Identification
37
Object Identification – Best Practices
Object Repository
Object Spy
Constructs
Conditional Statements
If .. Then
Else
End If
ElseIf … Then
Loop Statements
For … Next
Do… while
Do… until
If..Else Example
If window("Notepad").Exist(2) Then
reporter.ReportEvent micPass, "Notepad dialog",
"Notepad dialog exists"
Else
reporter.ReportEvent micFail, "Notepad dialog",
"Notepad dialog exists"
End If
Checkpoints
A Checkpoint is a verification point that compares a current value
for a specified property with the expected value for that property
How Checkpoint Works ?
When you add a checkpoint, QuickTest adds a
checkpoint with an icon in the test tree, and
adds a Check CheckPoint statement in the
Expert View.
When you run the test, QuickTest compares
the expected results of the checkpoint to the
current results. If the results do not match, the
checkpoint fails
Checkpoint Types
Output Value
An Output Value is a value captured during the test run and
entered in the run-time Data Table for use at another point in the
test run.
Parameterization
QuickTest enables you to expand the scope of
a basic test by replacing fixed values with
parameters. This process, known as
parameterization, greatly increases the power
and flexibility of your tests.
Parameter Types
Test/action parameters
Data Table parameters
Environment variable parameters
Random number parameters
Descriptive program - Advanced
'Record Method
Browser("Untitled Page").Page("Untitled Page").Link("Login To HCL LOS").Click
'One way
'Browser("Name:=Untitled Page").Page("Title:=Untitled Page").Link("text:=Login To HCL
LOS","html tag:=A").Click
'Other Way
Set MyObject = Browser("Name:=Untitled Page").Page("Title:=Untitled
Page").Link("text:=Login To HCL LOS","html tag:=A")
Myobject.Click
To connect QTP to Quality Center Choose Tools > Quality Center Connection or click the Quality Center Connection
toolbar button. The Quality Center Connection dialog box opens
In the Server box, type the URL address of the Web server where Quality Center is installed
To view the current Quality Center connection, point to the Quality Center icon. To open
the Quality Center Connection dialog box, double-click the Quality Center icon
Connecting QTP to TestDirector: The connection process has two stages. First, you connect QTP to a local or remote
Quality Center Web server. This server handles the connections between QTP and the Quality Center project
Next, you choose the project you want QTP to access. The project stores tests or components and run session information for the Web site or application you are testing.
Integration of QC with QTP:
Note: Quality Center projects are password protected, so you must provide a user name and a password
QC Server Name
Working with Quality Center Connectivity Add-in
If you are working with Quality Center 8.0, this add-in is installed automatically
To work with TestDirector 7.6 or earlier, you must manually install the TestDirector
Connectivity Add-in that corresponds to your TestDirector version on your QTP
Computer
Integration of QC with QTP (Contd):
Note that the Save Test to Quality
Center dialog box opens only when QTP
is connected to a TestDirector Project
To save a test directly in the file system,
click the File System button to open the
Save QTP Test dialog box. (From the
Save QTP Test dialog box, you can
return to the Save Test to TestDirector
Project dialog box by clicking the
Quality Center button)
Saving Tests to a QC Project
Integration of QC with QTP (Contd):
Running the Test Scripts
Mapping the test cases into Test lab
In the Run Stage, for the Host
Tab we have to specify the Host
Number on which you want to
run the test scripts, The Status
Tab will give the status of the
Test Script during the execution
of the Test Script and after
executing the script
Host Number
Integration of QC with QTP (Contd):
When you click on the left side run button in the test lab, you will get one window
which is shown on the right side of this slide. In this window we have a tab called
“Launch Report”; with this tab we will get the Results Summary of that particular
test script.
Disconnecting QTP from Quality Center:
You can disconnect from a Quality Center project or a Web server. Note that if
you disconnect QTP from a Web server without first disconnecting from a
project, QTP’s connection to that project database is automatically disconnected
Integration of QC with QTP (Contd):
54
DEMO
Demo
55
QC Access – IV & VS Lab
IP addresses of the USAA Test lab machines
1) 10.104.57.92 2) 10.104.57.179 3) 10.104.57.123 4) 10.104.57.100
URL to access Quality Center
http://10.103.91.73/qabin/
Login credentials
Login ids - Tester2, Tester3, Tester4 (case insensitive)
Passwords – (do contact [email protected], Phone 9791037245)
Constraints of usage
1. 15 concurrent licenses
2. Training licenses, Can not be used for evaluation or project purposes
3. Do not use at the cost of impacting assigned work or schedule
56
What is expected from you
Practice all learnt knowledge
Practice the features extensively using such as Synchronization,
Recovery Manager, Verification, Custom report etc.
Bring the learning to use in assigned work / schedule, as applicable
When in doubt about a feature or functionality – please get the
clarifications by
Contacting others around
Contacting speakers of this presentation
Contacting software testing community on arKMedis
Browsing on internet
When you accomplish some thing new – please share the knowledge
On software testing community on arKMedis
With IV&VS academy which takes care of Learning / Growth
57
What is expected from you
A few exercises
Problem – Automate LOS – Module (Create, Edit & Verify) – Mandatory for all
Demonstrate the creation of Automation scenarios for LOS – Module which includes Create, Edit & Verify.
Problem – Usage of Check Points
Demonstrate the usage of SQL, XML check points
Problem – Usage of Recovery Manager
Demonstrate the usage of Recovery Manger features
Problem – QTP Integration with QC
Demonstrate the integration of QC with logging defects via QTP scripting, passing dynamic parameters from
QC to QTP
Problem – Develop Custom Defined Functions to connect RDBMS Data Base
Demonstrate the creation of Custom functions to connect RDBMS Data base & execute queries
Problem – Develop Customized HTML/XML Result file
Demonstrate the creation of Custom functions to create/update user defined result into HTML/XML file
Problem – Result notification via email
Demonstrate the creation of Custom functions to notify the stake holders on Test Execution status via email
58
What is expected from you … continued
Conditions :
Do not combine multiple solutions into one document. Submit separate documents for responses to
each problem
1.No more than 2 people can be teamed up for answering a question. Please mention “the names of the
people” and “problem you are submitting the response for” on the first page.
2.Each response must be provided with the step-by-step explained along with screen shots wherever
possible
3.Use the demo/training version of the licenses for practice purpose. Do not use any client provided
licenses as far as possible
4.Last Date for completion: 20 days from circulation of materials
59
Questions and Answers