Date post: | 19-May-2015 |
Category: |
Technology |
Upload: | stag-software-private-limited |
View: | 1,761 times |
Download: | 3 times |
Are your test cases fit for automation?Powered by HBT
T AshokFounder & CEOSTAG Software Private Limited
in.linkedin.com/in/AshokSTAG Ash_Thiru
Webinar Presentation Nov 27, 2013
© 2013 STAG Software Private Limited. All rights reserved.
Good automation is more than frameworks, tools and scripting.
2
The inherent structure of scenarios/cases plays a vital role in ensuring rapid development and optimised maintenance.
An ill formed structure makes the script complex making it error prone and costly to maintain.
We will discuss an interesting idea "Fitness for automation" wherein we examine properties of test scenarios/cases to ascertain if they are good enough to automate.
Also what needs to be done in the case where it is "unfit" ?
This is one of the applications of "Hypothesis Based Testing" (HBT) to ensure rapid automation and optimised maintenance.
© 2013 STAG Software Private Limited. All rights reserved.
Expectation and Challenges
3
๏ Develop scripts rapidly๏ Adapt to newer functionally quickly
๏ Complexity of scenario
© 2013 STAG Software Private Limited. All rights reserved.
Structure of a script
4
Setup
Execute
Oracle
Cleanup
Prerequisites to be done before “excitation”
“Excite” the system with inputs
Check outcomes to determine pass/fail
Inverse of ‘setup’
© 2013 STAG Software Private Limited. All rights reserved.
What contributes to complexity?
5
Setup
Execute
Oracle
Cleanup
Prerequisites to be done before “excitation”
“Excite” the system with inputs
Check outcomes to determine pass/fail
Inverse of ‘setup’
#Steps to do this
#Inputs and the ‘kind of inputs(+/-)’Order, dependency, source
The variety of behaviours to check & #Checks for each behaviour
#Steps to do this
© 2013 STAG Software Private Limited. All rights reserved.
“Clarity of purpose”
6
Setup
Execute
Oracle
Cleanup
Prerequisites to be done before “excitation”
“Excite” the system with inputs
Check outcomes to determine pass/fail
Inverse of ‘setup’
What “type of bug” are we looking for?...enables focused & simpler checking
© 2013 STAG Software Private Limited. All rights reserved.
“Clarity of purpose”
7
Setup
Execute
Oracle
Cleanup
Prerequisites to be done before “excitation”
“Excite” the system with inputs
Check outcomes to determine pass/fail
Inverse of ‘setup’
... results in ‘just-enough’ steps,enables smaller/crisper script
© 2013 STAG Software Private Limited. All rights reserved.
Characteristics of TS that affect ‘automatability’
8
Clarity of purpose1
Distinctiveness of behaviours (Single output)2
Similarity of inputs3
What types of defects are being targeted by the test cases?
Do TS contain multiple behaviours ? or Single minded?
Are the various inputs for the TS similar ?
© 2013 STAG Software Private Limited. All rights reserved. 9
Quality Levels & PDTs - Powered by HBT
Input cleanliness
Input interface cleanliness
Structural integrity
Behaviour correctness
Environment cleanliness
Attributes met
Flow correctness
Clean Deployment
End user value
L1
L2
L3
L4
L5
L6
L7
L8
L9
That inputs are handled wellPDTs related to Input data correctness
That the functional behaviour is correctPDTs related to Functionality
That the internal structure is robustPDTs related to Structural aspects
That the user interface is cleanPDTs related to UI
That end-to-end flows work correctlyPDTs related to Flow behaviour, Interactions
That it does not mess up the environmentPDTs related to Resource leaks, Compatibility...
That the stated attributes are metPDTs related to Performance, Load, Volume......
That it deploys well in the real environmentPDTs related to Compatibility, Migration
That user expectations are metPDTs related to User flows, User experience
Clarity of purpose1
© 2013 STAG Software Private Limited. All rights reserved. 10
Input cleanliness
Input interface cleanliness
Structural integrity
Behaviour correctness
Environment cleanliness
Attributes met
Flow correctness
Clean Deployment
End user value
L1
L2
L3
L4
L5
L6
L7
L8
L9
Functional automation covers...
That inputs are handled wellPDTs related to Input data correctness
That the functional behaviour is correctPDTs related to Functionality
That the internal structure is robustPDTs related to Structural aspects
That the user interface is cleanPDTs related to UI
That end-to-end flows work correctlyPDTs related to Flow behaviour, Interactions
That it does not mess up the environmentPDTs related to Resource leaks, Compatibility...
That the stated attributes are metPDTs related to Performance, Load, Volume......
That it deploys well in the real environmentPDTs related to Compatibility, Migration
That user expectations are metPDTs related to User flows, User experience
© 2013 STAG Software Private Limited. All rights reserved.
Scripts automate scenarios, not cases
11
C1
C2
C3
C4i1
i2
i3}
Outputs1
2
3
45
Inputs Conditions
Behaviour StimuliEntity under test
Conditions govern behaviourOutcome is Test Scenarios
Input (data) is StimuliOutcome is Test Cases
Distinctiveness of behaviours2
Similarity of inputs3
© 2013 STAG Software Private Limited. All rights reserved.
Fitness for automation
12
Does the TS look for a variety for defects?1
Does the TS check for a specific behaviour?2
What types of defects are being targeted by the test cases?
Is a specific behaviour being checked?
© 2013 STAG Software Private Limited. All rights reserved.
What if TS are “unfit”?
13
The TS is looking for a variety for defects?1
The TS is checking a variety of behaviours2
Simplify a scenario.Simplify it by decomposing it into various levels.“Levelize”
Make the single minded.Split the scenarios such that each TS checks a specific combination of conditions.
© 2013 STAG Software Private Limited. All rights reserved.
Hypothesis Based Testing - HBT
14
System Under Test
Cleanliness Criteria
Potential Defect Types
Test CasesRequirementstraceability“what to test”
Faulttraceability“test for what”
should satisfy impeded by
© 2013 STAG Software Private Limited. All rights reserved.
HBT OverviewSIX staged purposeful activities, powered by EIGHT disciplines of thinking
15
D1D2
D4
D3
D5D6
D7
D8
Core Concepts
S1
S2
S3S4
S5
S6SIX Stages of DOINGS1: Understand expectationsS2: Understand contextS3: Formulate hypothesisS4: Devise proofS5: Tooling supportS6: Assess & Analyse
EIGHT Disciplines of ThinkingD1: Business value understandingD2: Defect hypothesisD3: Strategy & PlanningD4: Test designD5: ToolingD6: VisibilityD7: Execution & ReportingD8: Analysis & Management
Uses 32 Core ConceptsFor Problem solving - Techniques, Principles, Guideline
Click here to know more about HBT.http://stagsoftware.com/blog?p=570
© 2013 STAG Software Private Limited. All rights reserved.
HBT is the intellectual property of STAG Software Private Limited.STEMTM is the trademark of STAG Software Private Limited.
www.stagsoftware.com
@stagsoft
blog.stagsoftware.com
Connect with us...
Are your test cases fit for automation? Powered by HBT
Thank you.