Date post: | 28-Oct-2014 |
Category: |
Technology |
Upload: | techwellpresentations |
View: | 262 times |
Download: | 2 times |
TF AM Tutorial
10/1/2013 8:30:00 AM
"Alan Page: On Testing"
Presented by:
Alan Page
Microsoft Corporation
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
Alan Page
Microsoft
Alan Page is a principal SDET—a fancy name for tester—on the Xbox console team at
Microsoft. Edging up on twenty years in software testing, Alan has previously worked on a
variety of Microsoft products including Windows, Windows CE, Internet Explorer, and Office
Lync. He spent some time as Microsoft’s director of test excellence where he developed and ran
technical training programs for testers throughout the company.
8/27/2013
1
Alan Page: On Testing
Test Ideas are…Ideas
8/27/2013
2
8/27/2013
3
The Adjacent Possible
Liquid Networks
The Slow Hunch
Serendipity
Error
Exaptation
Platforms
On Test Automation
On Exploratory Testing
On Test DesignOn Test Careers
On the Value of Test
On Test Management
On working with others
On The Death of Test
On the Future of Test
On Coding Testers
On Test Communities
On Agile and Testing
On Writing On Learning On Leading On Innovation
8/27/2013
4
Test Automation
8/27/2013
5
Exercise 1
8/27/2013
6
Exploratory Testing
8/27/2013
7
See Test DesignSee Test Design
Test Design
8/27/2013
8
Test This
What is Test Design?Testing Ideas
8/27/2013
9
Critique Product
Support Programming
B us iness (Customer ) Fac ing
Techno l og y Fac ing
Testing Quadrants: Marick, Crispin, Gregory, Hendrickson, others…
Runtime Analysis Tools
Functional Testing
Acceptance Tests
Scenarios
Usability Testing
World Readiness
Exploratory Testing
Take home / Beta Testing
Unit Testing
Code Coverage
Static Analysis
Code Churn Analysis
Performance Testing
Security / Privacy Testing
Stress Testing
Critique Product
Support Programming
B us iness (Customer ) Fac ing
Techno l og y Fac ing
Testing Quadrants: Marick, Crispin, Gregory, Hendrickson, others…
8/27/2013
10
Functionality First?
Why are we looking for functional bugs? They're easy. Stop looking for functional bugs. @docjamesw
“…testing, for a long time and in many places, has been myopically focused on functional correctness, rather than on value to people.”
-Michael Bolton
Random Internet Quotes “I don't care about features or plug-ins or add-ons or whatever. Just load the page quickly and let me get on with my work”
“ <some company> makes wonderfully full featured and functionally correct software, but my wife can’t figure out how to use it”
8/27/2013
11
Customer Focused Test Design
Customer Focused Test Design
8/27/2013
12
Upside Down Testing
8/27/2013
13
The Cost of Bugs
Is The Cost Curve Always True?
8/27/2013
14
Hypothetical Cost Curve –Functional Bugs
Hypothetical Cost Curve –Design Bugs
8/27/2013
15
Security Privacy
Reliability Performance
Usability Globalization
Reliabi l i ty
Send Web Query 10,000 times
Send malformed web query 10,000 time
8/27/2013
16
P e r f o rm a n c e
Test timing / latency of an action or scenario
Test large number of simultaneous connections
Scenario Testing
Customer Experience Validation
8/27/2013
17
Customer Experience Validation
Assess the experience
Collaborate with user research
Balance with other approaches
Evaluating a ScenarioEase of use
Responsiveness
Usefullness
Visual Appeal
8/27/2013
18
Example:
Showcase (marketing speak)
Scenario (project manager speak)
End-to-End Test (something
actionable)
(Optional) Variations
Xbox Knows You Better Identity
Step in front of the console, get recognized and see
my curated content
Light/Dark settings
Child/Adult/Gender/Height/Size/Apparel
Curated content
Ease of Use
• Am I able to complete the scenario?
• Is it complicated? Do I need to perform extraordinary steps to get what I need done?
• Are there glitches in the system that make it difficult?
• Was it hard to find how to execute this scenario? Are the features hidden?
• Is the experience consistent?
8/27/2013
19
Responsiveness
• Did I feel like the scenario was fast and fluid?
• At any point did I feel like I had to wait a frustrating amount of time?
• Was I effectively distracted while waiting for an action to take place? (ex. Movie or animation while I wait)
Usefulness
• Would you use this scenario yourself to accomplish this specific outcome?
• Does this scenario meet a need for our consumer?
• Does this let me do something that I want to do?
8/27/2013
20
Visual Appeal
• Is the experience exciting to see and hear?
• Is the UI polished?
• Does the UX make the experience enjoyable?
Evaluation Scale5 Love It! – You love it so much you’d shout it out at the top of
your lungs through a bullhorn from every rooftop you encounter
4 Like It – You like it a lot and might mention it during a lull in conversation at a dinner party
3 Meh – You can live with it and neither like it or hate it. It’s
nothing special. There are some improvements that can be made
2 Don’t Like It – The experience leaves a bad taste in your
mouth. You’d use it if you really had to otherwise, you’d stay away from it.
1 Hate it! – You hate it so much that you would only use it if you were under a Hogwarts compulsion spell
0 Not Implemented
8/27/2013
21
Scenario Love It Like It Meh Don't Like Hate
Do this 43 10 1 1 1
Do that 3 2 9 14 20
Do the other thing 40 14 5 0 1
Do something else 20 3 5 7 15
Test Management
8/27/2013
22
What is the role of a manager?What is the role of a manager?
What are the activities of a manager?What are the activities of a manager?
learn your A-C-B’s
8/27/2013
23
find the steepest learning curve
ride the gravy train
8/27/2013
24
who do you know?
follow the leader - lead the follower
8/27/2013
25
find a mentor
never let yourself get blocked
8/27/2013
26
try a new way
don‘t flip the bozo bit
8/27/2013
27
…and don’t burn bridges either
the three p’s
8/27/2013
28
a career is a journey, not a sprint
have fun