Date post: | 07-Jan-2017 |
Category: |
Technology |
Upload: | qualitest-group |
View: | 352 times |
Download: | 0 times |
Testing for a Great App and Web Experience
16:00 GMT November 8th 2016
Webinar Presenters:
Aviram ShottenUK BM and Global Head of Partnerships and Alliance
Stephen Ellwood-WadeMobile Product Manager
2
Agenda
| Who are we
| Pillars of Quality
| User Focused Testing
| How do the Racing Post partner with QualiTest
| Product Feasibility Testing
| UAT and The Crowd
| Regression and Smoke Testing
| Q&A
3
About QualiTest Group
| World’s 2nd Largest Independent Testing Company
| QualiTest was recognized as the only visionary pure play QA and testing Service Company in the “ Magic Quadrant”
4
Pillars of Quality
Predictable Quality & Ux
Func
tion
al
Acce
ssib
ilit
y Loca
lisat
ion
Perf
orm
ance
Secu
rity
Usa
bilit
y an
d U
x
5
Predictable Quality & Ux
Func
tion
al
Acce
ssib
ility
Loca
lisat
ion
Perf
orm
ance
Secu
rity
Usa
bilit
y an
d U
x
Cross Platform
Test Automation
Desktops
Phones
TabletsBrowsers
Operating Systems
| One solution – automation and devices management
| X-Platform
| Scalable
| Consider open source vs. ease of use
Cross-Platform Test Automation
6
Predictable Quality & Ux
Func
tion
al
Acce
ssib
ility
Loca
lisat
ion
Perf
orm
ance
Secu
rity
Usa
bilit
y an
d U
x
Managed Crowd Testing Local and Global Functional Testing Localisation
Testing and Quality Champion owns the delivery of crowd testing and meeting the KPI and SLA
Testing for content, context, geo-restrictions/ local integration, local addresses, language
Dedicated Crowd Test Lead
Exploratory Functional + Test execution hours
Ux/ Cx
Exploratory Test Cycles
Dedicated Test Case Execution Hours
Every testing cycle will be accompanied by a set of standard and specific set of Ux questions about the SUT, will be used for improvement and for benchmarking
Crowd Delivery
Internal Evangelist
7
Predictable Quality & Ux
Func
tion
al
Acce
ssib
ility
Loca
lisat
ion
Perf
orm
ance
Secu
rity
Usa
bilit
y an
d U
x
Accessibility| Works for people with
Accessibility needs| Deliverables:
| WCAG 2.0 A, AA and AAA Compliant| Automated Tools Testing| Professional manual functional by
dedicated accessibility experts| Utilize testers with genuine accessibility
needs| Focused on both Mobile and Web
Applications
8
Predictable Quality & Ux
Func
tion
al
Acce
ssib
ility
Loca
lisat
ion
Perf
orm
ance
Secu
rity
Usa
bilit
y an
d U
x
Localisation
| The challenge of making an English app a global app| Localisation process and Development process should work
together closely from early stages:• Localisation Prototyping – readiness for multilingual app• Coverage • Timeline• Key concerns and sector
| The QA of a localised product should include functional testing and specifically linguistic and cosmetic testing.
9
Predictable Quality & Ux
Func
tion
al
Acce
ssib
ility
Loca
lisat
ion
Perf
orm
ance
Secu
rity
Usa
bilit
y an
d U
x
App Performance – Complete Vision
• Traditional load generation of Virtual (synthetic) users
• Supports the • Hybrid load approach
• Helps analysing the implications of networks quality on performance
• Injecting low VU load • Stats gathering from servers and google
analytics• Costumed metrics analysis• Immediate notification and scheduled
automated• reports 24/7 Grey-
box Monitoring
Network Quality
Virtualisation
Protocol Level
Performance
Functional Automation/
Crowd
10
Predictable Quality & Ux
Func
tion
al
Acce
ssib
ility
Loca
lisat
ion
Perf
orm
ance
Secu
rity
Usa
bilit
y an
d U
x
Security Testing| Agile’s quest for Speed drives us toward embedded
security testing – Deliver Secured SW, Fast | Automated (and Manual) security code scan| Automated Pen-Test (automated dynamic analysis) for
web-based applications and Mobile behavior analysis for Apps
| Manual “Ethical Hacking” Pen-Test
11
Predictable Quality & Ux
Func
tion
al
Acce
ssib
ility
Loca
lisat
ion
Perf
orm
ance
Secu
rity
Usa
bilit
y an
d U
x
Ux/Cx vs. Usability
12
User Focused Testing
Deployment Go live
Continuous Performance Monitoring
Pre-Test Activities
Create Baseline Expectations
End User ExperienceEarly Test and Report
Accessibility
13
Hybrid Load /Hybrid Performance
Ux Focus & Device Fragmentation
Continual improvement
Continual feedback
Section 508 and WCAG 2.0 A, AA and AAA focus
Efficient Baseline
Functional
Security
Maintenance Ux Surveys
Surveys &Trend monitoring
Feature Ux ValidationSuggestions for Improvement
About Racing Post“Racing Post aims to be most valued brand for regular horse and dog racing bettors in the UK and Ireland and their preferred partner for form study (online and offline) and preferred digital betting partner. ”
We have spent the last few years, and in particular the last six months, getting to know our customers better.
It has become clear that we cannot be all things to all people and in order to grow, we need to focus on those areas which will drive growth.
14
Racing Post and QualiTest
15
To achieve the goals set in our vision Racing Post make use of two flexible services that QualiTest provide:
| Crowd testing| Special test case hours
These can be applied in a number of ways to support our product development process:
| Hypothesis testing| New feature testing| Regression and Acceptance testing
General Crowd Testing
16
Racing Post leverages QualiTest as part of regression testing and as a fail safe mechanism. As part of each round of testing we can define the following:
| Scope of testing| Known bugs - out of scope| Raised bug detail requirementsGeneralSpecific (crashes for example)| Devices and OSs
Following a release we often schedule another round to cover smoke testing. Although this is done internally this can help especially if bugs have been raised through other channels.
Product Feasibility TestingIn order to deliver value, Racing Post have to ensure products that are delivered are of use to our customers. Using the dedicated test case hours, we use for feasibility testing to allow for Quantitative testing.
Example Hypothesis: If we apply a Nav-Bar instead of a drop down nav on our Android app engagement and conversion will increase in app.
17
Product Feasibility TestingA number of questions were posed to get a well rounded response around whether we should try changing our navigation. The response was excellent and we now have the piece of work lined up as a future A/B test.
18
UAT and the CrowdQualiTest and the Racing Post are specifically recruiting bettors in the UK & Ireland to be part of our test groups meaning Racing Post can involve the service in the UAT process. Specialised scripts can be created in order to target any new functionality for an upcoming release.
Below is a snippet of a test script covering the specifics of the Tricast/Forecast betting integration.
19
In short
20
QualiTest help link up our product development roadmap in a fewhelpful ways in order for Racing Post to be able to deliver qualityproducts.
| Product feasibility testing (quantitative)| UAT| Regression testing| Smoke testing
Trending| Feature Trending
| Individual tracking of features and functionality on demand
| Results broken down by feature/question
| Trend-line reporting
21
Realistic Usable Recommendations
| Suggested outputs may include:| Functional changes, layout changes| Device Fragmentation Cycles| Additional or updated internal / external test case scripts| Specific cycles focusing on Ux Wireframes | Non-traditional output may include: Additional focus required on Localisation/Accessibility
22
Benefits – Client Feedback
It helps you react to changes, quickly – player feedback, new devices etc.
We can test and learn before we go live
It helps build confidence in the products and systems we build for your consumers
It gives you the device and OS coverage you need during your testing
It saves you money as you don’t have to buy all devicesIt helps give real user
feedback during your delivery cycles
It allows for you to easily test out of hours and on weekends
It helps shorten and reduce the cost of your test cycles
23
24