Post on 09-May-2015
description
transcript
Mobile Application TestingFrederik Carlierww Technology Strategist
11th September 2013
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
2013 Gartner CIO PrioritiesRanking 2013 2012 2011 2010 2009 2008
Analytics and Business Intelligence 1 1 5 5 1 1
Mobile Technologies 2 2 3 6 12 12
Cloud Computing 3 3 1 2 16 *
Collaboration Technologies (workflow) 4 4 8 11 5 8
Legacy Modernization 5 6 7 15 4 4
IT Management 6 7 4 10 * *
Customer Relastionship Management 7 8 18 * * *
Virtualization 8 5 2 1 3 3
Security 9 10 12 9 8 8
ERP Applications 10 9 13 14 2 2
* Not an option this yearSource: http://www.gartner.com/imagesrv/cio/pdf/cio_agenda_insights2013.pdf
2013 Gartner Strategic Technology Trends
Source: http://www.gartner.com/technology/research/top-10-technology-trends/
Tablets
Android63%
iOS32%
Windows Phone4%
Android iOS Windows PhoneWindows RT Others
Smartphone
Android79%
iOS13%
Windows Phone4%
BlackBerry3%
Linux1%
Android iOS Windows PhoneBlackBerry Linux SymbianOthers
Mobile Market Share
Source: http://www.idc.com/getdoc.jsp?containerId=prUS24253413
Source: http://www.idc.com/getdoc.jsp?containerId=prUS24257413
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
Waterfall approach for testing
• Party• Project Definition• Project Realization• Testing (used a project buffer)• Deployment to production• Party
Result: low return on investment on testing effort
Test Case Implementation & Execution – ROI?• Reusablity across
• Different mobile OS versions (new)• Different mobile OS vendors (new)• Different hardware (new)• Different versions of the app under test• Different browser
• Independent of• Screen resolution• Keyboards• Localization
• Re-use of existing• Testing tools• Automation expertise• Scripting / Programming language
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
Manual Testing
• Not reproducible• Time consuming• On Premise?• Test Coverage
Jamo Solutions - Mission
The mission of Jamo Solutions is to provide technology and expertise that enables successful testing and monitoring solutions around the mobile end user experience.
Object Attributes & Identification
• Objects are identified by a unique combination of attributes
• This combination can be set seperately for every object or globaly for all objects of the same type
• Access to all attributes of an object
• Key advantage of object recognition based approach
• Depending on the types of application you are creating — mobile browser, native, hybrid and the level of interactivity (use of gesture sets, access to platform features), this lower-level access will be the stronger path.
Gartner, Magic Quadrant for Integrated Software Suites, 11 July 2013
A typical UI
Consider the differences between testing this application using OCR (screen scraping/bitmap recognition) and object recognition based approach
• All buttons share the same label• All buttons share the same form• All buttons share the same size
• Object recognition: direct access• Reliable• Reusable
• OCR: Need for more intelligence• Error prone• Reduced reusability
Portrait view vs. landscape view
Native, Web & Hybrid Applications
• Native app: packaged application deployed to the phone
• Web app: accessed through the browser
• Hybrid app: native app with embedded web views
Native App• Java • C# • C++• Objective C• ....
Web App• HTML• CSS• Java Script• Runs inside a
browser
Hybrid App
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
Functional Testing - Overview
Mobile Test Automation• M-eux Test allows you to automate your mobile testing
scenarios• Based on future-proof, object recognition based
technology (no jail breaking / rooting, no OCR, iOS7 compatible)
• Wide range of support across mobile platforms (iOS, Android, Windows, BlackBerry)
Demo
Functional Testing - Overview
Integration with exiting tools & processes• Integration with industry-standard development tools
(HP UTP/QTP, Microsoft,...)• Integration with application life cycle management tools
(HP ALM/QC, Microsoft, IBM RQM, ...)• Allows effective re-use of and integration with your
existing testing processes and tools• Minimizes the total investment, increases the ROI
Browser-Based Functional Testing
What?• Allows you to automate the testing of web applications on mobile platforms
• Based on our proven object recognition based technology
• Allows for seamless cross-browser, cross-platform, cross-device scripting
• You test the real app; on real devices or simulators/emulators
Demo
Browser-Based Functional Testing
Cross Browser Script
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
26
Performance/Capacity Management
Aspects of performance/capacity management• Performance Testing
• Load Generation• Response Time Measurement
• Performance Monitoring
04/11/2023 © Jamo Solutions
27
Performance/Capacity Management
Load Generation• M-eux Test allows you to create automated test cases• You can replay these devices on multiple devices
simultaneously (physical or virtual)• We integrate with load generation tools (LoadRunner,
VS,…) through UFT/VS integration
04/11/2023 © Jamo Solutions
28
Performance/Capacity Management
Response Time Measurement• M-eux Test allows you to create scripts that execute an
action and wait for the operation to complete , and report timing data
• You can integrate this data with ALM tools (HP, Microsoft,…) to track performance over time
04/11/2023 © Jamo Solutions
29
Performance/Capacity Management
Performance Monitoring• M-eux Test allows you to create scripts that execute an
action and wait for the operation to complete or fail, and report timing data
• You can integrate this data feed with your NOC operational tools (HP, Microsoft,…)
04/11/2023 © Jamo Solutions
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
31
Best Practices Management
• Re-use scripting best practice from PC world:• Descriptive programming• Exception handling• Object reconfiguration• Frameworks ( Linear framework, function decomposion
framework, keyword-driven framework, data-driven framework, etc.)
• Best pracitices on setup• Lab setup• Scripting setup
Agenda
• The Mobile Market• Testing for Desktop and Mobile Apps: Now and Then• Methods for Test Automation• Functional Testing• Performance Testing• Frameworks• Conclusion
Conclusion
• Let your test be reproducable• Manage frequent test cycles• Create re-usable tests• Object recognition is the way to go• Change your system and app as less as possbile by
instrumentation• Note that the app is not the only thing to test
Questions?frederik.carlier@jamosolutions.com+32 485 869 861
www.jamosolutions.com
Thank You!frederik.carlier@jamosolutions.com+32 485 869 861
www.jamosolutions.com