Post on 22-Dec-2014
description
transcript
GUI TestingGUI Testing
By Norbert HachéBy Norbert Haché
ContentsContents
What is GUI testingWhat is GUI testing Elements of GUI testingElements of GUI testing Old Approach (TRUMP Project)Old Approach (TRUMP Project) ScriptingScripting Capture / ReplayCapture / Replay Full Test IntegrationFull Test Integration Evaluation of CAPBAKEvaluation of CAPBAK DemoDemo
What is GUI TestingWhat is GUI Testing
Graphical User Interface (GUI) Graphical User Interface (GUI) TestingTesting
Methods used to identify and Methods used to identify and
conduct GUI tests, including the conduct GUI tests, including the use of automated tools. use of automated tools.
Source: (www.systemhouse.mci.com)Source: (www.systemhouse.mci.com)
Elements of GUI TestingElements of GUI Testing
A processA process A GUI Test PlanA GUI Test Plan A set of supporting toolsA set of supporting tools
source : www.csst-technologies.com/guioutln.htmlsource : www.csst-technologies.com/guioutln.html
Old Approach Example Old Approach Example (TRUMP)(TRUMP)
Was Done by manually stepping Was Done by manually stepping through thousands of pages of test through thousands of pages of test procedures.procedures.
Labour intensive, highly error Labour intensive, highly error prone.prone.
Needed to be redone each time Needed to be redone each time regression testing was required.regression testing was required.
Very expensive.Very expensive.
ScriptingScripting
Another Programming Language.Another Programming Language. Needs to be subjected to some Needs to be subjected to some
form of formal verification.form of formal verification. Eliminates human error during Eliminates human error during
execution of the test.execution of the test. Can be used (sometimes with Can be used (sometimes with
modifications) for regression modifications) for regression testing.testing.
Scripting cont’dScripting cont’d
Script Replay tool
Capture/Replay ToolsCapture/Replay Tools
A capture replay tool is a set of A capture replay tool is a set of software programs that capture user software programs that capture user inputs and stores it into a format (a inputs and stores it into a format (a script) suitable to be used at a later script) suitable to be used at a later time to replay the user inputs.time to replay the user inputs.
Note: Throughout this presentation I use Note: Throughout this presentation I use capture/replay and capture/playback to mean capture/replay and capture/playback to mean the same.the same.
Available Capture replay Available Capture replay toolstools
QA Partner [Segue Software, Inc)QA Partner [Segue Software, Inc) Xrunner & WinRunner (Mercury Interactive)Xrunner & WinRunner (Mercury Interactive) QC/Replay (CenterLine Sftware)QC/Replay (CenterLine Sftware) Evaluator (Eastern Systems)Evaluator (Eastern Systems) CAPBAK (Software Research, Inc)CAPBAK (Software Research, Inc) Vermont High Test Plus (Vermont Creative Vermont High Test Plus (Vermont Creative
Software)Software) WITT (IBM)WITT (IBM) ATS/X-Tester (Automated Testing Solutions Inc.)ATS/X-Tester (Automated Testing Solutions Inc.)
source : www.stlabs.com/marick/faqs/t-gui.htmsource : www.stlabs.com/marick/faqs/t-gui.htm
Capture Replay toolsCapture Replay tools
Capture/Replay
Script
Capture Replay ModelsCapture Replay Models
OS
Device Driver Device DriverDevice Driver
Capture/Playback tool Application
Application Events Self Events
Capture Replay Models Capture Replay Models (cont’d)(cont’d)
OS
Device Driver Device DriverDevice Driver
Application
Application Events Self Events
Capture/Playback tool
Capture Replay Models Capture Replay Models [cont’d][cont’d]
OS
Device Driver Device DriverDevice Driver
Capture/Playback tool
Application
Application Events Self Events
Full Test IntegrationFull Test Integration
Major drawback in Capture/Playback Major drawback in Capture/Playback tool is that when the GUI changes, tool is that when the GUI changes, input sequences previously recorded input sequences previously recorded may no longer be valid.may no longer be valid.
A test system which makes the A test system which makes the maintenance of Capture/Playback maintenance of Capture/Playback generated test scripts easy and fast is generated test scripts easy and fast is a must for such a tool to be of any a must for such a tool to be of any use.use.
Full Test Integration Full Test Integration (cont’d)(cont’d)
A capture/playback tool that support the A capture/playback tool that support the following capabilities could be used in a more following capabilities could be used in a more capable and fully integrated test development capable and fully integrated test development environment:environment:• record scripts of user/system interactionsrecord scripts of user/system interactions• user access to scripts for user access to scripts for
editing/maintenanceediting/maintenance• user ability to insert validation commands in user ability to insert validation commands in
the scriptthe script• allows replay of the recorded script.allows replay of the recorded script.
Full Test Integration Full Test Integration (cont’d)(cont’d)
A fully integrated GUI test development A fully integrated GUI test development environment would also require the following environment would also require the following additional characteristics:additional characteristics:• Script editing using higher level abstractions Script editing using higher level abstractions
such as icons etc.such as icons etc.• High level view of what functionality is High level view of what functionality is
being tested.being tested.• The ability to generate many variations of a The ability to generate many variations of a
recorded script without having to manually recorded script without having to manually edit the script itself.edit the script itself.
Full Test Integration Full Test Integration (cont’d)(cont’d)
A product called TDE under A product called TDE under development by Siemens is to provide development by Siemens is to provide exactly this kind of functionality exactly this kind of functionality (currently only at the prototype level)(currently only at the prototype level)
Source: ISSTA 98 Proceedings of the ACM SigSoft Source: ISSTA 98 Proceedings of the ACM SigSoft International Symposium on Software Testing and International Symposium on Software Testing and Analysis.Analysis.
Full Test Integration Full Test Integration (cont’d)(cont’d)
TDE capabilities:TDE capabilities:• uses higher level scenario language instead of uses higher level scenario language instead of
scripting. This allows graphical editing of the test scripting. This allows graphical editing of the test sequence and easy creation of variations.sequence and easy creation of variations.
• Has a test designer, which through user Has a test designer, which through user interactions with the system, builds an internal interactions with the system, builds an internal model of the system’s GUI to produce a high level model of the system’s GUI to produce a high level test design representing many executable scripts.test design representing many executable scripts.
• Test design library.Test design library.• Test generator engine to convert high level Test generator engine to convert high level
scenario into tests scripts.scenario into tests scripts.
TDE
Test Scripts
TestScript
GUIInfo
GUIReplay
record
Test DesignLibrary
Test DesignerTest
DesignTest
GenerationEngine
Tester
Specify Variations
Test Scenario ScreenTest Scenario Screen
TDE SUT RepresentationTDE SUT Representation
Full Test Integration Full Test Integration (cont’d)(cont’d)
When the GUI changes, instead of editing the When the GUI changes, instead of editing the hundreds of generated test scripts, the editing hundreds of generated test scripts, the editing is done at the scenario level where it is much is done at the scenario level where it is much easier and faster. This is followed by the easier and faster. This is followed by the automatic regenaration of the test scripts from automatic regenaration of the test scripts from the scenario.the scenario.
TDE can detect and analyse the differences TDE can detect and analyse the differences between a new GUI and its previous version. It between a new GUI and its previous version. It then makes assumptions about the changes then makes assumptions about the changes that can be subsequently overriden by the that can be subsequently overriden by the tester prior to script regeneration.tester prior to script regeneration.
Full Test Integration Full Test Integration (cont’d)(cont’d)
Using the prototype it was shown that in Using the prototype it was shown that in 30 minutes, a tester was able to create a 30 minutes, a tester was able to create a single scenario that produced 2500 test single scenario that produced 2500 test cases which exercised every significant cases which exercised every significant combination of input values and action combination of input values and action choices available for the particular choices available for the particular application.application.
Source: ISSTA 98 Proceedings of the ACM SigSoft Source: ISSTA 98 Proceedings of the ACM SigSoft International Symposium on Software Testing and AnalysisInternational Symposium on Software Testing and Analysis
Evaluation of CAPBAKEvaluation of CAPBAK
CAPBAK is a capture/replay tool CAPBAK is a capture/replay tool developed by Software Research developed by Software Research Inc of San Fransisco.Inc of San Fransisco.
Versions are available for Windows Versions are available for Windows 95, NT as well as for X-Windows.95, NT as well as for X-Windows.
My evaluation for the Windows 95 My evaluation for the Windows 95 version.version.
CAPBAK User InterfaceCAPBAK User Interface
CAPBAKCAPBAK
Very simple and intuitive to use.Very simple and intuitive to use. Documentation provided was dated.Documentation provided was dated. Had several bugs that need to be Had several bugs that need to be
fixed to make the product more fixed to make the product more robust.robust.
Provides many synchronization Provides many synchronization mechanisms, not all of which I could mechanisms, not all of which I could get to work properly.get to work properly.
CAPBAKCAPBAK
Provides OCR to save window text Provides OCR to save window text for baselining window image.for baselining window image.
Provides automatic detection of Provides automatic detection of changes to the window at the changes to the window at the bitmap and text level.bitmap and text level.
Provides an object mode to record Provides an object mode to record at the widget level. (I could not get at the widget level. (I could not get this to work properly).this to work properly).
CAPBAKCAPBAK
For more information, you may For more information, you may read the hand out on my read the hand out on my evaluation of CAPBAK.evaluation of CAPBAK.
DemoDemo
I have developed a basic MS Windows I have developed a basic MS Windows application that experiments with the basic application that experiments with the basic principles of user entry capture.principles of user entry capture.
It is very basic and does not attempt to filter It is very basic and does not attempt to filter out any particular events. I did it to show how out any particular events. I did it to show how easy it is to do such thing using the first model easy it is to do such thing using the first model showed earlier in the presentation which I am showed earlier in the presentation which I am showing here again.showing here again.
OS
Device Driver Device DriverDevice Driver
Capture/Playback tool Application
Application Events
USER32.EXE
Applicationbeing
captured
CaptureMain Window
Application
CaptureDLL
Console output
SummarySummary
GUI Testing using capture/replay GUI Testing using capture/replay tools is a useful technology if it can tools is a useful technology if it can be used within a test system that be used within a test system that allows efficient and high level allows efficient and high level maintenance capabilities of the maintenance capabilities of the test design.test design.