+ All Categories
Home > Documents > Capture and Replay

Capture and Replay

Date post: 03-Jan-2016
Category:
Upload: eugenia-lynch
View: 63 times
Download: 3 times
Share this document with a friend
Description:
Capture and Replay. Often used for regression test development Tool used to capture interactions with the system under test. Inputs must be captured; outputs may also be recorded and (possibly) checked. Examples: Telelogic Tau: save command history in simulator GUI testing tools - PowerPoint PPT Presentation
Popular Tags:
31
Capture and Replay Often used for regression test development Tool used to capture interactions with the system under test. Inputs must be captured; outputs may also be recorded and (possibly) checked. – Examples: – Telelogic Tau: save command history in simulator – GUI testing tools Capture requires a working system to be available already!
Transcript
Page 1: Capture and Replay

Capture and Replay

• Often used for regression test development

– Tool used to capture interactions with the system under test.

– Inputs must be captured; outputs may also be recorded and (possibly) checked.

– Examples:

– Telelogic Tau: save command history in simulator

– GUI testing tools

• Capture requires a working system to be available already!

Page 2: Capture and Replay

Testing on Replay

• During the replay of events, there are various output checking possibilities:

– Manual: user has to watch the system for anomalies

– Complete: all outputs were recorded during capture, and system must reproduce them “exactly”.

– “Exactly”: to the level of detail of the recording.

– Checkpoints: system output is only checked at certain points for specified values.

– These could be inserted manually or automatically.

Page 3: Capture and Replay

The capture record

• Inputs, outputs, and other information needed to reproduce a session with the system under test need to be recorded during the capture process.

• Examples:

– General information: date/time of recording, etc.

– System start-up information

– Events from test tool to system

– Point of control, event

– Events from system to test tool

– Checkpoints / expected outputs

– Time stamps

Page 4: Capture and Replay

Event-driven architecture

• Basis of GUI frameworks

– Various controls create events when they are created, activated, modified, deactivated, or disposed.

• Input devices create events as per their functions: key pressed, key released, mouse moved, …

• Events are sent to an event dispatcher

• “Listeners” register with the event dispatcher to receive events

– Listeners specify the type of events in which they are interested, and which method should be called when the event occurs

• When an event is received by the event dispatcher, notifications are sent to each registered listener for the type of event.

– Each listener has their event notification method called in turn.

Page 5: Capture and Replay

Event-driven architecture

Eventsource 1

Eventsource 2

Eventdispatcher

Eventlistener 1

Eventlistener 2

eventinstances

notifications

registration

Page 6: Capture and Replay

Integrating a Capture and Replay tool

• During the capture process, the tool will register as an event listener

– Event notification method for the tool will record the details of all events that occurred.

• During the replay process, the tool will register as an event source (possibly also as a listener)

– For mouse and keyboard events, the tool has to substitute for the actual devices as the event source.

– Replay events should be initiated at the same relative time as during the capture.

– Other controls issue events as usual (e.g. GUI button deactivated)

Page 7: Capture and Replay

Capture

Eventsource 1

Eventsource 2

Eventdispatcher

Eventlistener 1

Eventlistener 2

Captureand record

Page 8: Capture and Replay

Replay

Eventsource 1

Eventdispatcher

Eventlistener 1

Eventlistener 2

Replaytool input

[mouse + keyboardreplaced]

Page 9: Capture and Replay

Integration Alternatives

• Create native system events

– Example: The Robot class in Java will, for example, actually move the mouse cursor on the underlying platform, instead of just recreating the mouse event that would be generated.

• Analyze screen images

– A screen capture of a tool’s GUI can be stored while the tool is running, analyzed for widgets, and then used for replay.

Page 10: Capture and Replay

Tests

• Alternatives:

– During the capture process, the tool may record events, and then compare the results during playback.

– At test checkpoints, the tool can ask various controls for their state and record it.– Examples:

– Contents of text fields – GUI radio button properties:

– enabled / disabled– selected / not selected

Page 11: Capture and Replay

Integration options

• Compile tool classes and application together

– Similar to JUnit approach

– User has to determine object instances that should be monitored, and then register instances of tool classes as event source and listeners

– Connections are also set up to query controls as to their states.

– Provides better integration, more stable replays

– Tester has to learn how to use tool classes

• Use virtual machine registration (when there is a VM)

– Tool has to “discover” objects of interest

Page 12: Capture and Replay

Example: Jacareto

• Java-based open source GUI capture + replay tool

– Authors: C. Spannagel et. al.

– Web site: http://jacareto.sourceforge.net

• Tool sub-parts:

– Jacareto: event record and playback

– Picorder: command line interface

– CleverPHL: graphical user interface (for Jacareto)

Page 13: Capture and Replay

Jacareto

• Uses virtual machine registration

• Batch file and XML script used to start Java virtual machine, start the test tool, and load the application’s classes.

– Main method of application called by test tool as required.

• Jacareto has to discover all instances of subclasses of Component (including JComponent for Swing)

Page 14: Capture and Replay

Event records

• Stored as XML

• Information recorded:– Type of event: mouse move, key pressed, etc.

– Includes event-specific information: X-Y location of mouse cursor, which key was pressed, …

– Relative time from previous event– Needed to keep events synchronized

– Example: <KeyEvent procTime="0" duration="240" source="SampleDialog_(1).JRootPane_(1).JLayeredPane_(1).JPanel_(1).JTextField_(1)" class="javax.swing.JTextField"uuid="89a95465-8cae-467b-bc48-6422a63bcfc4" ID="401" component="null" root="SampleDialog_(1)" xPos="0" yPos="0" width="0" height="0" when="1175219983088" isConsumed="false">

<KeyInfo keyCode="65" keyChar="A" modifiers="1" /> </KeyEvent>

Page 15: Capture and Replay

XML record displayed for user

Page 16: Capture and Replay

Tests

• At various user-defined checkpoints, the state of a control can be tested.

• Example: Test if a text field contains the specified text

– Also specify that we want to stop if there is an error (i.e. ignore error is false), and that we do not want to correct the state of the control if it does not conform to the record.

<JTextComponentTest component="MainFrame_(1).JRootPane_(1).JLayeredPane_(1).JPanel_(2).JTextField_(1)" isRegExp="false" isCorrecting= "false isEnabled="true" isIgnoring="false" hasFocus="false">Hello, Alan</JTextComponentTest>

Page 17: Capture and Replay

UI for Test creation

Page 18: Capture and Replay

Test reports

Page 19: Capture and Replay

Example: Marathon

• Capture and playback tool for testing Java/Swing graphical user interfaces.

– www.marathontesting.com

• Uses the “Jython” (Python for Java) scripting language to store test cases.

• Test cases can be captured interactively, or written as Jython scripts.

• Assertions can be added to check the state of various components.

• Uses test runner that is a non-standard version of JUnit.

Page 20: Capture and Replay

Sample Jython test script

useFixture(default)

def test():

java_recorded_version = '1.5.0_11'

if window('GUI demo'):

click('Click me')

assert_p('TextField', 'Text', 'Hello, Alan')

close()

Page 21: Capture and Replay

Running a Marathon test

Page 22: Capture and Replay

Test report

Page 23: Capture and Replay

Eclipse TPTP Auto GUI recorder

• Capture and replay of GUI events and actions within the Eclipse IDE.

• Test suite is set up as an Eclipse plug-in project that integrates itself with the IDE.

• Any application that is also an Eclipse plug-in can be tested with the auto GUI recorder.

– The plug-in mechanism exposes how Eclipse can interact with the plug-in, and the auto GUI recorder uses this information to send and receive events to/from the plug-in.

Page 24: Capture and Replay

Pros and cons

• Advantages:

– Requiring the test application to be an Eclipse plug-in allows for better knowledge of the environment for the record / replay tool.

– Less difficulty and more reliability for locating user interface widgets.

– Tool can call methods directly, instead of simulating mouse operations and keystrokes.

• Disadvantages:

– Cannot be used for stand-alone applications

– Requires the use of Eclipse’s Standard Widget Toolkit (SWT), instead of Java’s Abstract Widget Toolkit (AWT) or the Swing toolkit.

Page 25: Capture and Replay

Additional features

• Verification hooks

– Points at which tests can be inserted to check the state of various widgets.

• Variable substitution

– Allows script to substitute variable values for variable names during run-time.

– Example: specification of temporary directory, relative file names

Page 26: Capture and Replay

TPTP Auto GUI recorder

Page 27: Capture and Replay

Action script

<macro version="1.0"> <shell descriptive="Test - AutomatedGUISuite - Eclipse SDK" referenceId="0" return-code="-1"> <command descriptive="Project..." type="select" referenceId="1"/> <shell descriptive="New Project" referenceId="2" return-code="0"> <command descriptive="Java Project“ type="item-select" referenceId="3"> <item referenceId="4"/> </command> <command type="focus" referenceId="3"/> <command descriptive="Next &amp;gt;" type="select" referenceId="5"/> <command type="modify" referenceId="6"> <![CDATA[A Java Project]]> </command> <command descriptive="Next &amp;gt;" type="select" referenceId="5"/> <command descriptive="Finish" type="select" referenceId="7"/> </shell> </shell></macro>

Page 28: Capture and Replay

TPTP Auto GUI recorder –test setup

Page 29: Capture and Replay

TPTP Auto GUI recorder –execution log

Page 30: Capture and Replay

TPTP Auto GUI recorder – execution log details

Page 31: Capture and Replay

TPTP API recorder

• Another tool being developed for Eclipse is an API recorder.

• Essentially, this performs capture and replay without having a user interface.

– Uses the TPTP profiler.

– As methods are called, the parameter values are stored on entry, and the values that are returned are stored as well.

– After execution, the saved parameter values are used to create JUnit test cases.

• Still in the “technology preview” phase.


Recommended