Acceptance Testing (1)Acceptance Testing (1)

Jerzy.Nawrocki@put.poznan.plwww.cs.put.poznan.pl/jnawrocki/require/

Copyright, 2006 Jerzy R. Nawrocki

Requirements Engineering & Project ManagementLecture 7

J.Nawrocki, Acceptance Testing

The V ModelThe V Model

Requirements spec.Requirements spec.

CodingCoding

DesignDesign Integration testsIntegration tests

Acceptance testsAcceptance tests

Unit testsUnit tests

J.Nawrocki, Acceptance Testing

Model VModel V

Requirements spec.Requirements spec. Acceptance testsAcceptance tests

DesignDesign Integration testsIntegration tests

CodingCoding Unit testsUnit tests

Customer

J.Nawrocki, Acceptance Testing

IntroductionIntroduction

Test-driven development

Acceptance tests: customer

Automated accept. testing

A. testing: customer+tester

Progress measurement based on acceptance testing

J.Nawrocki, Acceptance Testing

Extreme Programming Extreme Programming

Acceptance testing -> Progress measurement

Weeks

Num

ber o

f tes

ts

J.Nawrocki, Acceptance Testing

Introduction – What is Rational Robot?Introduction – What is Rational Robot?

Baseline Baseline

Robot

J.Nawrocki, Acceptance Testing

What is test automation?What is test automation?

That’s simple!

1. Buy a test execution tool.

2. Record the manual tests.

3. Play the tests back whenever you want to.

„Just as there is more to software design than knowing a programming language, there is more to automating testing than knowing a testing tool.”

-- M. Fewster & D. Graham

J.Nawrocki, Acceptance Testing

Manual or Automatic Testing?Manual or Automatic Testing?

The amount of effort to automate a test case:

2 – 10 times the effort required to run the test manually (!)

J.Nawrocki, Acceptance Testing

Introduction – Rational Robot and test-first codingIntroduction – Rational Robot and test-first coding

Remarks:

• When you start testing an application is ready.

• Rational Robot is for regression testing, not for test-first coding.

• The only chance for test-first coding is SQABasic.

J.Nawrocki, Acceptance Testing

Testing HTML ApplicationsTesting HTML Applications

StartBrowser [ URL$, ] [ WindowTag= Name$ ]

• Starts the browser,• enables HTML testing, and • loads a page if specified.

Web page address

Identifies this instance of the browser

StartBrowser "http://www.cs.put.poznan.pl/jnawrocki/models/", "WindowTag= Parent"

J.Nawrocki, Acceptance Testing

Testing HTML ApplicationsTesting HTML Applications

Browser action$, recMethod$, parameters$

• Performs an action on a Web browser

Back, Forward, NewPage, SetFrame,

CloseWin, ..

Wait= % default

time is 30s

HTMLTitle=$, Name=$,

"" ..

Browser NewPage, ""

Wait for the topmost frame

J.Nawrocki, Acceptance Testing

Testing HTML ApplicationsTesting HTML Applications

HTML action%, recMethod$, parameters$

• Performs a mouse action on an HTML tag.

Click, .. Coords= x, y

HTML Click, "Name= Autor", "Coords= 10,12"

HTMLTitle=$, Name=$,

"" ..

J.Nawrocki, Acceptance TestingTestDatastore\DefaultTestScriptDatastore\TMS_Script\vp\test7.Weryfikacja_daty.base.grd

SQABasicSQABasic

HTMLDocument Click, "Type=HTMLDocument;Index=1", "Coords=25,14 "

Result = HTMLTableVP (CompareData, "Type=HTMLTable;Index=1", "VP=Weryfikacja_daty")

J.Nawrocki, Acceptance Testing

An alternative to Rational RobotAn alternative to Rational Robot

httpUnit

Based on jUnit

For programmers!

J.Nawrocki, Acceptance Testing

Data-driven scriptsData-driven scripts

Data-driven scripting = Test inputs stored in a separate (data) file.

countries, Sweden, USA, countries2

countries, France, Germany, test2

countries, Austria, Italy, test3

J.Nawrocki, Acceptance Testing

Data-driven scriptsData-driven scripts

Data-driven scripting = Test inputs stored in a separate (data) file.OpenFile ‘ScribbleData’For each record in ScribbleData Read INPUTFILE Read NAME1 Read NAME2 Read OUTPUTFILE Call ScribbleOpen (INPUTFILE) FocusOn ‘Scribble’ SelectOption ‘List/Add Item’ FocusOn ‘Add Item’ Type NAME1 LeftMouseClick ‘OK’ ...EndFor

countries, Sweden, USA, countries2countries, France, Germany, test2countries, Austria, Italy, test3

ScribbleData

Control script

J.Nawrocki, Acceptance Testing

A more sophisticated data-driven scriptA more sophisticated data-driven scriptOpenFile ‘ScribbleData’Read INPUTFILECall ScribbleOpen (INPUTFILE)Go to next record (ie row)For each record in ScribbleData Read ADDNAME If ADDNAME <> Blank Then { FocusOn ‘Scribble’ SelectOption ‘List/Add Item’ FocusOn ‘Add Item’ Type ADDNAME LeftMouseClick ‘OK} Read MOVEFROM Read MOVETO ...EndFor

ScribbleData

countries

SwedenUSA

4 1Norway

27

countries2

Control script

J.Nawrocki, Acceptance Testing

Data-driven scriptsData-driven scripts

Software under test

Key automated regression test cases

Additional automated regression test cases

J.Nawrocki, Acceptance Testing

Keyword-driven scriptsKeyword-driven scripts

Keyword-driven script = A data-driven scripts augmented with keywords representing user actions.

SQABasic???

ScribbleOpen countriesAddToList Sweden USASaveAs countries2

Test file

For each TEST_ID OpenFile TEST_ID For each record in test file Read KEYWORD Call KEYWORD EndFor CloseFile TEST_IDEndFor

Control script

ScribbleOpen...

AddToList...

SaveAs...

Supporting scripts

J.Nawrocki, Acceptance Testing

Shared scriptsShared scripts

Shared script = A script used (shared) by more than one test case.

SQABasic

Call ScribbleOpen (‘countries’)

FocusOn ‘Scribble’

SelectOption ‘List/Add Item’

...

Call ScribbleSaveAs (‘countries2’)

J.Nawrocki, Acceptance Testing

Automating test executionAutomating test execution

Automated scripts are not like manual scripts

Don’t automate testing by simply recording

Automated execution with manual verification?

How much shoul be compared (whole, minimum, something between)?

Dynamic comparison and post-execution comparison

Automated comparison messages have to be manually checked

Why tests fail the second time?

J.Nawrocki, Acceptance Testing

Test activitiesTest activities

Identify test conditions (‘what’ to test) and prioritize

Design test cases (‘how’ to test)

Build test cases (scripts, data etc.)

Execute test cases

Compare test outcomes to expected outcomes

J.Nawrocki, Acceptance Testing

QuestionsQuestions??