Date post: | 10-May-2015 |
Category: |
Technology |
Upload: | craig-smith |
View: | 9,228 times |
Download: | 1 times |
© A
SE
RT
2006-2
009
Dr Paul King
@paulk_asert
ASERT, Australia
How to make your Testing
more GroovyCraig Smith
@smithcdau
Suncorp, Australia
Topics• Why Groovy for Testing?
• Groovy Intro
• Web Drivers
• Test Runners
• Non-web Drivers
• Other Tools
• Going beyond
AJUG_SEP2009 - 2
© A
SE
RT
2006-2
009
Focus is on using the Groovy dynamic language
for primarily functional and acceptance testing
with a forward looking perspective. Also considers
polyglot options. The techniques and lessons
learned can be applied to other kinds of testing
and are also applicable to similar languages.
Topics not covered in detail
• Coverage– But coverage options available
• Mocking– But built-in support and many libraries available
• CI Support– Support in Hudson, Team City, Anthill Pro to call
Groovy
• Code Metrics– CodeNarc and other tools available
• Build Tools– Ant, Maven, GMaven, Gant, Gradle, ...
AJUG_SEP2009 - 3
© A
SE
RT
2006-2
009
TopicsWhy Groovy for Testing?
• Groovy Intro
• Web Drivers– Native Groovy, HttpBuilder, HtmlUnit
WebTest, Watij, Selenium, WebDriver, ...
• Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave, ...
• Non-web Drivers– SOAP/REST, Database, FEST, ...
• Other Tools– SoapUI, ITest2, Sahi, JMeter, Twist, ...
• Going beyond– Polyglot, Model-driven, Constraint/logic languages, Concurrency
AJUG_SEP2009 - 4
© A
SE
RT
2006-2
009
Why Test With Groovy?
?AJUG_SEP2009 - 5
© A
SE
RT
2006-2
009
Why Test With Groovy?
• Wrong first question!
• Perfect second question?
XConsider first the task at hand and the
organization and people fit!AJUG_SEP2009 - 6
© A
SE
RT
2006-2
009
“People Fit” / “Organization Fit”
• People– Developers (familiarity with languages)
– Testers (tools language familiarity)
– Responsibility to create/run/maintain
– BAs/SMEs (ability to read technical tests)
– Consider both development and maintenance
– Expected feedback from tests (JUnit/Reports)
• Organization– Maturity level
– Degree of automation
– Tools in use
– Need for adaptability
AJUG_SEP2009 - 7
© A
SE
RT
2006-2
009
Why Test With Groovy?
• Advantages– Easy to learn
– Particularly good fit if the application you are testing
is built for the JVM or your developers work with Java
/ JVM languages
– Supports polyglot programming when needed
– Easy to plug and play different testing tools
– Good community & tools for professional agile
• Disadvantages– Requires JVM
– Less advantages if your developers using Python,
.Net, PHP
– Maybe your testers already know Ruby
AJUG_SEP2009 - 8
© A
SE
RT
2006-2
009
Scripting/Dynamic Languages
• Advantages– Lend themselves to succinct code/DSLs
– Powerful
– Increased Refactoring
– Increased Reuse
– Less prone to Brittleness
– Flexibility for tool integration
– Open APIs provide extensibility
• Disadvantages– Can be too low level (but many options now)
– Sometimes less tool support (but changing now)
AJUG_SEP2009 - 9
© A
SE
RT
2006-2
009
Test Characteristics• Coverage/Traceability
– Requirement coverage/traceability
– Code coverage: line, branch, path, state
– Transactional Tracing
• Purpose– Unit, Integration, System, Customer
• Manageability– Removing duplication
– Managing lifecycle: shared setUp/tearDown (@Before @After)
• Handling test data– Data-driven, databases, Spreadsheets, tables, keywords, random, model-driven
• Large Test Volumes– Speed of feedback, performance testing
• Tool evolution/longevity/cost/support/documentation– Open Source/Commercial, Critical mass/popularity
• Combinability
AJUG_SEP2009 - 10
© A
SE
RT
2006-2
009
Key Testing Practices • Use testing DSL’s
• Look to move up the testing stack– It used to be all about the driver
– Now the driver is hidden in the framework or tool stack
• Apply good testing practices– Pareto analysis, bug clusters, mutation testing, test early
– All pairs/equivalence partitions/orthogonal array testing
– Risk-based test selection, coding for testability, use CI
– Boundary value analysis, defensive programming
• Plug and play testing tools– Run different drivers with different runners and different tools
• Complement automated tests with exploration
• Expand testing scope– Test environment readiness, test deployments
AJUG_SEP2009 - 11
© A
SE
RT
2006-2
009
Groovy and Testing Tool Spectrum*
AJUG_SEP2009 - 12
© A
SE
RT
2006-2
009
Database
Drivers
DbUnit
DataSets
SqlUnit
groovy.sql
JPA
JDO
BigTable
JDBC
SOAP /
REST
Drivers
GroovyWS
XML-RPC
CXF
Axis2
JAX-WS
JAX-RS
Utilities
AllPairs, Combinations
Polyglot languages
Logic programming
Threads, Parallel /
Concurrency libraries
Data-driven libraries
Networking libraries
XML Processing
Read/write files /
Excel / Word / CSV
Reporting, Logging
Other
Drivers
FEST
FTP
AntUnit
Telnet
SSH
ExecTools
iTest2, SoapUI, Twist,
IDEs, JMeter, Text
editors, Recorders,
Build Tools, CI
Web
Drivers
WebTest
WebDriver
JWebUnit
Tellurium
Selenium
HtmlUnit
Watij
HttpBuilder
Cyberneko
Runners
Native Groovy, JUnit, TestNG, Spock, EasyB,
JBehave, Cucumber, Robot Framework
* Tools/libraries/frameworks don't always neatly fall into one category – still useful conceptually
Topics• Why Groovy for Testing?
Groovy Intro
• Web Drivers– Native Groovy, HttpBuilder, HtmlUnit
WebTest, Watij, Selenium, WebDriver, ...
• Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave, ...
• Non-web Drivers– SOAP/REST, Database, FEST, ...
• Other Tools– SoapUI, ITest2, Sahi, JMeter, Twist, ...
• Going beyond– Polyglot, Model-driven, Constraint/logic languages, Concurrency
AJUG_SEP2009 - 13
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 14
© A
SE
RT
2006-2
009
What is Groovy?
• “Groovy is like a super version
of Java. It can leverage Java's
enterprise capabilities but also
has cool productivity features like closures,
DSL support, builders and dynamic typing.”
Groovy = Java – boiler plate code+ optional dynamic typing+ closures+ domain specific languages+ builders+ metaprogramming
AJUG_SEP2009 - 15
© A
SE
RT
2006-2
009
Groovy Goodies Overview• Fully object oriented
• Closures: reusable
and assignable
pieces of code
• Operators can be
overloaded
• Multimethods
• Literal declaration for
lists (arrays), maps,
ranges and regular
expressions
• GPath: efficient
object navigation
• GroovyBeans
• grep and switch
• Templates, builder,
swing, Ant, markup,
XML, SQL, XML-RPC,
Scriptom, Grails,
tests, Mocks
Growing Acceptance …
A slow and steady start but now gaining in
momentum, maturity and mindshare
Making
Java
Groovy
(soon)
Now free
… Growing Acceptance …
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 17
… Growing Acceptance …
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 18
Groovy and
Grails downloads:
70-90K per month
and growing
… Growing Acceptance …
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 19
Source: http://www.grailspodcast.com/
Source: http://www.micropoll.com/akira/mpresult/501697-116746
… Growing Acceptance …
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 20http://www.java.net
http://www.jroller.com/scolebourne/entry/devoxx_2008_whiteboard_votes
… Growing Acceptance …
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 21http://www.leonardoborges.com/writings
What alternative JVM language are you using or intending to use
… Growing Acceptance …
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 22
http://it-republik.de/jaxenter/quickvote/results/1/poll/44 (translated using http://babelfish.yahoo.com)
… Growing Acceptance
AJUG_SEP2009 - 23
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 24
© A
SE
RT
2006-2
009
The Landscape of JVM Languages
Java bytecode calls
for static types
Dynamic features call
for dynamic types
optional
static
types
The terms “Java Virtual Machine” and “JVM” mean a Virtual Machine for the Java™ platform.
AJUG_SEP2009 - 25
© A
SE
RT
2006-2
009
Groovy StarterSystem.out.println("Hello, World!"); // optional semicolon,println 'Hello, World!' // System.out, brackets,
// main() method, class defn
def name = 'Guillaume' // dynamic typingprintln "$name, I'll get the car." // GString
String longer = """${name}, the caris in the next row.""" // multi-line string
// with static typing
assert 0.5 == 1/2 // BigDecimal equals()
def printSize(obj) { // optional duck typingprint obj?.size() // safe dereferencing
}
def pets = ['ant', 'bee', 'cat'] // native list syntaxpets.each { pet -> // closure support
assert pet < 'dog' // overloading '<' on String} // or: for (pet in pets)...
A Better Java...
AJUG_SEP2009 - 26
© A
SE
RT
2006-2
009
import java.util.List;import java.util.ArrayList;
class Erase {private List removeLongerThan(List strings, int length) {
List result = new ArrayList();for (int i = 0; i < strings.size(); i++) {
String s = (String) strings.get(i);if (s.length() <= length) {
result.add(s);}
}return result;
}public static void main(String[] args) {
List names = new ArrayList();names.add("Ted"); names.add("Fred");names.add("Jed"); names.add("Ned");System.out.println(names);Erase e = new Erase();List shortNames = e.removeLongerThan(names, 3);System.out.println(shortNames.size());for (int i = 0; i < shortNames.size(); i++) {
String s = (String) shortNames.get(i);System.out.println(s);
}}
}
This code
is valid
Java and
valid Groovy
Based on an
example by
Jim Weirich
& Ted Leung
...A Better Java...
AJUG_SEP2009 - 27
© A
SE
RT
2006-2
009
import java.util.List;import java.util.ArrayList;
class Erase {private List removeLongerThan(List strings, int length) {
List result = new ArrayList();for (int i = 0; i < strings.size(); i++) {
String s = (String) strings.get(i);if (s.length() <= length) {
result.add(s);}
}return result;
}public static void main(String[] args) {
List names = new ArrayList();names.add("Ted"); names.add("Fred");names.add("Jed"); names.add("Ned");System.out.println(names);Erase e = new Erase();List shortNames = e.removeLongerThan(names, 3);System.out.println(shortNames.size());for (int i = 0; i < shortNames.size(); i++) {
String s = (String) shortNames.get(i);System.out.println(s);
}}
}
Do the
semicolons
add anything?
And shouldn‟t
we us more
modern list
notation?
Why not
import common
libraries?
...A Better Java...
AJUG_SEP2009 - 28
© A
SE
RT
2006-2
009
class Erase {private List removeLongerThan(List strings, int length) {
List result = new ArrayList()for (String s in strings) {
if (s.length() <= length) {result.add(s)
}}return result
}
public static void main(String[] args) {List names = new ArrayList()names.add("Ted"); names.add("Fred")names.add("Jed"); names.add("Ned")System.out.println(names)Erase e = new Erase()List shortNames = e.removeLongerThan(names, 3)System.out.println(shortNames.size())for (String s in shortNames) {
System.out.println(s)}
}}
...A Better Java...
AJUG_SEP2009 - 29
© A
SE
RT
2006-2
009
class Erase {private List removeLongerThan(List strings, int length) {
List result = new ArrayList()for (String s in strings) {
if (s.length() <= length) {result.add(s)
}}return result
}
public static void main(String[] args) {List names = new ArrayList()names.add("Ted"); names.add("Fred")names.add("Jed"); names.add("Ned")System.out.println(names)Erase e = new Erase()List shortNames = e.removeLongerThan(names, 3)System.out.println(shortNames.size())for (String s in shortNames) {
System.out.println(s)}
}}
Do we need
the static types?
Must we always
have a main
method and
class definition?
How about
improved
consistency?
...A Better Java...
AJUG_SEP2009 - 30
© A
SE
RT
2006-2
009
def removeLongerThan(strings, length) {def result = new ArrayList()for (s in strings) {
if (s.size() <= length) {result.add(s)
}}return result
}
names = new ArrayList()names.add("Ted")names.add("Fred")names.add("Jed")names.add("Ned")System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())for (s in shortNames) {
System.out.println(s)}
...A Better Java...
AJUG_SEP2009 - 31
© A
SE
RT
2006-2
009
def removeLongerThan(strings, length) {def result = new ArrayList()for (s in strings) {
if (s.size() <= length) {result.add(s)
}}return result
}
names = new ArrayList()names.add("Ted")names.add("Fred")names.add("Jed")names.add("Ned")System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())for (s in shortNames) {
System.out.println(s)}
Shouldn‟t we
have special
notation for lists?
And special
facilities for
list processing?
Is „return‟
needed at end?
...A Better Java...
AJUG_SEP2009 - 32
© A
SE
RT
2006-2
009
def removeLongerThan(strings, length) {strings.findAll{ it.size() <= length }
}
names = ["Ted", "Fred", "Jed", "Ned"]System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())shortNames.each{ System.out.println(s) }
...A Better Java...
AJUG_SEP2009 - 33
© A
SE
RT
2006-2
009
def removeLongerThan(strings, length) {strings.findAll{ it.size() <= length }
}
names = ["Ted", "Fred", "Jed", "Ned"]System.out.println(names)shortNames = removeLongerThan(names, 3)System.out.println(shortNames.size())shortNames.each{ System.out.println(s) }
Is the method
now needed?
Easier ways to
use common
methods?
Are brackets
required here?
...A Better Java...
AJUG_SEP2009 - 34
© A
SE
RT
2006-2
009
names = ["Ted", "Fred", "Jed", "Ned"]println namesshortNames = names.findAll{ it.size() <= 3 }println shortNames.size()shortNames.each{ println it }
...A Better Java
AJUG_SEP2009 - 35
© A
SE
RT
2006-2
009
names = ["Ted", "Fred", "Jed", "Ned"]println namesshortNames = names.findAll{ it.size() <= 3 }println shortNames.size()shortNames.each{ println it }
["Ted", "Fred", "Jed", "Ned"]3TedJedNed
Better Lists, Maps, Ranges
• Lists– Special syntax for list literals
– Additional common methods (operator overloading)
• Maps– Special syntax for map literals
– Additional common methods
• Ranges– Special syntax for various kinds of ranges
AJUG_SEP2009 - 36
© A
SE
RT
2006-2
009
def list = [3, new Date(), 'Jan']assert list + list == list * 2
def map = [a: 1, b: 2]assert map['a'] == 1 && map.b == 2
def letters = 'a'..'z'def numbers = 0..<10
Closures
• Traditional mainstream languages– Data can be stored in variables, passed around,
combined in structured ways to form more complex
data; code stays put where it is defined
• Languages supporting closures– Data and code can be stored in variables, passed
around, combined in structured ways to form more
complex algorithms and data
AJUG_SEP2009 - 37
© A
SE
RT
2006-2
009
doubleNum = { num -> num * 2 }println doubleNum(3) // => 6
processThenPrint = { num, closure ->num = closure(num); println "num is $num"
}processThenPrint(3, doubleNum) // => num is 6processThenPrint(10) { it / 2 } // => num is 5
SwingXBuilder
AJUG_SEP2009 - 38
© A
SE
RT
2006-2
009
import groovy.swing.SwingXBuilderimport static java.awt.Color.*import static java.lang.Math.*
def swing = new SwingXBuilder()def frame = swing.frame(size: [300, 300]) {
graph(plots: [[GREEN, {value -> sin(value)}],[BLUE, {value -> cos(value)}],[RED, {value -> tan(value)}]
])}.show()
Topics• Why Groovy for Testing?
• Groovy Intro
Web Drivers– Native Groovy, HttpBuilder, HtmlUnit
WebTest, Watij, Selenium, WebDriver
Tellurium, JWebUnit
• Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave, ...
• Non-web Drivers– SOAP/REST, Database, FEST, ...
• Other Tools– SoapUI, ITest2, Sahi, JMeter, Twist, ...
• Going beyond– Polyglot, Model-driven, Constraint/logic languages, Concurrency
AJUG_SEP2009 - 39
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 40
© A
SE
RT
2006-2
009
Concept
Driver
Runner<webtest name="myTest">
<steps>
<invoke
description="get Login Page"
url="login" />
<verifyTitle
description="we should see the login title"
text="Login Page" />
</steps>
</webtest>
Web Server
HTTP Request / Response
HTTP Request / Response
Read
Script
Manual
Automated
Driver Category• Real browser invoker
– Runs on platform
supported by real
browser
– May need multiple
platforms, e.g. IE6/IE7
– Uses actual JavaScript
engine
– Can be easier to use
with test recorders
– Automation
capabilities differ
across browsers
– Can typically get to all
aspects of browser
• Browser Emulators
– Can simulate multiple
browsers
– Less platform
restrictions
– Good for CI
– Easier to not download
images, resources
– Ability to optimise
JavaScript interactions
– More extensible
– Ability to disable
JavaScript
– Scope for parallelism
AJUG_SEP2009 - 41
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 42
© A
SE
RT
2006-2
009
Application under Test…
AJUG_SEP2009 - 43
© A
SE
RT
2006-2
009
…Application under Test...
AJUG_SEP2009 - 44
© A
SE
RT
2006-2
009
…Application under Test
Native Groovy...
• Access URLs
• Built-in XML parsing
• Built-in friendly regular expression syntax
• Even for advanced cases, there is friendly
access to low-level things:– Sockets, Processes
– Databases and other things
– Files
• Huge range of Java libraries– PDF
– Reading, writing Excel
AJUG_SEP2009 - 45
© A
SE
RT
2006-2
009
...Native Groovy...
• Useful URL methods
• Simple enough for GAE– For public sites
– Can share test scripts easily
– No setup required
AJUG_SEP2009 - 46
© A
SE
RT
2006-2
009
def html = new URL('http://localhost:8080').text
assert html.contains('<title>Welcome to SimpBlog</title>')
html.find(~'<title>(.*)</title>') { all, title ->assert title == 'Welcome to SimpBlog'
}
...Native Groovy...
• Built-in XML Parsing
AJUG_SEP2009 - 47
© A
SE
RT
2006-2
009
def page = newXmlSlurper().parse('http://localhost:8080/viewPost?id=1')
assert page.body.h1.text().contains('Tis the season')assert page.body.h3[1].text() == 'Category: Home'assert page.body.h3[2].text() == 'Author: Bart'assert page.body.table.tr.td.p.text() ==
"Aren't we forgeting the true meaning of Christmas? You know, the birth of Santa."
...Native Groovy
• Easy access to Java libraries
AJUG_SEP2009 - 48
© A
SE
RT
2006-2
009
@Grab('nekohtml:nekohtml:1.9.6.2')import org.cyberneko.html.parsers.SAXParser
def parser = new XmlSlurper(new SAXParser())def page = parser.parse('http://localhost:8080/viewPost?id=1')assert page.BODY.H1.text().contains('Tis the season')assert page.BODY.H3[1].text() == 'Category: Home'assert page.BODY.H3[2].text() == 'Author: Bart'assert page.BODY.TABLE.TR.TD.P.text() ==
"Aren't we forgeting the true meaning of Christmas? You know, the birth of Santa."
• Builder for Http interactions– Flexible: bogus posts, response codes, JSON, non-
HTML
HttpBuilder
© A
SE
RT
2006-2
009
@Grab(group='org.codehaus.groovy.modules.http-builder',module='http-builder', version='0.5.0-RC1')
import groovyx.net.http.*import static groovyx.net.http.ContentType.URLENC
def http = new HTTPBuilder('http://localhost:8080')def postBody = [title:'Bart was here (and so was HttpBuilder)',
content:'Cowabunga Dude!', author:'1', category:'3']http.post(path:'/addPost', body: postBody,
requestContentType: URLENC) { resp, html ->assert resp.contentType == 'text/html'assert resp.status == 200assert html.BODY.H1.text().matches('Post.*: Bart was here.*')assert html.BODY.H3[1].text() == 'Category: Home'assert html.BODY.H3[2].text() == 'Author: Bart'assert html.BODY.TABLE.TR.TD.P.text() == 'Cowabunga Dude!'
}AJUG_SEP2009 - 49
HtmlUnit• 100% Java-based headless browser emulator
– Can test any Web site: Java, .Net, PHP, Rails, ...
• Open Source– Apache 2 license
– Hosted at SourceForge
– 7 committers (3 very active)
– Very mature
• Useful for:– Integration and acceptance testing
– Screen scraping, deployment automation, ...
• Used by other drivers:– Canoo WebTest , JWebUnit , WebDriver , JSFUnit , Celerity
• Special features:– Easy ajax mode, emulation of multiple browsers
AJUG_SEP2009 - 50
© A
SE
RT
2006-2
009
HtmlUnit Features...• Support for the HTTP and HTTPS protocols
• Support for cookies
• Ability to specify whether failing responses from
the server should throw exceptions or should be
returned as "error" pages
• Support for submit methods POST and GET– As well as HEAD, DELETE, ...
• Ability to customize the request headers being
sent to the server
• Support for HTML responses– Wrapper for HTML pages that provides easy access to all
information contained inside them
– Support for submitting forms and clicking links
– Support for walking the DOM model of HTML documentsAJUG_SEP2009 - 51
© A
SE
RT
2006-2
009
...HtmlUnit Features• Proxy server support
• Support for basic and NTLM authentication
• Excellent JavaScript support– jQuery 1.2.6: Full support
– MochiKit 1.4.1: Full support
– GWT 1.6.4: Full support
– Sarissa 0.9.9.3: Full support
– MooTools 1.2.1: Full support
– Prototype 1.6.0: Very good support
– Ext JS 2.2: Very good support
– Dojo 1.0.2: Good support
– YUI 2.3.0: Good support
AJUG_SEP2009 - 52
© A
SE
RT
2006-2
009
HtmlUnit: Testing New Blog Post...
AJUG_SEP2009 - 53
© A
SE
RT
2006-2
009
@Grab('net.sourceforge.htmlunit:htmlunit:2.6')import com.gargoylesoftware.htmlunit.WebClient
def client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')// check page titleassert 'Welcome to SimpBlog' == page.titleText
// fill in blog entry and post itdef form = page.getFormByName('post')form.getInputByName('title').
setValueAttribute('Bart was here (and so was HtmlUnit)')form.getSelectByName('category').getOptions().find{
it.text == 'Home' }.setSelected(true)form.getTextAreaByName('content').setText('Cowabunga Dude!')def result = form.getInputByName('btnPost').click()
...
...HtmlUnit: Testing New Blog Post
AJUG_SEP2009 - 54
© A
SE
RT
2006-2
009
...
// check blog post detailsassert result.getElementsByTagName('h1').item(0).
textContent.matches('Post.*: Bart was here.*')def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home'assert h3headings.item(2).textContent == 'Author: Bart'
// expecting:// <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.getFirstChild()assert para.textContent == 'Cowabunga Dude!'
AJUG_SEP2009 - 55
© A
SE
RT
2006-2
009
Canoo WebTest• Description– Open source tool for automated testing of web applications
– Declarative approach in XML or testing DSL in Groovy
– Has Test Recorder
– Excellent reporting options
– Ant-based under the covers
<target name="login" >
<testSpec name="normal" >
&config;
<steps>
<invoke stepid="get Login Page"
url="login.jsp" />
<verifytitle stepid="we should see the login title"
text="Login Page" />
<setinputfield stepid="set user name"
name="username"
value="scott" />
<setinputfield stepid="set password"
name="password"
value="tiger" />
<clickbutton stepid="Click the submit button"
label="let me in" />
<verifytitle stepid="Home Page follows if login ok"
text="Home Page" />
</steps>
</testSpec>
</target>
Canoo WebTest Features• Strongly encourages declarative testing
– Supports testing DSLs, test structuring and reuse through
macrodefs and imports for XML flavor &
methods and closures for Groovy flavor
• Extensive support for HTML pages– Including JavaScript
• Also supports other MIME types– Generically as binary streams
– Special support for PDF, Excel, Emails
• Ant heritage provides easy IDE/CI hooks
• Excellent Documentation
• Excellent Community
• Eats own dog food– high quality codebase
AJUG_SEP2009 - 56
© A
SE
RT
2006-2
009
Canoo WebTest Steps...
AJUG_SEP2009 - 57
© A
SE
RT
2006-2
009
...Canoo WebTest Steps...
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 58
...Canoo WebTest Steps...
AJUG_SEP2009 - 59
© A
SE
RT
2006-2
009
...Canoo WebTest Steps
AJUG_SEP2009 - 60
© A
SE
RT
2006-2
009
WebTest: Testing New Blog Post...
AJUG_SEP2009 - 61
© A
SE
RT
2006-2
009
<webtest name="Testing Posting a new Blog Entry">
<invoke url="http://localhost:8080/" description="Home Page"/>
<verifyTitle text="Welcome to SimpBlog"/>
<group description="Post New Blog Entry">
<clickLink label="New Blog Entry"/>
<setInputField name="title"
value="Bart was here (and so was WebTest)"/>
<setSelectField name="category" text="School"/>
<setInputField name="content" value="Cowabunga Dude!"/>
<clickButton name="btnPost"/>
</group>
...
...WebTest: Testing New Blog Post...
AJUG_SEP2009 - 62
© A
SE
RT
2006-2
009
...
<group description="Check Blog Post">
<verifyElementText type="h1" regex="true"
text="Post.*: Bart was here.*"/>
<verifyXPath xpath="//h3[2]/text()" text="Category: School"/>
<verifyXPath xpath="//h3[3]/text()" text="Author: Bart"/>
<verifyElementText type="p" text="Cowabunga Dude!"/>
</group>
<groovy>
println "Test run at: ${new Date()}"
</groovy>
</webtest>
...WebTest: Testing New Blog Post...
AJUG_SEP2009 - 63
© A
SE
RT
2006-2
009
ant.webtest(name: 'Test SimpBlog') {invoke url: "http://localhost:8080/",
description: "Home Page"verifyTitle text: "Welcome to SimpBlog"group description: "Post New Blog Entry", {clickLink label: "New Blog Entry"setInputField name: "title",
value: "Bart was here (and so was WebTest with Groovy)"setSelectField name: "category", text: "School"setInputField name: "content", value: "Cowabunga Dude!"clickButton name: "btnPost"
}group description: "Check Blog Post", {verifyElementText type: "h1", regex: "true",
text: "Post.*: Bart was here.*"verifyXPath xpath: "//h3[2]/text()", text: "Category: School"verifyXPath xpath: "//h3[3]/text()", text: "Author: Bart"verifyElementText type: "p", text: "Cowabunga Dude!"
}}
...WebTest: Testing New Blog Post...
AJUG_SEP2009 - 64
© A
SE
RT
2006-2
009
...WebTest: Testing New Blog Post
AJUG_SEP2009 - 65
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 66
© A
SE
RT
2006-2
009
Firefox Recorder
Watij
• Description– Java API that provides control and automation of
Internet Explorer
– Supports actions like navigating, clicking links, filling
out forms, etc.
– Also supports more complex actions like file
downloading and uploading, popup windows and
dialogs, and screen captures
• Special Features– Ability to work with IE interactively
– Can attach to an existing browser session
– Special browser commands, e.g. ie.fullScreen(true)
– Handles child browsers and popup dialogs
AJUG_SEP2009 - 67
© A
SE
RT
2006-2
009
Watij Features
• Finders– tag(String tagName)
– attribute(String name, String value)
– index(int index)
– text(String text)
– name(String value)
– value(String value)
– caption(String value)
– id(String value)
– title(String value)
– alt(String value)
– src(String value)
– action(String value)
– method(String value)
– url(String value)
– href(String value)
– xpath(String expression)AJUG_SEP2009 - 68
© A
SE
RT
2006-2
009
Watij: Testing New Blog Post...
AJUG_SEP2009 - 69
© A
SE
RT
2006-2
009
import watij.runtime.ie.IEimport static watij.finders.SymbolFactory.*import static watij.finders.FinderFactory.*
def ie = new IE()ie.start('http://localhost:8080/postForm')
// check page titleassert ie.title() == 'Welcome to SimpBlog'
// fill in query form and submit itie.textField(name, 'title').
set('Bart was here (and so was Watij)')ie.textField(name, 'content').set('Cowabunga dude!')ie.selectList(name, "category").
option(text, "Home").select()ie.button(name, 'btnPost').click()
...
...Watij: Testing New Blog Post...
AJUG_SEP2009 - 70
© A
SE
RT
2006-2
009
...
// check entered post is being displayedassert ie.htmlElement(tag, 'H1').text().
matches('Post.*: Bart was here.*')def h3headers = ie.htmlElements(tag, 'H3')assert h3headers.get(1).text() == 'Category: Home'assert h3headers.get(2).text() == 'Author: Bart'
// try a more advanced finder// content is at: //TABLE/TBODY/TR/TD/Pdef row = ie.htmlElement(xpath('//TABLE/TBODY/TR'))assert row.cell(0).htmlElement(tag, 'P').text() ==
'Cowabunga dude!'
ie.close()
...Watij: Testing New Blog Post
AJUG_SEP2009 - 71
© A
SE
RT
2006-2
009
Selenium...
• Description– Tools to help
automate testing
for web-based
applications
– Support for
running tests on
multiple browser
platforms
• Components– Selenium Core
– Selenium IDE
Selenium RC
– Selenium Grid
AJUG_SEP2009 - 72
© A
SE
RT
2006-2
009
Source: http://seleniumhq.org/projects/remote-control/
Our focus
...Selenium
AJUG_SEP2009 - 73
© A
SE
RT
2006-2
009
Source: http://seleniumhq.org/docs/01_introducing_selenium.html
Selenium: Testing New Blog Post...
AJUG_SEP2009 - 74
© A
SE
RT
2006-2
009
import com.thoughtworks.selenium.DefaultSeleniumimport org.openqa.selenium.server.SeleniumServer
// start auxiliary serverdef server = new SeleniumServer()server.start()
// uncomment one of below//def browser = "*iexplore"//def browser = "*firefox3" // if Firefox already in your path//def browser = "*firefox3 C:/Program Files/Mozilla Firefox/firefox.exe"def browser = "*firefox3 C:/Program Files (x86)/Mozilla Firefox/firefox.exe"
def selenium = new DefaultSelenium("localhost", 4444, browser,"http://localhost:8080")
selenium.start()
...
...Selenium: Testing New Blog Post
AJUG_SEP2009 - 75
© A
SE
RT
2006-2
009
...
// post blogselenium.open "/postForm"selenium.type "title", "Bart was here (and so was Selenium)"selenium.select "category", "Home"selenium.type "content", "Cowabunga Dude!"selenium.click "btnPost"selenium.waitForPageToLoad "5000"
// checksassert selenium.isTextPresent('regex:Post.*: Bart was here')assert selenium.isElementPresent('//h3[text()="Author: Bart"]')assert selenium.isElementPresent('//h3[text()="Category: Home"]')assert selenium.isElementPresent(
'//table//tr/td/p[text()="Cowabunga Dude!"]')
selenium.stop()server.stop()
Selenium IDE...
AJUG_SEP2009 - 76
© A
SE
RT
2006-2
009
Features:
• Easy record and playback
• Intelligent field selection will use
IDs, names, or XPath as needed
• Autocomplete for all common
Selenium commands
• Walk through tests
• Debug and set breakpoints
• Save tests as HTML, Ruby
scripts, or any other format
• Support for Selenium user-
extensions.js file
• Option to automatically assert the
title of every page
...Selenium IDE
AJUG_SEP2009 - 77
© A
SE
RT
2006-2
009
more info: http://limcheekin.blogspot.com/2009/07/behavior-driven-development-generating.html
Selenium Other Tools...
AJUG_SEP2009 - 78
© A
SE
RT
2006-2
009
...Selenium Other Tools...
AJUG_SEP2009 - 79
© A
SE
RT
2006-2
009
...Selenium Other Tools
AJUG_SEP2009 - 80
© A
SE
RT
2006-2
009
Source: http://selenium-grid.seleniumhq.org/how_it_works.html
WebDriver
• Description– Simple API to drive both real browsers
• for testing javascript heavy apps
– and a pure 'in memory' emulator solution
• for faster testing of simpler apps
• uses HtmlUnit under the covers in emulator mode
– Represents next generation of Selenium RC
• though merging into Selenium may happen under the covers
if you are a Selenium user
– Roadmap has plans to leverage some advanced
Selenium like features
• RemoteWebDriver
• FarmedWebDriver (think Selenium Grid)
AJUG_SEP2009 - 81
© A
SE
RT
2006-2
009
WebDriver: Testing New Blog Post...
AJUG_SEP2009 - 82
© A
SE
RT
2006-2
009
import org.openqa.selenium.Byimport org.openqa.selenium.htmlunit.HtmlUnitDriver
def driver = new HtmlUnitDriver()driver.get('http://localhost:8080/postForm')assert driver.title == 'Welcome to SimpBlog'
// fill in query form and submit itdriver.findElement(By.name('title')).
sendKeys('Bart was here (and so was WebDriver)')driver.findElement(By.name('content')).
sendKeys('Cowabunga dude!')def select = driver.findElement(By.name('category'))select.findElements(By.tagName("option")).find{
it.text == 'Home' }.setSelected()driver.findElement(By.name('btnPost')).click()
...
...WebDriver: Testing New Blog Post
AJUG_SEP2009 - 83
© A
SE
RT
2006-2
009
...
assert driver.findElement(By.tagName("h1")).text.matches('Post.*: Bart was here.*')
def h3headers = driver.findElements(By.tagName("h3"))assert h3headers[1].text == 'Category: Home'assert h3headers[2].text == 'Author: Bart'
// try a more advanced finder// content is at: //TABLE/TBODY/TR/TD/Pdef row = driver.findElement(By.xpath("//table/tbody/tr"))def col = row.findElement(By.tagName("td"))def para = col.findElement(By.tagName("p"))assert para.text == 'Cowabunga dude!'
Tellurium
• Description– built on top of Selenium but tries to solve several
shortcomings
– "record and reply" style, difficult to refactor and
maintain, so instead define UI components
declaratively then write tests in terms of UI
– Provides many predefined UI objects for you to use
directly, such as Button, CheckBox, InputBox,
Selector, TextBox, and Table but also ability to write
your own custom UI objects
– Supports advanced locating mechanisms: composite
locator, "group locating"
– Supports testing DSL
– Supports data-driven tests
AJUG_SEP2009 - 84
© A
SE
RT
2006-2
009
Architecture
AJUG_SEP2009 - 85
© A
SE
RT
2006-2
009
Source: http://code.google.com/p/aost/wiki/Introduction
Tellurium Example
• Selenium:
• Tellurium UI:
• Tellurium DSL Test:
AJUG_SEP2009 - 86
© A
SE
RT
2006-2
009
selenium.type("//input[@title='Google Search']", input)selenium.click("//input[@name='btnG' and @type='submit']")
ui.Container(uid: "google_start_page",clocator: [tag: "td"], group: "true") {
InputBox(uid: "searchbox",clocator: [title: "Google Search"])
SubmitButton(uid: "googlesearch",clocator: [name: "btnG", value: "Google Search"])
}
type "google_start_page.searchbox", inputclick "google_start_page.googlesearch"
Tellurium: Testing New Blog Post...
AJUG_SEP2009 - 87
© A
SE
RT
2006-2
009
ui.UrlLink(uid: "create_new_post",clocator: [tag:'a', text: "New Blog Entry"])
ui.Form(uid: "blogform",clocator: [tag: 'form', name:'post'], group: "true") {
InputBox(uid: "title", clocator: [name: "title"])InputBox(uid: "content", clocator: [tag:'textarea', name: "content"])Selector(uid: "category", clocator: [name: "category"])Selector(uid: "author", clocator: [name: "author"])SubmitButton(uid: "post_button",
clocator: [name: 'btnPost', value: "Create Post"])}
ui.TextBox(uid: 'main_header', clocator: [tag: 'h1'])ui.TextBox(uid: 'category_header', clocator: [tag: 'h3', position: '2'])ui.TextBox(uid: 'author_header', clocator: [tag: 'h3', position: '3'])ui.Container(uid: 'table', clocator: [tag: 'table']) {
ui.TextBox(uid: 'content_para', locator: '//tr/td/p')}
...Tellurium: Testing New Blog Post
AJUG_SEP2009 - 88
© A
SE
RT
2006-2
009
openUrl "http://localhost:8080/"click "create_new_post"waitForPageToLoad 5000assert title == 'Welcome to SimpBlog'
// post blogtype "blogform.title", "Bart was here (and so was Tellurium)"selectByLabel "blogform.category", "Home"selectByLabel "blogform.author", "Bart"type "blogform.content", "Cowabunga Dude!"click "blogform.post_button"waitForPageToLoad 5000
// check contentsassert getText('main_header').matches('Post.*: Bart was here.*')assert getText('category_header') == 'Category: Home'assert getText('author_header') == 'Author: Bart'assert getText('table.content_para') == 'Cowabunga Dude!'shutDown
TrUMP IDE
AJUG_SEP2009 - 89
© A
SE
RT
2006-2
009
JWebUnit...
• Description– Java-based testing framework for web applications
– Intention is to provide a high-level "driver" Java API
– Wraps existing testing frameworks such as HtmlUnit
and Selenium with a unified, simple testing interface
– Support includes navigation via links, form entry and
submission, validation of table contents, and other
verification steps
– Includes some runner capabilities
– Useful in that it allows you to switch between
different lower level drivers without re-writing your
tests
AJUG_SEP2009 - 90
© A
SE
RT
2006-2
009
...JWebUnit
AJUG_SEP2009 - 91
© A
SE
RT
2006-2
009
Source: http://jwebunit.sourceforge.net/
JWebUnit: Testing New Blog Post
AJUG_SEP2009 - 92
© A
SE
RT
2006-2
009
import net.sourceforge.jwebunit.junit.*
class TestSimpBlog extends WebTestCase {void setUp() { setBaseUrl("http://localhost:8080") }
void testPostBlog() {beginAt "/postForm"assertTitleEquals "Welcome to SimpBlog"setTextField "title", "Bart was here (and so was JWebUnit)"setTextField "content", "Cowabunga Dude!"selectOption "category", "Home"clickButtonWithText "Create Post"assert getElementByXPath('//H1').textContent.matches('Post.*: Bart was here.*')
def h3headings = getElementsByXPath('//H3')assert h3headings[1].textContent == "Category: Home"assert h3headings[2].textContent == "Author: Bart"def cell = getElementByXPath('//TABLE//TR/TD')assert cell.children[0].textContent == 'Cowabunga Dude!'
}}
AJUG_SEP2009 - 93
Topics• Why Groovy for Testing?
• Groovy Intro
• Web Drivers– Native Groovy, HttpBuilder, HtmlUnit
WebTest, Watij, Selenium, WebDriver, ...
Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave,
Cucumber, Robot Framework, FitNesse/Slim
• Non-web Drivers– SOAP/REST, Database, FEST, ...
• Other Tools– SoapUI, ITest2, Sahi, JMeter, Twist, ...
• Going beyond– Polyglot, Model-driven, Constraint/logic languages, Concurrency
© A
SE
RT
2006-2
009
Native Groovy
• Groovy has a friendly ‘==‘
• Built-in assert
• Scripts are low ceremony
• By utilising @Grab are easy to share
• Many in-built testing capabilities are
accessible even from scripts
• Easy to version control or treat like
operating system scripts
• Out of the box detection of JUnit and
TestNG tests
AJUG_SEP2009 - 94
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 95
© A
SE
RT
2006-2
009
Built-in JUnit 3...
• Groovy distributions up to 1.6.X include JUnit 3
• Automatically invokes text runner
• Example uses HtmlUnit driverclass TestSimpBlogJUnit extends TestCase {def page
void setUp() {page = new WebClient().getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleText
}
void testBartWasHere() {// fill in blog entry and post it
def form = page.getFormByName('post')form.getInputByName('title').setValueAttribute('Bart was here (and so was form.getSelectByName('category').getOptions().find { it.text == 'Home'form.getTextAreaByName('content').setText('Cowabunga Dude!')def result = form.getInputByName('btnPost').click()
...
...Built-in JUnit 3...
AJUG_SEP2009 - 96
© A
SE
RT
2006-2
009
...def form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(
'Bart was here (and so was HtmlUnit)')form.getSelectByName('category').getOptions().find {
it.text == 'Home' }.setSelected(true)form.getTextAreaByName('content').setText('Cowabunga Dude!')def result = form.getInputByName('btnPost').click()
// check blog post detailsassert result.getElementsByTagName('h1').item(0).
textContent.matches('Post.*: Bart was here.*')def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home'assert h3headings.item(2).textContent == 'Author: Bart'
// expecting: <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.getFirstChild()assert para.textContent == 'Cowabunga Dude!'
}}
...Built-in JUnit 3
AJUG_SEP2009 - 97
© A
SE
RT
2006-2
009
GroovyTestCase Tests
• Like JUnit but with some enhancements– Additional assert methods
– fewer imports
– clean shouldFail syntax
AJUG_SEP2009 - 98
© A
SE
RT
2006-2
009
class TestSimpBlogGUnit extends GroovyTestCase {def page
void setUp() {// ...
}
void testBartWasHere() {// ...
...
AJUG_SEP2009 - 99
© A
SE
RT
2006-2
009
JUnit 4.X
import org.junit.*
class TestSimpBlogJUnit4 {def page
@Beforevoid setUp() {
// ...}
@Testvoid bartWasHere() {
// ...
• Groovy distributions from 1.7 include JUnit 4
• Automatically invokes text runner if needed
• Example uses HtmlUnit driver (not shown)
AJUG_SEP2009 - 100
© A
SE
RT
2006-2
009
JUnit 4.X Parameterized Tests
@RunWith(Parameterized)class TestSimpBlogJUnit4DD {
def page, author, title, category, content
TestSimpBlogJUnit4DD(author, title,category, content) {
this.author = authorthis.title = titlethis.category = categorythis.content = content
}
@Parameters static data() {return [
['Bart', 'Title 1', 'Home', 'Content 1'],['Homer', 'Title 2', 'Work', 'Content 2'],['Marge', 'Title 3', 'Food', 'Content 3']
].collect{ it as String[] }}
...
TestNG
AJUG_SEP2009 - 101
© A
SE
RT
2006-2
009
• Groovy automatically invokes text runner if run
as a script
• Example shows grouping, driver not shown
import org.testng.annotations.*
class TestSimpBlogTestNG {def page
@BeforeClassvoid setUp() {
// ...}
@Test(groups = [ "slow" ])void bartWasHere() {
// ...
TestNG Data Driven
AJUG_SEP2009 - 102
© A
SE
RT
2006-2
009
import org.testng.annotations.*
class TestSimpBlogTestNGDD {// ...
@DataProvider(name='SimpBlogDataProvider')Object[][] data() {
return [['Bart', 'Title 1', 'Home', 'Content 1'],['Homer', 'Title 2', 'Work', 'Content 2'],['Marge', 'Title 3', 'Food', 'Content 3']
].collect{ it as Object[] } as Object[]}
@Test(dataProvider = "SimpBlogDataProvider")void bartWasHere(author, title, category, content) {
// ...
Spock Testing Framework...
AJUG_SEP2009 - 103
© A
SE
RT
2006-2
009
...Spock Testing Framework
AJUG_SEP2009 - 104
© A
SE
RT
2006-2
009
• Testing framework for Java and Groovy
• Highly expressive specification language– No assertion API
– No record &
replay
mocking API
– No
superfluous
annotations
– Meaningful
assert error
messages
– Extensible
– Compatible
with JUnit
reportingwise
@Speck@RunWith(Sputnik)class PublisherSubscriberSpeck {def "events are received by all subscribers"() {def pub = new Publisher()def sub1 = Mock(Subscriber)def sub2 = Mock(Subscriber)pub.subscribers << sub1 << sub2
when:pub.send("event")
then:1 * sub1.receive("event")1 * sub2.receive("event")
}}
Spock Example...
AJUG_SEP2009 - 105
© A
SE
RT
2006-2
009
import com.gargoylesoftware.htmlunit.WebClientimport spock.lang.*import org.junit.runner.RunWith
@Speck ()@RunWith (Sputnik)class TestSimpBlogSpock {def page, subheadings, para, form, result
@Unroll("When #author posts a #category blog with content '#content' it should succeed"def "when creating a new blog entry"() {given:page = new WebClient().getPage('http://localhost:8080/postForm')form = page.getFormByName('post')
when:form.getInputByName('title').setValueAttribute("$author was here (and so was Spock)"form.getSelectByName('category').getOptions().find { it.text == category }.form.getSelectByName('author').getOptions().find { it.text == author }.setSelectedform.getTextAreaByName('content').setText(content)result = form.getInputByName('btnPost').click()subheadings = result.getElementsByTagName('h3')para = result.getByXPath('//TABLE//TR/TD/P')[0]
...
...Spock Example
AJUG_SEP2009 - 106
© A
SE
RT
2006-2
009
...then:page.titleText == 'Welcome to SimpBlog'result.getElementsByTagName('h1').item(0).textContent.matches("Post.*: $author was here.*"subheadings.item(1).textContent == "Category: $category"subheadings.item(2).textContent == "Author: $author"
and:para.textContent == content
where:author << ['Bart', 'Homer', 'Lisa']category << ['Home', 'Work', 'Food']content << ['foo', 'bar', 'baz']
}}
// Optional use of 'and:'
AJUG_SEP2009 - 107
© A
SE
RT
2006-2
009
EasyB
• Description: BDD, Rspec-like testing librarynarrative 'segment flown', {
as_a 'frequent flyer'i_want 'to accrue rewards points for every segment I fly'so_that 'I can receive free flights for my dedication to the airline'
}
scenario 'segment flown', {given 'a frequent flyer with a rewards balance of 1500 points'when 'that flyer completes a segment worth 500 points'then 'that flyer has a new rewards balance of 2000 points'
}
scenario 'segment flown', {given 'a frequent flyer with a rewards balance of 1500 points', {
flyer = new FrequentFlyer(1500)}when 'that flyer completes a segment worth 500 points', {
flyer.fly(new Segment(500))}then 'that flyer has a new rewards balance of 2000 points', {
flyer.pointsBalance.shouldBe 2000}
}
EasyB Example ...
• When run will be marked as pending– perfect for ATDD
AJUG_SEP2009 - 108
© A
SE
RT
2006-2
009
scenario "Bart posts a new blog entry", {given "we are on the create blog entry page"when "I have entered 'Bart was here' as the title"and "I have entered 'Cowabunga Dude!' into the content"and "I have selected 'Home' as the category"and "I have selected 'Bart' as the author"and "I click the 'Create Post' button"then "I expect the entry to be posted"
}
...EasyB Example...
AJUG_SEP2009 - 109
© A
SE
RT
2006-2
009
description "Post Blog Entry Feature"
narrative "for feature", {as_a "Blogger"i_want "to be able to post a blog"so_that "I can keep others informed"
}
before "posting blog", {given "we are on the create blog entry page", {
webClient = new com.gargoylesoftware.htmlunit.WebClient()page = webClient.getPage('http://localhost:8080/postForm')
}}
scenario "Bart was here blog", {
when "I have entered 'Bart was here' as the title", {form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(
'Bart was here (and so was EasyB)')}
...
...EasyB Example...
AJUG_SEP2009 - 110
© A
SE
RT
2006-2
009
...and "I have entered 'Cowabunga Dude!' into the content", {
form.getTextAreaByName('content').setText('Cowabunga Dude!')}
and "I have selected 'Home' as the category", {form.getSelectByName('category').getOptions().find { it.text == 'Home' }.setSelected(true
}
and "I click the 'Create Post' button", {result = form.getInputByName('btnPost').click()
}
then "I expect the entry to be posted", {// check blog post detailsassert result.getElementsByTagName('h1').item(0).textContent.matches('Post.*: Bart was here.*'def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == 'Category: Home' // traditional styleh3headings.item(2).textContent.shouldBe 'Author: Bart' // BDD style
// expecting: <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>def cell = result.getByXPath('//TABLE//TR/TD')[0]def para = cell.firstChildassert para.textContent == 'Cowabunga Dude!'// para.shouldHave textContent: 'Cowabunga Dude!'
}}
...EasyB Example...
AJUG_SEP2009 - 111
© A
SE
RT
2006-2
009
...EasyB Example
AJUG_SEP2009 - 112
© A
SE
RT
2006-2
009
2 scenarios (including 1 pending) executed successfully.
Story: simp blog initial
scenario Bart posts a new blog entry [PENDING]
given we are on the create blog entry page
when I have entered 'Bart was here' as the title
when I have entered 'Cowabunga Dude!' into the content [PENDING]
when I have selected 'Home' as the category [PENDING]
when I have selected 'Bart' as the author [PENDING]
when I click the 'Create Post' button [PENDING]
then I expect the entry to be posted [PENDING]
Story: simp blog
Post Blog Entry Feature
for feature
As a Blogger
I want to be able to post a blog
So that I can keep others informed
given we are on the create blog entry page
scenario Bart was here blog
when I have entered 'Bart was here' as the title
when I have entered 'Cowabunga Dude!' into the content
when I have selected 'Home' as the category
when I click the 'Create Post' button
then I expect the entry to be posted
easyb is preparing to process 2 file(s)
Running simp blog initial story (SimpBlogInitialStory.groovy)
Scenarios run: 1, Failures: 0, Pending: 1, Time elapsed: 1.049 sec
Running simp blog story (SimpBlogStory.groovy)
Scenarios run: 1, Failures: 0, Pending: 0, Time elapsed: 1.356 sec
2 total behaviors ran (including 1 pending behavior) with no failures
easyb execution passed
Cucumber• Description
– Loose coupling
between text spec
and step defns
AJUG_SEP2009 - 113
© A
SE
RT
2006-2
009
# language: enFeature: Addition
In order to avoid silly mistakesAs a math idiot I want to be told the sum of two numbers
Scenario Outline: Add two numbersGiven I have entered <input_1> into the calculatorAnd I have entered <input_2> into the calculatorWhen I press <button>Then the stored result should be <output>
Examples:| input_1 | input_2 | button | output || 20 | 30 | add | 50 || 2 | 5 | add | 7 || 0 | 40 | add | 40 |
# language: enFeature: DivisionIn order to avoid silly mistakesCashiers must be able to calculate a fraction
Scenario: Regular numbersGiven I have entered 3 into the calculatorAnd I have entered 2 into the calculatorWhen I press divideThen the stored result should be 1.5
Cucumber Example...
AJUG_SEP2009 - 114
© A
SE
RT
2006-2
009
# language: en
@newpost
Feature: New Blog Post
In order to create a new blog entry
Bloggers should be able to select their name and category and enter text
Scenario: New Posting
Given we are on the create blog entry page
When I have entered "Bart was here" as the title
And I have entered "Cowabunga Dude!" as the content
And I have selected "Home" from the "category" dropdown
And I have selected "Bart" from the "author" dropdown
And I click the 'Create Post' button
Then I should see a heading message matching "Post.*: Bart was here.*"
...Cucumber Example...
AJUG_SEP2009 - 115
© A
SE
RT
2006-2
009
...Cucumber Example
AJUG_SEP2009 - 116
© A
SE
RT
2006-2
009
import com.gargoylesoftware.htmlunit.WebClientthis.metaClass.mixin(cuke4duke.GroovyDsl)
Given ~/we are on the create blog entry page/, { ->page = new WebClient().getPage('http://localhost:8080/postForm')
}
When(~/I have entered "(.*)" as the title/) {String title ->form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(title + ' (and so was Cucumber)')
}
When(~'I have entered "(.*)" as the content') {String content ->form.getTextAreaByName('content').setText(content)
}
When(~'I have selected "(.*)" from the "(.*)" dropdown') {String option, String name ->form.getSelectByName(name).getOptions().find {
it.text == option }.setSelected(true)}
When(~"I click the 'Create Post' button") { ->result = form.getInputByName('btnPost').click()
}
Then(~'I should see a heading message matching "(.*)"') {String pattern ->// ensureThat result.getElementsByTagName('h1').item(0).textContent.matches(pattern)
assert result.getElementsByTagName('h1').item(0).textContent.matches(pattern)}
Cucumber Data Driven Example...
AJUG_SEP2009 - 117
© A
SE
RT
2006-2
009
# language: en@newpostFeature: New Blog PostIn order to create a new blog entryBloggers should be able to select their name and category and enter text
Scenario Outline: New PostingGiven we are on the create blog entry pageWhen I have entered "<title>" as the titleAnd I have entered "<content>" as the contentAnd I have selected "<category>" from the "category" dropdownAnd I have selected "<author>" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: <title>.*"
Examples:| title | content | category | author || Title 1 | Content 1 | Home | Bart || Title 2 | Content 2 | Work | Homer || Title 3 | Content 3 | Food | Marge |
...Cucumber Data Driven Example
AJUG_SEP2009 - 118
© A
SE
RT
2006-2
009
JBehave
• Description– Behaviour-driven development in Java
• Also works out of the box for Groovy
– Behavior scenarios written in text
• Use the words Given, When, Then and And.
– Mapped using regular expressions and annotations
to step methods
– Web Runner available for non-technical users to
easily run tests
– Hooks to Selenium available in JBehave Web
• Other Java libraries (e.g. HtmlUnit) easy to use too
– Supports parameter converters
• Getting 'String' parameters into appropriate Object values
– Supports a 'StepDoc' function
• For listing available scenario clausesAJUG_SEP2009 - 119
© A
SE
RT
2006-2
009
JBehave Example...
AJUG_SEP2009 - 120
© A
SE
RT
2006-2
009
Given we are on the create blog entry pageWhen I have entered "Bart was here" as the titleAnd I have entered "Cowabunga Dude!" as the contentAnd I have selected "Home" from the "category" dropdownAnd I have selected "Bart" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: Bart was here.*"
import org.jbehave.scenario.Scenarioimport org.jbehave.scenario.steps.Steps
class NewPostScenario extends Scenario {NewPostScenario() {
super([new CreateBlogSteps()] as Steps[])}
}Scenario:
Given we are on the create blog entry page
When I have entered "Bart was here" as the title
And I have entered "Cowabunga Dude!" as the content
And I have selected "Home" from the "category" dropdown
And I have selected "Bart" from the "author" dropdown
And I click the 'Create Post' button
Then I should see a heading message matching "Post.*: Bart was here.*"
...JBehave Example...
AJUG_SEP2009 - 121
© A
SE
RT
2006-2
009
import org.jbehave.scenario.steps.Stepsimport org.jbehave.scenario.annotations.*import com.gargoylesoftware.htmlunit.WebClient
class CreateBlogSteps extends Steps {def page, form, result
@Given("we are on the create blog entry page")void gotoEntryPage() {
page = new WebClient().getPage('http://localhost:8080/postForm')}
@When('I have entered "$title" as the title')void enterTitle(String title) {
form = page.getFormByName('post')form.getInputByName('title').
setValueAttribute(title + ' (and so was JBehave)')}
@When('I have entered "$content" as the content')void enterContent(String content) {
form.getTextAreaByName('content').setText(content)}
...
Example uses HtmlUnit which must be added to CLASSPATH
...JBehave Example
AJUG_SEP2009 - 122
© A
SE
RT
2006-2
009
...@When('I have selected "$option" from the "$name" dropdown')void selectOption(String option, String name) {
form.getSelectByName(name).getOptions().find {it.text == option }.setSelected(true)
}
@When("I click the 'Create Post' button")void clickPostButton() {
result = form.getInputByName('btnPost').click()}
@Then('I should see a heading message matching "$message"')void checkPost(String pattern) {
assert result.getElementsByTagName('h1').item(0).textContent.matches(pattern)
}}
...void checkPost(String pattern) {
ensureThat result.getElementsByTagName('h1').item(0).textContent.matches(pattern)
}...
Choose either traditional
style or BDD style
JBehave Web Runner...
AJUG_SEP2009 - 123
© A
SE
RT
2006-2
009
...JBehave Web Runner
AJUG_SEP2009 - 124
© A
SE
RT
2006-2
009
Hacked* "JBehave Aware" GroovyConsole
AJUG_SEP2009 - 125
© A
SE
RT
2006-2
009
* Not currently publically available
Robot Framework...
• Description– Keyword-driven test automation framework
for acceptance level testing and acceptance
test-driven development (ATDD)
– Easy to use tabular syntax for creating test
– Easy to use test libraries implemented either with
Python or Java
– Open source, Apache License 2.0
– Supports creating data-driven test cases.
– Provides tagging to categorize and select test cases
to be executed
– Provides easy-to-read reports and logs in HTML
format
– XML-RPC interface for remote testing
AJUG_SEP2009 - 126
© A
SE
RT
2006-2
009
...Robot Framework...
• Tabular tests / Executable Specs– In HTML or Text
AJUG_SEP2009 - 127
© A
SE
RT
2006-2
009
***Settings***
Library OperatingSystem
***Variables***
${MESSAGE} Hello, world!
***Test Cases***
My Test [Documentation] Example test
Log ${MESSAGE}
My Keyword /tmp
Another Test
Should Be Equal ${MESSAGE} Hello, world!
***Keywords***
My Keyword [Arguments] ${path}
Directory Should Exist ${path}
...Robot Framework...
• Reporting
AJUG_SEP2009 - 128
© A
SE
RT
2006-2
009
...Robot Framework...
• Reporting
AJUG_SEP2009 - 129
© A
SE
RT
2006-2
009
...Robot Framework...
• Tools
AJUG_SEP2009 - 130
© A
SE
RT
2006-2
009
Robot Framework Example...
AJUG_SEP2009 - 131
© A
SE
RT
2006-2
009
***Test Cases***
Submit postGiven we are on the create blog entry pageWhen I have entered "Bart was here" as the titleAnd I have entered "Cowabunga Dude!" as the contentAnd I have selected "Home" from the "category" dropdownAnd I have selected "Bart" from the "author" dropdownAnd I click the 'Create Post' buttonThen I should see a heading message matching "Post.*: Bart was here.*"
***Keywords***
Given we are on the create blog entry pageOpen Browser http://localhost:8080/postForm ${BROWSER}# Set Selenium Speed ${DELAY}Title Should Be Welcome to SimpBlog
When I have entered "${text}" as the titleInput Text title ${text}
...
First look at the Python version using Selenium
...Robot Framework Example
AJUG_SEP2009 - 132
© A
SE
RT
2006-2
009
...
And I have entered "${text}" as the contentInput Text content ${text}
And I have selected "${option}" from the "${list}" dropdownSelect From List ${list} ${option}
And I click the 'Create Post' buttonClick Button btnPost
Then I should see a heading message matching "${expected}"${text} = Get Text xpath=//h1Should Match Regexp ${text} ${expected}
***Settings***Library ${LIBRARY}Test Teardown Close Browser
***Variables***${BROWSER} *firefox3 C:/Program Files (x86)/Mozilla Firefox/firefox.exe${DELAY} 0${LIBRARY} SeleniumLibrary
Robot Framework with Groovy...
AJUG_SEP2009 - 133
© A
SE
RT
2006-2
009
***Settings***Library Remote localhost:8270 WITH NAME Groovy
***Test Case***Multiple Blog PostsPost Bart Home Title 1 Content 1Post Homer Work Title 2 Content 2Post Marge Food Title 3 Content 3
Pybot
RunnerTestCase.txt
Groovy
XML RPC
Server
Groovy/Java
Driver
XML-RPC
...Robot Framework with Groovy...
AJUG_SEP2009 - 134
© A
SE
RT
2006-2
009
import groovy.net.xmlrpc.*import java.net.ServerSocketimport com.gargoylesoftware.htmlunit.WebClient
def server = new XMLRPCServer()server.get_keyword_names = { ["Post"] }server.get_keyword_documentation = { "" }server.get_keyword_arguments = { ["*args"] }server.run_keyword = { name, args ->
assert name == 'Post'def (author, category, title, content) = argsdef client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleTextdef form = page.getFormByName('post')form.getInputByName('title').setValueAttribute("$title (entered with Robot Framework)")form.getSelectByName('author').getOptions().find{ it.text == author }.setSelected(true)form.getSelectByName('category').getOptions().find{ it.text == category }.setSelected(true)form.getTextAreaByName('content').setText(content)def result = form.getInputByName('btnPost').click()assert result.getElementsByTagName('h1').item(0).textContent.matches("Post.*: $title.*")def h3headings = result.getElementsByTagName('h3')assert h3headings.item(1).textContent == "Category: $category"assert h3headings.item(2).textContent == "Author: $author"def para = result.getByXPath('//TABLE//TR/TD/P')[0]assert para.textContent == contentreturn [status:'PASS', output:"$name $args", error:'bad arg']
}
def serverSocket = new ServerSocket(8270)server.startServer(serverSocket)
In the future, a generic version of this file may be possible
...Robot Framework with Groovy...
AJUG_SEP2009 - 135
© A
SE
RT
2006-2
009
...Robot Framework with Groovy
AJUG_SEP2009 - 136
© A
SE
RT
2006-2
009
FitNesse/SLIM
• Description– Tool for enhancing collaboration
in software development
– Allows customers,
developers and
testers to easily
create and run
tests to compare
expected with
actual results
– Tests are captured
in wiki format and
mapped into code
using fixtures
AJUG_SEP2009 - 137
© A
SE
RT
2006-2
009
Source: http://fitnesse.org/
SLIM with Groovy...
AJUG_SEP2009 - 138
© A
SE
RT
2006-2
009
...SLIM with Groovy
AJUG_SEP2009 - 139
© A
SE
RT
2006-2
009
package simpblog
import com.gargoylesoftware.htmlunit.WebClient
class NewBlogPost {String author, title, content, categoryprivate resultdef execute() {
def client = new WebClient()def page = client.getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleTextdef form = page.getFormByName('post')form.getInputByName('title').
setValueAttribute("$title (entered with Robot Framework)")form.getSelectByName('author').getOptions().find{
it.text == author }.setSelected(true)form.getSelectByName('category').getOptions().find{
it.text == category }.setSelected(true)form.getTextAreaByName('content').setText(content)result = form.getInputByName('btnPost').click()
}def mainHeading() {
def m = result.getElementsByTagName('h1').item(0).textContent =~/Post .*: (.*) \([^)]*\)/
m[0][1]}
}
Example uses HtmlUnit to call SimpBlog web site but those details aren't important here
Topics• Why Groovy for Testing?
• Groovy Intro
• Web Drivers– Native Groovy, HttpBuilder, HtmlUnit
WebTest, Watij, Selenium, WebDriver, ...
• Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave, ...
Non-web Drivers– SOAP/REST, Database, FEST, ...
• Other Tools– SoapUI, ITest2, Sahi, JMeter, Twist, ...
• Going beyond– Polyglot, Model-driven, Constraint/logic languages, Concurrency
AJUG_SEP2009 - 140
© A
SE
RT
2006-2
009
Better SQL Manipulation...
AJUG_SEP2009 - 141
© A
SE
RT
2006-2
009
import java.sql.Connection;import java.sql.DriverManager;import java.sql.Statement;import java.sql.ResultSet;import java.sql.SQLException;
public class JdbcJava {public static Connection getConnection() throws Exception {String driver = "org.hsqldb.jdbcDriver";String url = "jdbc:hsqldb:file:EMPDB";String username = "sa"; String password = "";Class.forName(driver);return DriverManager.getConnection(url, username, password);
}
public static void main(String[] args) {Connection conn = null;Statement stmt = null;ResultSet rs = null;try {conn = getConnection();stmt = conn.createStatement();String query ="select id, lastname, firstname from Employees";
rs = stmt.executeQuery(query);while (rs.next()) {System.out.println(rs.getString("id") +": " + rs.getString("firstname") +" " + rs.getString("lastname"));
}}catch (Exception e) {// handle the exceptione.printStackTrace();System.err.println(e.getMessage());
}// ...
// ...finally {// release database resourcesclose(rs); // close the ResultSetclose(stmt); // close the Statementclose(conn); // close the Connection
}}
private static void close(ResultSet rs) {try {if (rs != null) rs.close();
} catch (SQLException e) {e.printStackTrace();
}}
private static void close(Statement st) {try {if (st != null) st.close();
} catch (SQLException e) {e.printStackTrace();
}}
private static void close(Connection cn) {try {if (cn != null) cn.close();
} catch (SQLException e) {e.printStackTrace();
}}
}
...Better SQL Manipulation...
AJUG_SEP2009 - 142
© A
SE
RT
2006-2
009
import java.sql.Connection;import java.sql.DriverManager;import java.sql.Statement;import java.sql.ResultSet;import java.sql.SQLException;
public class JdbcJava {public static Connection getConnection() throws Exception {String driver = "org.hsqldb.jdbcDriver";String url = "jdbc:hsqldb:file:EMPDB";String username = "sa"; String password = "";Class.forName(driver);return DriverManager.getConnection(url, username, password);
}
public static void main(String[] args) {Connection conn = null;Statement stmt = null;ResultSet rs = null;try {conn = getConnection();stmt = conn.createStatement();String query ="select id, lastname, firstname from Employees";
rs = stmt.executeQuery(query);while (rs.next()) {System.out.println(rs.getString("id") +": " + rs.getString("firstname") +" " + rs.getString("lastname"));
}}catch (Exception e) {// handle the exceptione.printStackTrace();System.err.println(e.getMessage());
}// ...
// ...finally {// release database resourcesclose(rs); // close the ResultSetclose(stmt); // close the Statementclose(conn); // close the Connection
}}
private static void close(ResultSet rs) {try {if (rs != null) rs.close();
} catch (SQLException e) {e.printStackTrace();
}}
private static void close(Statement st) {try {if (st != null) st.close();
} catch (SQLException e) {e.printStackTrace();
}}
private static void close(Connection cn) {try {if (cn != null) cn.close();
} catch (SQLException e) {e.printStackTrace();
}}
}
boilerplate
...Better SQL Manipulation
AJUG_SEP2009 - 143
© A
SE
RT
2006-2
009
import groovy.sql.Sql
def url = 'jdbc:hsqldb:file:EMPDB'def username = 'sa'def password = ''def driver = 'org.hsqldb.jdbcDriver'
def db = Sql.newInstance(url, username, password, driver)
db.eachRow("SELECT id, firstname, lastname FROM Employees") {println "$it.id: $it.firstname $it.lastname"
}
AJUG_SEP2009 - 144
© A
SE
RT
2006-2
009
More Details: Working with Databases
• Using standard SQL statements
• Using DataSets
import groovy.sql.Sql
def foo = 'cheese'def db = Sql.newInstance("jdbc:mysql://localhost:3306/mydb",
"user", "pswd", "com.mysql.jdbc.Driver")
db.eachRow("select * from FOOD where type=${foo}") {println "Gromit likes ${it.name}"
}
import groovy.sql.Sql
def db = Sql.newInstance("jdbc:mysql://localhost:3306/mydb","user", "pswd", "com.mysql.jdbc.Driver")
def food = db.dataSet('FOOD')def cheese = food.findAll { it.type == 'cheese' }cheese.each { println "Gromit likes ${it.name}" }
AJUG_SEP2009 - 145
© A
SE
RT
2006-2
009
WebTest testing Web Sitesdef ant = new AntBuilder()
def webtest_home = System.properties.'webtest.home'
ant.taskdef(resource:'webtest.taskdef') {
classpath {
pathelement(location:"$webtest_home/lib")
fileset(dir:"$webtest_home/lib", includes:"**/*.jar")
}
}
def config_map = [:]
['protocol','host','port','basepath','resultfile',
'resultpath', 'summary', 'saveresponse','defaultpropertytype'].each {
config_map[it] = System.properties['webtest.'+it]
}
ant.testSpec(name:'groovy: Test Groovy Scripting at creation time') {
config(config_map)
steps {
invoke(url:'linkpage.html')
for (i in 1..10) {
verifyText(description:"verify number ${i} is on pages", text:"${i}")
}
}
}
WebTest testing Emails
AJUG_SEP2009 - 146
© A
SE
RT
2006-2
009
def ant = new AntBuilder()
def webtest_home = System.properties.'webtest.home'
ant.taskdef(resource:'webtest.taskdef'){
classpath(){
pathelement(location:"$webtest_home/lib")
fileset(dir:"$webtest_home/lib", includes:"**/*.jar")
}
}
ant.testSpec(name:'Email Test'){
steps {
emailSetConfig(server:'localhost', password:'password',
username:'[email protected]', type:'pop3')
emailStoreMessageId(subject:'/Build notification/',
property:'msg')
emailStoreHeader(property:'subject',
messageId:'#{msg}', headerName:'Subject')
groovy('''def subject = step.webtestProperties.subject
assert subject.startsWith('Build notification')''')
emailMessageContentFilter(messageId:'#{msg}')
verifyText(text:'Failed build')
}
}
SOAP Client and Server
AJUG_SEP2009 - 147
© A
SE
RT
2006-2
009
class MathService {double add(double a, double b) {
a + b}double square(double c) {
c * c}
}
import groovy.net.soap.SoapServer
def server = new SoapServer('localhost', 6789)server.setNode('MathService')server.start()
import groovy.net.soap.SoapClient
def math = new SoapClient('http://localhost:6789/MathServiceInterface?wsdl')
assert math.add(1.0, 2.0) == 3.0
assert math.square(3.0) == 9.0
FEST• Description
– Framework for testing Swing GUIs (among other things)
– Simulation of user interaction with a GUI (e.g. mouse / keyboard input)
– Reliable GUI component lookup• by type, by name or custom search criteria
– Support for all Swing components included in the JDK
– Compact and powerful API for creation and maintenance of functional
GUI tests
– Regular expression matching
– Supports Applet testing
– Ability to embed screenshots of failed GUI tests in HTML test reports
– Can be used with either TestNG or JUnit
– Supports testing violations of Swing's threading rules
– Experimental Groovy Builder support (coming soon!)
AJUG_SEP2009 - 148
© A
SE
RT
2006-2
009
dialog.comboBox("domain").select("Users")dialog.textBox("username").enterText("leia.organa")dialog.button("login").click()dialog.optionPane().requireErrorMessage()
.requireMessage("Please enter your .*")
Topics• Why Groovy for Testing?
• Groovy Intro
• Web Drivers– Native Groovy, HttpBuilder, HtmlUnit
WebTest, Watij, Selenium, WebDriver, ...
• Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave, ...
• Non-web Drivers– SOAP/REST, Database, FEST, ...
Other Tools– SoapUI, ITest2, Sahi, JMeter, Twist, ...
• Going beyond– Polyglot, Model-driven, Constraint/logic languages, Concurrency
AJUG_SEP2009 - 149
© A
SE
RT
2006-2
009
AJUG_SEP2009 - 150
© A
SE
RT
2006-2
009
SoapUI...
...SoapUI
• Tool for testing Web Services has a built-
in Groovy editor for custom steps
AJUG_SEP2009 - 151
© A
SE
RT
2006-2
009
iTest2
• Tool for recording, refactoring and
running watir style tests
AJUG_SEP2009 - 152
© A
SE
RT
2006-2
009
We have had success cutting and pasting recorded test steps – using metaprogramming to map to Groovy testing DSL
Sahi...• Description
– Tool for recording and running web tests
• Features– Supports in-browser controls (cross-platform/browser)
– Ability to add assertions (check points) for validation
– Text based programmable scripts with ability to
• parametrize variables
• re-factor into functions
• organize and include other scripts
– Can run scripts in batch mode for automated testing
– Command line and ant integration
– Automatic html reporting with error logs
– Easily extensible via simple javascript
– Support for data-driven testing
– Multi-threaded playback
– HTTP, HTTPS and AJAX support
AJUG_SEP2009 - 153
© A
SE
RT
2006-2
009
...Sahi
AJUG_SEP2009 - 154
© A
SE
RT
2006-2
009
We have had success cutting and
pasting recorded test steps – using
metaprogramming to map to Groovy
testing DSL
Further Info:
http://sahi.co.in/w/
AJUG_SEP2009 - 155
© A
SE
RT
2006-2
009
JMeter
• What is Apache JMeter?– Java desktop application designed to load test
functional behavior and measure performance
– Originally designed for testing Web Applications but
has since expanded to other test functions
Performance Testing
JMeter can call out to Groovy – reusing your functional tests
JMeter Case Study...
AJUG_SEP2009 - 156
© A
SE
RT
2006-2
009
Part 1: Native JMeter Tests
* No JavaScript
* No reuse of functional tests
...JMeter Case Study...
AJUG_SEP2009 - 157
© A
SE
RT
2006-2
009
Further details about Statistical Aggregate plugin: http://rubenlaguna.com/wp/better-jmeter-graphs/
...JMeter Case Study...
AJUG_SEP2009 - 158
© A
SE
RT
2006-2
009
Part 2: JMeter Groovy JUnitTests
* Reuse functional tests
import junit.framework.TestCase
class TestSimpBlogJUnit extends TestCase {static lisaPostAndCheck = nullstatic bartPostAndCheck = nullstatic String script = '''@Grab('net.sourceforge.htmlunit:htmlunit:2.5')import com.gargoylesoftware.htmlunit.WebClientdef page = new WebClient().getPage('http://localhost:8080/postForm')def form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(title)form.getSelectByName('category').getOptions().find { it.text == category }.setSelected(true)form.getSelectByName('author').getOptions().find { it.text == author }.setSelected(true)form.getTextAreaByName('content').setText(content)def result = form.getInputByName('btnPost').click()def titleResult = result.getElementsByTagName('h1').item(0).textContentdef h3headings = result.getElementsByTagName('h3')def categoryResult = h3headings.item(1).textContentdef authorResult = h3headings.item(2).textContentdef para = result.getByXPath('//TABLE//TR/TD/P')[0]def contentResult = para.textContentreturn new ResultHolder(titleResult, contentResult, authorResult, categoryResult)'''void setUp() {
lisaPostAndCheck = setUpBlogger(lisaPostAndCheck, 'Lisa', "I'm Hungry", "Food")bartPostAndCheck = setUpBlogger(bartPostAndCheck, 'Bart', "Don't have a cow dude", "Home")
}...
...JMeter Case Study...
AJUG_SEP2009 - 159
© A
SE
RT
2006-2
009
...private setUpBlogger(orig, blogger, content, category) {
if (orig) return origdef result = new GroovyShell().parse(script)def binding = new Binding()binding.setVariable('title', blogger + ' was here (and so was JMeter)')binding.setVariable('content', content)binding.setVariable('author', blogger)binding.setVariable('category', category)result.binding = bindingresult
}
void testBartWasHere() {def result = bartPostAndCheck.run()assert result.title.contains('Bart was here')
}
void testLisaWasHere() {def result = lisaPostAndCheck.run()assert result.title.contains('Lisa was here')
}}
class ResultHolder {String title, content, author, categoryResultHolder(String title, String content, String author, String category) {
this.title = titlethis.content = contentthis.author = authorthis.category = category
}}
...JMeter Case Study...
AJUG_SEP2009 - 160
© A
SE
RT
2006-2
009
...JMeter Case Study
AJUG_SEP2009 - 161
© A
SE
RT
2006-2
009
Twist
AJUG_SEP2009 - 162
© A
SE
RT
2006-2
009
Future version will support Groovy
Topics• Why Groovy for Testing?
• Groovy Intro
• Web Drivers– Native Groovy, HttpBuilder, HtmlUnit
WebTest, Watij, Selenium, WebDriver, ...
• Test Runners– Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave, ...
• Non-web Drivers– SOAP/REST, Database, FEST, ...
• Other Tools– SoapUI, ITest2, Sahi, JMeter, Twist, ...
Going beyond– Polyglot, Model-driven, Constraint/logic languages, Concurrency
AJUG_SEP2009 - 163
© A
SE
RT
2006-2
009
All Combinations
• Description– Don't have a bunch
of hard-coded, hard
to maintain manual
test data or even
manually generated
CSV file
– Much better to
generate test cases
from succinct
expressions of
what you are trying
to achieve
AJUG_SEP2009 - 164
© A
SE
RT
2006-2
009
[['MacOS', 'Linux', 'Vista'],['2G', '4G', '6G', '8G'],['250G', '350G', '500G']
].combinations().each{os, mem, disk ->test(os, mem, disk)
}
test('MacOS', '4G', '250G')test('Linux', '4G', '250G')test('Vista', '4G', '250G')test('MacOS', '8G', '500G')test('Linux', '8G', '500G')test('Vista', '8G', '500G')// 30 more rows
All Combinations Case Study
AJUG_SEP2009 - 165
© A
SE
RT
2006-2
009
import com.gargoylesoftware.htmlunit.WebClient
def combos = [["Bart", "Homer", "Marge", "Lisa", "Maggie"],["Work", "School", "Home", "Travel", "Food"],["foo", "bar", "baz"]].combinations()
println "Found ${combos.size()} combos"combos.each { author, category, content ->
postAndCheck category, author, content}
def postAndCheck(String category, String author, String content) {// ...// details not shown (ran with HtmlUnit)// ...
}
Found 75 combos
All Pairs
• Description– Sometimes
called
pairwise
testing or
orthogonal
array testing
– Technique
to limit the
explosion of test cases by identifying samples of
important classes of test cases (equivalence classes)
• providing maximum coverage with minimum testing
– Instead of all combinations, systematically use pair-
wise combinations of interactions between objects
• as most faults result from adverse two-way interactions
AJUG_SEP2009 - 166
© A
SE
RT
2006-2
009
All Pairs Case Study...
AJUG_SEP2009 - 167
© A
SE
RT
2006-2
009
class AllPairs {private initialResults, results, rest
private buildPairs(Map partialCombinations, inputsLeft) {def first = getFirstEntry(inputsLeft)def partialResults = []first.value.each {
def next = [(first.key): it]getFirstEntry(next)next.putAll(partialCombinations)partialResults << next
}if (inputsLeft.size() == 1) {
initialResults.addAll(partialResults)} else {
partialResults.each {def rest = inputsLeft.clone()rest.remove(first.key)buildPairs(it, rest)
}}
}
private adjustPairs() {results = initialResults.clone()initialResults.each {
def restResults = results.clone()restResults.remove(it)if (allPairsCovered(it, restResults)) {
results.remove(it)}
}}
...
...private getFirstEntry(Map map) {
return map.entrySet().toList().get(0)}
private getAllPairsFromMap(map) {if (!map || map.size() <= 1) return nulldef allPairs = new HashSet()def first = getFirstEntry(map)def restMap = map.clone()restMap.remove(first.key)restMap.each {
def nextPair = new HashSet()nextPair << firstnextPair << itallPairs << nextPair
}def restPairs = getAllPairsFromMap(rest)if (restPairs != null) {
allPairs.addAll(restPairs)}return allPairs
}...
Details here not important:
Given here for completeness as this
library is not publically available yet
...All Pairs Case Study...
AJUG_SEP2009 - 168
© A
SE
RT
2006-2
009
...private boolean allPairsCovered(candidate, remaining) {
def totalCount = 0def pairCombos = getAllPairsFromMap(candidate)pairCombos.each {candidatePair ->
def pairFound = falsedef pairs = candidatePair.toList()for (it in remaining) {
def entries = it.entrySet()if (!pairFound && entries.contains(pairs[0]) && entries.contains(pairs[1])) {
pairFound = truetotalCount++
}}
}return (totalCount == pairCombos.size())
}
private updateUsedPairs(map) {getAllPairsFromMap(map).each { usedPairs << it }
}
def generate(configurations) {initialResults = new HashSet()results = new HashSet()buildPairs([:], configurations)adjustPairs()results
}}
Details here not important:
Given here for completeness as this
library is not publically available yet
...All Pairs Case Study...
AJUG_SEP2009 - 169
© A
SE
RT
2006-2
009
import com.gargoylesoftware.htmlunit.WebClient
def pairs = new AllPairs().generate(author: ["Bart", "Homer", "Marge", "Lisa", "Maggie"],category: ["Work", "School", "Home", "Travel", "Food"],content: ["foo", "bar", "baz"])
println "Found ${pairs.size()} pairs"pairs.each {
println it // just for debugging purposespostAndCheck it.category, it.author, it.content
}
def postAndCheck(String category, String author, String content) {// ...// details not shown (ran with HtmlUnit)// ...
}
...All Pairs Case Study
AJUG_SEP2009 - 170
© A
SE
RT
2006-2
009
Found 18 pairs[content:bar, category:Food, author:Bart][content:bar, category:School, author:Homer][content:foo, category:Work, author:Bart][content:baz, category:School, author:Homer][content:bar, category:Home, author:Maggie][content:foo, category:School, author:Marge][content:bar, category:Work, author:Bart][content:baz, category:Travel, author:Bart][content:foo, category:Home, author:Homer][content:bar, category:Travel, author:Marge][content:baz, category:Work, author:Homer][content:bar, category:Travel, author:Lisa][content:baz, category:Travel, author:Maggie][content:baz, category:Home, author:Marge][content:baz, category:Food, author:Homer][content:baz, category:Travel, author:Lisa][content:foo, category:Food, author:Maggie][content:foo, category:Travel, author:Lisa]
gpars/GParallelizer...
• Description– Library classes and DSL sugar providing intuitive
ways for Groovy developers to handle tasks
concurrently. Four logical parts:
• Actors provide a Groovy implementation of Scala-like
actors, both thread-bound actors and thread pool-
bound (event-driven) ones
• Dataflow Concurrency allows for very natural shared-
memory concurrency model, based on single-
assignment variables
• Asynchronizer extends the Java 1.5 built-in support
for executor services to enable multi-threaded
collection and closure processing
• Parallelizer uses JSR-166y Parallel Arrays to enable
multi-threaded collection processingAJUG_SEP2009 - 171
© A
SE
RT
2006-2
009
...gpars/GParallelizer...
AJUG_SEP2009 - 172
© A
SE
RT
2006-2
009
// run multiple closures in parallelAsynchronizer.withAsynchronizer {
assert [10, 20] == AsyncInvokerUtil.doInParallel({calculateA()},{calculateB()}
)}
// multiply numbers asynchronouslyParallelizer.withParallelizer(5) {
final List result = [1, 2, 3, 4, 5].collectAsync {it * 2}assert ([2, 4, 6, 8, 10].equals(result))
}
...gpars/GParallelizer...
AJUG_SEP2009 - 173
© A
SE
RT
2006-2
009
// support for dataflow to avoid doing synchronisationimport static org.gparallelizer.dataflow.DataFlow.thread
final def x = new DataFlowVariable()final def y = new DataFlowVariable()final def z = new DataFlowVariable()
thread {z << x.val + y.valprintln "Result: ${z.val}"
}
thread { x << 10 }
thread { y << 5 }
...gpars/GParallelizer
AJUG_SEP2009 - 174
© A
SE
RT
2006-2
009
// actor supportimport static org.gparallelizer.actors.pooledActors.PooledActors.*
def me = actor {friend.send('Hi')react(10.seconds) {
// continue conversation}
}me.metaClass.onTimeout = {->friend.send('I see, busy as usual. Never mind.')}me.start()
gpars and SimpBlog
AJUG_SEP2009 - 175
© A
SE
RT
2006-2
009
@Grab('net.sourceforge.htmlunit:htmlunit:2.6')import com.gargoylesoftware.htmlunit.WebClient@Grab('org.gparallelizer:GParallelizer:0.8.3')import static org.gparallelizer.Parallelizer.*
def testCases = [['Home', 'Bart', 'Content 1'],['Work', 'Homer', 'Content 2'],['Travel', 'Marge', 'Content 3'],['Food', 'Lisa', 'Content 4']
]
withParallelizer(3) {testCases.eachAsync{ category, author, content ->
postAndCheck category, author, content}
}
private postAndCheck(category, author, content) {...
Note:
Testing
DSL makes
tests more
readable
Native Groovy Versions also possible
AJUG_SEP2009 - 176
© A
SE
RT
2006-2
009
@Grab('net.sourceforge.htmlunit:htmlunit:2.6')import com.gargoylesoftware.htmlunit.WebClient
Thread.start {postAndCheck 'Home', 'Bart', 'Content 1'
}Thread.start {
postAndCheck 'Work', 'Homer', 'Content 2'}Thread.start {
postAndCheck 'Travel', 'Marge', 'Content 3'}Thread.start {
postAndCheck 'Food', 'Lisa', 'Content 4'}
private postAndCheck(category, author, content) {...
Or use:
ant.parallel {
//...
}
Or use:
"command".execute()
Constraint/Logic Programming...
• Description– Style of programming where relations between
variables are stated in the form of constraints
– First made popular by logic programming languages
such as Prolog but the style is now also used outside
logic programming specific languages
– Constraints differ from the common primitives of
other programming languages in that they do not
specify one or more steps to execute but rather the
properties of a solution to be found
– Popular libraries used with Groovy supporting
constraint programming include Gecode/J, Choco
and tuProlog
– We'll look at Choco as an example
AJUG_SEP2009 - 177
© A
SE
RT
2006-2
009
...Constraint/Logic Programming...
AJUG_SEP2009 - 178
© A
SE
RT
2006-2
009
Source: http://xkcd.com/287/
...Constraint/Logic Programming...
AJUG_SEP2009 - 179
© A
SE
RT
2006-2
009
// requires choco 2.1.0-basic.jar from http://choco.emn.fr/import static choco.Choco.*import choco.kernel.model.variables.integer.IntegerVariable
def m = new choco.cp.model.CPModel()def s = new choco.cp.solver.CPSolver()
def menu = ['Mixed fruit' : 215,'French fries' : 275,'Side salad' : 335,'Hot wings' : 355,'Mozzarella sticks' : 420,'Sampler plate' : 580
]def numOrdered = new IntegerVariable[menu.size()]def priceEach = new int[menu.size()]def sum = 1505...
Found a solution:7 * Mixed fruit
Found a solution:1 * Mixed fruit2 * Hot wings1 * Sampler plate
...Constraint/Logic Programming
AJUG_SEP2009 - 180
© A
SE
RT
2006-2
009
...menu.eachWithIndex { name, price, i ->
// number ordered >= 0// number ordered * price <= sumnumOrdered[i] = makeIntVar(name, 0, sum.intdiv(price))priceEach[i] = price
}m.addConstraint(eq(scalar(numOrdered, priceEach), sum))s.read(m)
def more = s.solve()while (more) {
println "Found a solution:"numOrdered.each {
def v = s.getVar(it)if (v.val) println " $v.val * $v.name"
}more = s.nextSolution()
}
SimpBlog Case Study...
• You have been asked to set up some test
cases representing the Simpsons weekly
blogging habits
• After some careful study you observe the
following strange behavior– They never blog on the same day
– Marge blogs only on a Saturday or Sunday
– Maggie blogs only on a Tuesday or Thursday
– Lisa blogs only on a Monday, Wednesday or Friday
– Bart blogs only on the day after Lisa
– Homer only blogs if noone else blogged the previous
day and doesn't allow anyone to blog the next day
AJUG_SEP2009 - 181
© A
SE
RT
2006-2
009
...SimpBlog Case Study...
AJUG_SEP2009 - 182
© A
SE
RT
2006-2
009
// requires choco 2.1.0-basic.jar from http://choco.emn.fr/import static choco.Choco.*
def m = new choco.cp.model.CPModel()def s = new choco.cp.solver.CPSolver()
daysOfWeek = ["Sunday", "Monday", "Tuesday", "Wednesday","Thursday", "Friday", "Saturday"]
def bart = makeIntVar('Bart', 0, 6)def homer = makeIntVar('Homer', 0, 6)def marge = makeIntVar('Marge', 0, 6)def lisa = makeIntVar('Lisa', 0, 6)def maggie = makeIntVar('Maggie', 0, 6)def simpsons = [bart, homer, marge, lisa, maggie]...
...SimpBlog Case Study...
AJUG_SEP2009 - 183
© A
SE
RT
2006-2
009
...
// They never blog on the same dayfor (i in 0..<simpsons.size())
for (j in 0..<i) m.addConstraint(neq(simpsons[i], simpsons[j]))
// Marge blogs only on a Saturday or Sundaym.addConstraint(or(eq(marge, 0), eq(marge, 6)))
// Maggie blogs only on a Tuesday or Thursdaym.addConstraint(or(eq(maggie, 2), eq(maggie, 4)))
// Lisa blogs only on a Monday, Wednesday or Fridaym.addConstraint(or(eq(lisa, 1), eq(lisa, 3), eq(lisa, 5)))
// Bart blogs only on the day after Lisam.addConstraint(eq(plus(lisa, 1), bart))
// Homer only blogs if noone else blogged the previous// day and doesn't allow anyone to blog the next daym.addConstraint(and(distanceNEQ(homer, marge, 1),
distanceNEQ(homer, bart, 1),distanceNEQ(homer, maggie, 1),distanceNEQ(homer, lisa, 1)))
...
...SimpBlog Case Study
AJUG_SEP2009 - 184
© A
SE
RT
2006-2
009
...
s.read(m)def more = s.solve()if (!more) println "No Solutions Found"else println pad("Solutions:") +
simpsons.collect{ pad(it.name) }.join()while (more) {
print pad("")println simpsons.collect {
def v = s.getVar(it)pad(daysOfWeek[v.val])
}.join()more = s.nextSolution()
}
def pad(s) { s.padRight(12) }
Solutions: Bart Homer Marge Lisa Maggie Thursday Saturday Sunday Wednesday Tuesday Tuesday Saturday Sunday Monday Thursday Saturday Tuesday Sunday Friday Thursday Thursday Sunday Saturday Wednesday Tuesday
Polyglot Programming...
AJUG_SEP2009 - 185
© A
SE
RT
2006-2
009
@Grab('org.clojure:clojure:1.0.0')import clojure.lang.Compilerimport clojure.lang.RT
def src = new File('temp.clj')src.text = '''(ns groovy)(defn factorial [n]
(if (< n 2)1(* n (factorial (- n 1))))
'''
src.withReader { reader ->Compiler.load reader
}
def fac = RT.var('groovy', 'factorial')println fac.invoke(4)
...def jy = getEngine("jython")jy?.eval('''def factorial(n):
i=fact=1while i <= n:
fact=fact*ii=i+1
return fact
result = factorial(4)''')println jy?.result
...Polyglot Programming
• But so what?– I can use Groovy for Scripting my
environment and or leveraging its runners
and other testing capabilities
– I can call out to other languages when needed
• Cucumber via JRuby for more native control
• Watir instead of Watij
• ScalaCheck for test data generation
• Jython for Robot Framework for more native
control
• Rhino for JavaScript testing
• Rules engine integration
AJUG_SEP2009 - 186
© A
SE
RT
2006-2
009
ModelJUnit...
• Description– Supports model-based testing
– Allows you to write simple finite
state machine (FSM) models or
extended finite state machine
(EFSM) models in Java or Groovy
– You can then generate tests from
those models and measure various
model coverage metrics
AJUG_SEP2009 - 187
© A
SE
RT
2006-2
009
...ModelJUnit...
AJUG_SEP2009 - 188
© A
SE
RT
2006-2
009
// require modeljunit.jarimport nz.ac.waikato.modeljunit.coverage.*import nz.ac.waikato.modeljunit.*
class VendingMachineModel implements FsmModel {def state = 0 // 0,25,50,75,100void reset(boolean testing) {state = 0}
boolean vendGuard() {state == 100}@Action void vend() {state = 0}
boolean coin25Guard() {state <= 75}@Action void coin25() {state += 25}
boolean coin50Guard() {state <= 50}@Action void coin50() {state += 50}
}
def tester = new RandomTester(new VendingMachineModel())tester.buildGraph()def metrics = [new ActionCoverage(), new StateCoverage(),
new TransitionCoverage(), new TransitionPairCoverage()]metrics.each { tester.addCoverageMetric it }
tester.addListener "verbose"tester.generate 20
println '\nMetrics Summary:'tester.printCoverage()
...ModelJUnit...
AJUG_SEP2009 - 189
© A
SE
RT
2006-2
009
...
done (0, coin50, 50)done (50, coin25, 75)done (75, coin25, 100)done Random reset(true)done (0, coin50, 50)done (50, coin25, 75)done (75, coin25, 100)done (100, vend, 0)done (0, coin50, 50)done (50, coin50, 100)done (100, vend, 0)done (0, coin25, 25)done (25, coin25, 50)done Random reset(true)done (0, coin50, 50)done (50, coin25, 75)done (75, coin25, 100)done (100, vend, 0)done (0, coin50, 50)done (50, coin25, 75)...
...
Metrics Summary:action coverage: 3/3state coverage: 5/5transition coverage: 7/8transition-pair coverage: 8/12...
ModelJUnit: SimpBlog Case Study...
• Does the order in which form information
is entered affect the application?– Could AJAX effects be causing unexpected results?
AJUG_SEP2009 - 190
© A
SE
RT
2006-2
009
// require modeljunit.jar, htmlunit.jarimport nz.ac.waikato.modeljunit.coverage.*import nz.ac.waikato.modeljunit.*import com.gargoylesoftware.htmlunit.WebClient
class SimpBlogModel implements FsmModel {
boolean authorSelected = falseboolean categorySelected = falseboolean titleEntered = falseboolean contentEntered = falseint count = 0def client, page, form
// Special known method, allows equivalence class definition// example states: __ __ __ __, AU __ __ __, AU CA TI COdef getState() {"${authorSelected ? ' AU ' : ' __ '}${categorySelected ? ' CA ' : ' __ '}" +"${titleEntered ? ' TI ' : ' __ '}${contentEntered ? ' CO ' : ' __ '}"
}...
...void reset(boolean testing) {authorSelected = falsecategorySelected = falsetitleEntered = falsecontentEntered = falseclient = new WebClient()page = client.getPage('http://localhost:8080/postForm')assert 'Welcome to SimpBlog' == page.titleTextform = page.getFormByName('post')
}...
...ModelJUnit: SimpBlog Case Study...
AJUG_SEP2009 - 191
© A
SE
RT
2006-2
009
...boolean "enter title Guard"() { !titleEntered }
@Action void "enter title "() { titleEntered = trueform.getInputByName('title').setValueAttribute("Title ${count++}")
}
boolean enterContentGuard() { !contentEntered }
@Action void enterContent() { contentEntered = trueform.getTextAreaByName('content').setText("Content ${count++}")
}
boolean chooseAuthorGuard() { !authorSelected }
@Action void chooseAuthor() { authorSelected = true // simple version just Lisaform.getSelectByName('author').getOptions().find{ it.text == 'Lisa' }.setSelected(true)
}
boolean pickCategoryGuard() { !categorySelected }
@Action void pickCategory() { categorySelected = true // simple version just Homeform.getSelectByName('category').getOptions().find{ it.text == 'Home' }.setSelected(true)
}
boolean "submit post Guard"() { categorySelected && authorSelected &&titleEntered && contentEntered }
@Action void "submit post "() {def result = form.getInputByName('btnPost').click()assert result.getElementsByTagName('h1').item(0).textContent.matches('Post.*: Title .*')// could do more asserts herereset(true)
}} // end of SimpBlogModel class definition...
...ModelJUnit: SimpBlog Case Study...
AJUG_SEP2009 - 192
© A
SE
RT
2006-2
009
def tester = new RandomTester(new SimpBlogModel())tester.buildGraph()def metrics = [
new ActionCoverage(),new StateCoverage(),new TransitionCoverage(),new TransitionPairCoverage()
]metrics.each {
tester.addCoverageMetric it}
tester.addListener "verbose"tester.generate 50
println '\nMetrics Summary:'tester.printCoverage()
def graphListener = tester.model.getListener("graph")graphListener.printGraphDot "simpblog.dot"println "\nGraph contains " + graphListener.graph.numVertices() +
" states and " + graphListener.graph.numEdges() + " transitions."
...ModelJUnit: SimpBlog Case Study...
AJUG_SEP2009 - 193
© A
SE
RT
2006-2
009
done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , enterContent, __ CA __ CO )done ( __ CA __ CO , enter title , __ CA TI CO )done ( __ CA TI CO , chooseAuthor, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , chooseAuthor, AU CA __ __ )done ( AU CA __ __ , enter title , AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , chooseAuthor, AU __ __ __ )done ( AU __ __ __ , pickCategory, AU CA __ __ )done ( AU CA __ __ , enter title , AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , enterContent, __ __ __ CO )done ( __ __ __ CO , pickCategory, __ CA __ CO )done ( __ CA __ CO , chooseAuthor, AU CA __ CO )done ( AU CA __ CO , enter title , AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , enter title , __ CA TI __ )done ( __ CA TI __ , chooseAuthor, AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , chooseAuthor, AU __ __ __ )done ( AU __ __ __ , pickCategory, AU CA __ __ )done ( AU CA __ __ , enter title , AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )...
...done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , enterContent, __ CA __ CO )done ( __ CA __ CO , chooseAuthor, AU CA __ CO )done ( AU CA __ CO , enter title , AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , chooseAuthor, AU __ __ __ )done ( AU __ __ __ , pickCategory, AU CA __ __ )done ( AU CA __ __ , enterContent, AU CA __ CO )done ( AU CA __ CO , enter title , AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done ( __ __ __ __ , chooseAuthor, AU __ __ __ )done ( AU __ __ __ , enter title , AU __ TI __ )done ( AU __ TI __ , pickCategory, AU CA TI __ )done ( AU CA TI __ , enterContent, AU CA TI CO )done ( AU CA TI CO , submit post , __ __ __ __ )done Random reset(true)done ( __ __ __ __ , pickCategory, __ CA __ __ )done ( __ CA __ __ , enterContent, __ CA __ CO )done ( __ CA __ CO , enter title , __ CA TI CO )done ( __ CA TI CO , chooseAuthor, AU CA TI CO )
Metrics Summary:action coverage: 5/5state coverage: 12/16transition coverage: 19/33transition-pair coverage: 26/56
Graph contains 16 states and 33 transitions.
...ModelJUnit: SimpBlog Case Study...
AJUG_SEP2009 - 194
© A
SE
RT
2006-2
009
Simplified
version
(just Lisa)
Advanced
version
...ModelJUnit: SimpBlog Case Study
AJUG_SEP2009 - 195
© A
SE
RT
2006-2
009
ScalaCheck
• Description– Tool for testing Scala, Java and Groovy programs
– Based on property specifications and automatic
random test data generation
AJUG_SEP2009 - 196
© A
SE
RT
2006-2
009
> scala -classpath ScalaCheck-1.5.jarWelcome to Scala version 2.7.6.final
scala> import org.scalacheck.Prop.forAll
scala> val propConcatLists = forAll { (l1: List[Int], l2: List[Int]) =>| l1.size + l2.size == (l1 ::: l2).size }
propConcatLists: org.scalacheck.Prop = Prop
scala> propConcatLists.check+ OK, passed 100 tests.
scala> val propSqrt = forAll { (n: Int) => scala.Math.sqrt(n*n) == n }propSqrt: org.scalacheck.Prop = Prop
scala> propSqrt.check! Falsified after 2 passed tests.> ARG_0: "-1" (1 shrinks, original arg: "-2")
ScalaCheck: SimpBlog Case Study...
AJUG_SEP2009 - 197
© A
SE
RT
2006-2
009
class SimpBlogChecker {static postAndCheck = nullstatic clean(s) { s.replace('\\', '\\\\').replace('\n', '\\n') }static ResultHolder postAndReturn(String title, String content,
String author, String category) {if (!postAndCheck) postAndCheck = new GroovyShell().parse('''@Grab('net.sourceforge.htmlunit:htmlunit:2.5')import com.gargoylesoftware.htmlunit.WebClientdef page = new WebClient().getPage('http://localhost:8080/postForm')def form = page.getFormByName('post')form.getInputByName('title').setValueAttribute(title)form.getSelectByName('category').getOptions().find {
it.text == category }.setSelected(true)form.getSelectByName('author').getOptions().find {
it.text == author }.setSelected(true)form.getTextAreaByName('content').setText(content)def result = form.getInputByName('btnPost').click()def titleResult = result.getElementsByTagName('h1').item(0).textContentdef h3headings = result.getElementsByTagName('h3')def categoryResult = h3headings.item(1).textContentdef authorResult = h3headings.item(2).textContentdef para = result.getByXPath('//TABLE//TR/TD/P')[0]def contentResult = para.textContentreturn new ResultHolder(titleResult, contentResult, authorResult, categoryResult)''')...
...ScalaCheck: SimpBlog Case Study...
AJUG_SEP2009 - 198
© A
SE
RT
2006-2
009
...def binding = new Binding()binding.setVariable('title', title)binding.setVariable('content', clean(content))binding.setVariable('author', author)binding.setVariable('category', category)postAndCheck.binding = bindingpostAndCheck.run()
}}
class ResultHolder {String title, content, author, categoryResultHolder(String title, String content,
String author, String category) {this.title = titlethis.content = contentthis.author = authorthis.category = category
}}
> groovyc SimpBlogChecker.groovy
...ScalaCheck: SimpBlog Case Study...
AJUG_SEP2009 - 199
© A
SE
RT
2006-2
009
//CheckSimpBlog.scala
import org.scalacheck.Prop._import org.scalacheck.ConsoleReporter.testStatsEximport org.scalacheck.Test.checkimport org.scalacheck.Arbitrary._import org.scalacheck.Gen._
object CheckSimpBlog {
val fieldsGen = for {title <- elements("Title 1", "Title 2")content <- arbitrary[String]author <- elements("Bart", "Homer", "Lisa", "Marge", "Maggie")category <- elements("Home", "Work", "Food", "Travel")
} yield (title, content, author, category)...
> scalac -classpath .;../lib/ScalaCheck-1.5.jar;../../groovy-1.7-beta-2-SNAPSHOT/embeddable/groovy-all-1.7-beta-2-SNAPSHOT.jar CheckSimpBlog.scala
...ScalaCheck: SimpBlog Case Study...
AJUG_SEP2009 - 200
© A
SE
RT
2006-2
009
...val enterFieldsAcceptedAndEchoedProperty = forAll(fieldsGen)(
fields =>{
val (title, content, author, category) = fieldsval result = SimpBlogChecker.postAndReturn(title, content, author, category)result.getTitle contains titleresult.getContent contains contentresult.getAuthor contains authorresult.getCategory contains category
})
val tests = scala.List(("enterFieldsAcceptedAndEchoedProperty",
enterFieldsAcceptedAndEchoedProperty))
def main(args: scala.Array[String]) =tests foreach { case (name, p) => testStatsEx(name, check(p)) }
}
> scala -classpath .;../lib/ScalaCheck-1.5.jar;../../groovy-1.7-beta-2-SNAPSHOT/embeddable/groovy-all-1.7-beta-2-SNAPSHOT.jar;../../groovy-1.7-beta-2-SNAPSHOT/lib/ivy-2.1.0-rc2.jar CheckSimpBlog+ OK, passed 100 tests.
...ScalaCheck: SimpBlog Case Study
AJUG_SEP2009 - 201
© A
SE
RT
2006-2
009
More Information about Groovy
• Web sites– http://groovy.codehaus.org
– http://grails.codehaus.org
– http://pleac.sourceforge.net/pleac_groovy (many examples)
– http://www.asert.com.au/training/java/GV110.htm (workshop)
• Mailing list for users– [email protected]
• Information portals– http://www.aboutgroovy.org
– http://www.groovyblogs.org
• Documentation (1000+ pages)– Getting Started Guide, User Guide, Developer Guide, Testing
Guide, Cookbook Examples, Advanced Usage Guide
• Books– Several to choose from ...
AJUG_SEP2009 - 202
© A
SE
RT
2006-2
009
More Information: Groovy in Action
AJUG_SEP2009 - 203
© A
SE
RT
2006-2
009
Second edition of GinA, ‘ReGinA’ now under development