1
Introduction to TestNG
and its Application for
Test-Driven GUI Development
Zheng-Wen Shen
2007/05/14
2
References
1. "TestNG makes Java unit testing a breeze“, Filippo Diotalevi, developerWorks, January 2005
2. "In pursuit of code quality: JUnit 4 vs. TestNG“, Andrew Glover, developerWorks, Aug 2006
3. "In pursuit of code quality: Automate GUI testing with TestNG-Abbot“, Andrew Glover, developerWorks, February 2007
4. Alex Ruiz, Yvonne Wang Price, "Test-Driven GUI Development with TestNG and Abbot," IEEE Software, vol. 24, no. 3, pp. 51-57, May/Jun, 2007.
3
Outline
1. Introduction
2. TestNG Quick Start
3. TestNG Features
4. TestNG-Abbot: Test-Driven GUI
Development
5. Summary
4
1. Introduction 1/4
• JUnit: simple, pragmatic, and strict architecture
• Rules for using JUnit of unit testing
– Every single piece of code must be tested.
– Code must be tested in isolation (mock objects).
– Software must be easy testable (written with tests in
mind.)
5
1. Introduction 2/4
Two opposed factions
KISS principle
The simplicity is necessary
Advanced features!
more flexibility!
more power!
6
1. Introduction 3/4
Some peculiar features of JUnit
• The need to extend a TestCase class
• Impossible to pass parameters (setUp, tearDown)
• The execution model is a bit strange
– The test class is reinstantiated every time a test
method is executed.
• The management of different suites of tests in
complex projects can be very tricky.
7
1. Introduction 4/4
1. TestNG: Testing, the Next Generation
• To cover all categories of tests:
unit, functional, end-to-end, integration, …
• JDK 5 Annotation
• Flexible test configuration
• Powerful execution model (no more TestSuite)
2. TestNG-Abbot
• 0.3.2 Release, 2007/05/08
• Capture-Replay GUI Testing
• Test-Driven GUI Development
Cedric Beust
8
2. TestNG Quick Start 1/3
A Test Class
The Lang Component provides a host
of helper utilities for the java.lang API,
notably String manipulation methods,
basic numerical methods, object
reflection, creation and serialization,
and System properties.
2
1
3
9
2. TestNG Quick Start 2/3
Configuration XML
Suite
Test 1
groups
classes
Test 2
groups
classes
…
testing.xml
10
2. TestNG Quick Start 3/3
Test Results HTML output
11
3. TestNG Features
1. Configuration Methods
2. Test Group
3. Dependency Testing
4. Parametric Testing
5. Factories
6. Fail and Rerun
12
3.1 Configuration Methods 1/1
• @Configuration 1. beforeTestClass
2. afterTestClass
3. beforeTestMethod
4. afterTestMethod
beforeTestClass methods
afterTestClass methods
beforeTestMethod methods
afterTestMethod methods
first test method
beforeTestMethod methods
afterTestMethod methods
second test method
Enter
Exit
…
13
3.2 Test Group 1/2
• Groupings of test methods
• Maximum flexibility doesn’t recompile anything
• Check-in tests
– Run before submit new code
– Fast, just make sure no basic functionality was broken
• Functional tests
– Cover all the functionalities of software
– Run at least once a day
Functional Test
Check-in tests
14
3.2 Test Group 2/2
Functional Test
Check-in tests testMethod1
testMethod2
testMethod3 testMethod1
testMethod2
testing.xml
15
3.3 Dependency Testing 1/2
• Invoke test methods in a certain order.
– A certain number of test methods have completed
and succeeded before running more test methods.
– Initialization methods to be test methods
• Hard dependencies
• Soft dependencies
16
3.3 Dependency Testing 2/2
(Hard) Dependency on method
(Hard) Dependency on group
@Test {groups = {…} alwaysRun=true}
Soft Dependency
17
3.4 Parametric Testing 1/3
• Use arbitrary number of parameters on test
method
• Instruct TestNG to pass the correct parameters
with the @Parameters annotation.
– From testng.xml
– From DataProviders
18
3.4 Parametric Testing 2/3
From testing.xml
testing.xml
first-name = “Cedric”
19
3.4 Parametric Testing 3/3
From DataProviders
Same class
Different classes
20
3.5 Factories 1/2
• Create tests dynamically.
– Access a Web page several times with different
values
Test Method
21
3.5 Factories 2/2
22
3.6 Fail and Rerun 1/1
testing.xml testing_failed.xml
TestNG
Delineates the failed tests Run
23
4. TestNG-Abbot 1/4
Test-Driven GUI Development
• TestNG-Abbot (0.3.2 release, 2007/05/08)
• Intuitive fixture classes
– Logically linked to the code under test by name only.
– Tests do not rely on GUI components being
specifically located
– Tests can be authored early and withstand layout and
aesthetic changes during the development phase
24
4. TestNG-Abbot 2/4
Application Under Test
1. The user enters name
and email address
2. The email address
must be valid
3. The system displays
an error message if
the required input is
missing or invalid
25
4. TestNG-Abbot 3/4
The test class for testing Model
NewEmailAccountWizard View
Email Model
26
4. TestNG-Abbot 4/4
The test class for testing View
1
2
4
3
5
27
5. Summary
• TestNG
– annotations-based framework
– Make the whole testing process much more simple
and flexible
– Free to choose the testing strategy you prefer
• Practice TDD, even for GUIs