1

Introduction to TestNG

and its Application for

Test-Driven GUI Development

Zheng-Wen Shen

2007/05/14

2

References

1. "TestNG makes Java unit testing a breeze“, Filippo Diotalevi, developerWorks, January 2005

2. "In pursuit of code quality: JUnit 4 vs. TestNG“, Andrew Glover, developerWorks, Aug 2006

3. "In pursuit of code quality: Automate GUI testing with TestNG-Abbot“, Andrew Glover, developerWorks, February 2007

4. Alex Ruiz, Yvonne Wang Price, "Test-Driven GUI Development with TestNG and Abbot," IEEE Software, vol. 24, no. 3, pp. 51-57, May/Jun, 2007.

3

Outline

1. Introduction

2. TestNG Quick Start

3. TestNG Features

4. TestNG-Abbot: Test-Driven GUI

Development

5. Summary

4

1. Introduction 1/4

• JUnit: simple, pragmatic, and strict architecture

• Rules for using JUnit of unit testing

– Every single piece of code must be tested.

– Code must be tested in isolation (mock objects).

– Software must be easy testable (written with tests in

mind.)

5

1. Introduction 2/4

Two opposed factions

KISS principle

The simplicity is necessary

Advanced features!

more flexibility!

more power!

6

1. Introduction 3/4

Some peculiar features of JUnit

• The need to extend a TestCase class

• Impossible to pass parameters (setUp, tearDown)

• The execution model is a bit strange

– The test class is reinstantiated every time a test

method is executed.

• The management of different suites of tests in

complex projects can be very tricky.

7

1. Introduction 4/4

1. TestNG: Testing, the Next Generation

• To cover all categories of tests:

unit, functional, end-to-end, integration, …

• JDK 5 Annotation

• Flexible test configuration

• Powerful execution model (no more TestSuite)

2. TestNG-Abbot

• 0.3.2 Release, 2007/05/08

• Capture-Replay GUI Testing

• Test-Driven GUI Development

Cedric Beust

8

2. TestNG Quick Start 1/3

A Test Class

The Lang Component provides a host

of helper utilities for the java.lang API,

notably String manipulation methods,

basic numerical methods, object

reflection, creation and serialization,

and System properties.

2

1

3

9

2. TestNG Quick Start 2/3

Configuration XML

Suite

Test 1

groups

classes

Test 2

groups

classes

…

testing.xml

10

2. TestNG Quick Start 3/3

Test Results HTML output

11

3. TestNG Features

1. Configuration Methods

2. Test Group

3. Dependency Testing

4. Parametric Testing

5. Factories

6. Fail and Rerun

12

3.1 Configuration Methods 1/1

• @Configuration 1. beforeTestClass

2. afterTestClass

3. beforeTestMethod

4. afterTestMethod

beforeTestClass methods

afterTestClass methods

beforeTestMethod methods

afterTestMethod methods

first test method

beforeTestMethod methods

afterTestMethod methods

second test method

Enter

Exit

…

13

3.2 Test Group 1/2

• Groupings of test methods

• Maximum flexibility doesn’t recompile anything

• Check-in tests

– Run before submit new code

– Fast, just make sure no basic functionality was broken

• Functional tests

– Cover all the functionalities of software

– Run at least once a day

Functional Test

Check-in tests

14

3.2 Test Group 2/2

Functional Test

Check-in tests testMethod1

testMethod2

testMethod3 testMethod1

testMethod2

testing.xml

15

3.3 Dependency Testing 1/2

• Invoke test methods in a certain order.

– A certain number of test methods have completed

and succeeded before running more test methods.

– Initialization methods to be test methods

• Hard dependencies

• Soft dependencies

16

3.3 Dependency Testing 2/2

(Hard) Dependency on method

(Hard) Dependency on group

@Test {groups = {…} alwaysRun=true}

Soft Dependency

17

3.4 Parametric Testing 1/3

• Use arbitrary number of parameters on test

method

• Instruct TestNG to pass the correct parameters

with the @Parameters annotation.

– From testng.xml

– From DataProviders

18

3.4 Parametric Testing 2/3

From testing.xml

testing.xml

first-name = “Cedric”

19

3.4 Parametric Testing 3/3

From DataProviders

Same class

Different classes

20

3.5 Factories 1/2

• Create tests dynamically.

– Access a Web page several times with different

values

Test Method

21

3.5 Factories 2/2

22

3.6 Fail and Rerun 1/1

testing.xml testing_failed.xml

TestNG

Delineates the failed tests Run

23

4. TestNG-Abbot 1/4

Test-Driven GUI Development

• TestNG-Abbot (0.3.2 release, 2007/05/08)

• Intuitive fixture classes

– Logically linked to the code under test by name only.

– Tests do not rely on GUI components being

specifically located

– Tests can be authored early and withstand layout and

aesthetic changes during the development phase

24

4. TestNG-Abbot 2/4

Application Under Test

1. The user enters name

and email address

2. The email address

must be valid

3. The system displays

an error message if

the required input is

missing or invalid

25

4. TestNG-Abbot 3/4

The test class for testing Model

NewEmailAccountWizard View

Email Model

26

4. TestNG-Abbot 4/4

The test class for testing View

1

2

4

3

5

27

5. Summary

• TestNG

– annotations-based framework

– Make the whole testing process much more simple

and flexible

– Free to choose the testing strategy you prefer

• Practice TDD, even for GUIs