The Use of TTCN-3for Software TestingThe Use of TTCN-3for Software Testing
Ina Schieferdecker
FOKUS, [email protected]
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Content
• Overview on TTCN-3• TTCN-3 applied to software• TTCN-3 and XML• Test Execution
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Content
Overview on TTCN-3• TTCN-3 applied to software• TTCN-3 and XML• Test Execution
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Introduction• TTCN-3 is the Testing and Test Control Notation • The new standardised test specification and test
implementation language• Developed from 1999 – 2002 at the European
Telecommunications Standards Institute (ETSI).• Developed based on experiences from previous TTCN
editions• Removal of OSI specific concepts; Improvement of concepts;
Introduction of new concepts.• Applicable for all kinds of black-box testing for reactive
and distributed systems, e.g.,• Telecom systems (ISDN, ATM, GSM, UMTS); Internet (IP, IP
based protocols and applications); Software systems (Java, XML); Middleware platforms and component-based systems (CORBA, .Net, EJB).
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Overview on TTCN-3
Presentation Format n
Graphical Format
Tabular Format
TTCN-3CoreLanguage
Other Types & Values n
Other Types & Values 2
ASN.1 Types & Values
:testcase myTestcase () runs on MTCType system TSIType{ mydefault := activate (OtherwiseFail);
verdict.set(pass);
:connect(PTC_ISAP1:CP_ISAP1,mtc:CP_ISAP1);:map(PTC_ISAP1:ISAP1, system:TSI_ISAP1);:PTC_ISAP1.start(func_PTC_ISAP1());PTC_MSAP2.start(func_PTC_MSAP2());Synchronization(); all component.done;log(”Correct Termination”);
}
:
msc mi_synch1_conc1
mtc ISAP1 MSAP2
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
TTCN-3 Test Case
TTCN-3 – Based Black-Box Testing
Port.send(Stimulus) Port.receive(Response)
System Under Test
Port
• Assignmentof aTest Verdict
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
TTCN-3 Test Case
TC TCs
TC create
create
TC
TCsMTC
create
TC
SUT
TCs
TC start
start
start
TC
Component-Based Test System
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Main Elements of TTCN-3• Module covers declarations and control• Templates (test data description) and
matching mechanisms (pattern matching) • Test configurations
• Formally defined interfaces to the SUT• Dynamic creation of test component• Concurrency to describe distributed test setups
• Test cases• Small (complete) separate compilable programs • Share (type and data) information
• Test verdicts
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Content
• Overview on TTCN-3 TTCN-3 applied to software • TTCN-3 and XML• Test Execution
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
TTCN-3 and Software Testing
• Message-Based Software• Such as Protocols,
Messaging Services, Web site and Portals, ...
• Example technology is XML
• Signature-Based Software• Such as Client-Server,
Peer-to-Peer, Component-based Systems, ....
• Example technology is IDL
Reuse of XML/IDL/... data within TTCN-3 Define a mapping from XML/IDL/... to TTCN-3
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Presentation Format n
Graphical Format
Tabular Format
Other Types & Values 2
IDL
ASN.1 Types & Values
Other Types & Values n
TTCN-3CoreLanguage
Presentation Format n
TTCN-3 and Software Testing
C, C++, JAVA
XML
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Content
• Overview on TTCN-3• TTCN-3 applied to software TTCN-3 and XML• Test Execution
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
XML in Distributed Applications
• Increasing number of distributed applications use XML for
• Description of messages in datacom protocols• Exchange format between software components• Data description in Web-applications• etc.
• XML• is a structured method for putting data into a textual
presentation by marking up data• can have attributes that describe additional information• describes both attributes and content• is intuitive and self describing
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Mapping XML to TTCN-3
• Idea:
Map element tags and attributes to TTCN-3 fields
• Different grammar definitions and mappings• Schemas
• Embedded approach• Flat-Catalog approach• Named Type approach
• DTDs
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Example – The Dinosaur Database
XMLPaleontologist feeding the database
Student requesting data
ExampleExampleExampleExample
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
The Request Interface
XML
URLhttp://www.testingtech.de/TTCN-3_Example/dinolist.xml
XML File
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Web Service
Generation of test data structure
TestSystem
TestComponent
TestComponent
TestComponent
TestComponent
Compilationto Executable Tests
XML
ADAPTOR
Adaptor acc. to the
mapping rules
Generation of test behavior
Generation of test data
Principal Approach
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Structured Type Defintions
ExampleExampleExampleExample
XML DTD<!ELEMENT dinolist (dinosaur*)>
<!ELEMENT dinosaur (name, len, mass, time, place)>
<!ELEMENT name (#PCDATA)>
<!ELEMENT len (#PCDATA)>
<!ELEMENT mass (#PCDATA)>
<!ELEMENT time (#PCDATA)>
<!ELEMENT place (#PCDATA)>
type set of dinosaur dinolist;
type record dinosaur { charstring name, charstring len, charstring mass, charstring time, charstring place
}
Set of Type Definition
Field Definition
Record Type Definition
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Test Data Definitions
template dinolist DinoList := {?, ?, Brachiosaurus, ?, ?, ?, ?};
template dinosaur Brachiosaurus := {name := "Brachiosaurus",len := ?,mass := ?,time := ?,place := ?
}
ExampleExampleExampleExample
Any Value
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Test Port
/** communication port type definition */ type port httpTestPortType message {
out url;in dinolist;
}
Port Definition
ExampleExampleExampleExample
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Test Components
/** component type definitions */ type component httpTestComponent {
port httpTestPortType httpPort;timer localTimer := 3.0;
}
type component httpTestSystemComponent {port httpTestPortType httpTestSystemPort;
}
Component Definition
ExampleExampleExampleExample
Local Port
Local Timer
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Test Behavior
httpPort.send(requestURL);localTimer.start;alt { [] httpPort.receive(DinoList) {
localTimer.stop;setverdict(pass);
} [] httpPort.receive {
localTimer.stop;setverdict(fail);
} [] localTimer.timeout {
setverdict(fail); }}
Alternative
reactions
ExampleExampleExampleExample
Sending a
message
Starting the timer
The expected response
An unexpected
responseA timeout
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
altstep DinoList_Default_1() runs on httpTestComponent { [] httpPort.receive {
localTimer.stop;setverdict(fail);
} [] localTimer.timeout {
setverdict(fail); }
}
Handling of unexpected
response
Simplification: Altstep
Test Component
Type
Handling of timeouts
ExampleExampleExampleExample
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
...activate(DinoList_Default_1());
httpPort.send(requestURL);localTimer.start;
httpPort.receive(DinoList); localTimer.stop; setverdict(pass);
…
The expected response is
given here only, All other cases are handled by
the default
Simplification: Use of the Altstep
Default activation
ExampleExampleExampleExample
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
testcase DinoList_Test_1() runs on httpTestComponent system httpTestSystemComponent {
map(mtc:httpPort, system:httpTestSystemPort);
activate(DinoList_Default_1()); httpPort.send(requestURL);
localTimer.start; httpPort.receive(DinoList); localTimer.stop; setverdict(pass); }
A Test Case
MTC Type
ExampleExampleExampleExample
Test Case Definition
TSI Type
Mapping the Ports
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Graphical Formattestcase DinoList_Test_1 runs on httpTestComponent system httpTestSystemComponent
httpTestComponent
mtc
requestURL
DinoList
httpPort
activate(DinoList_Default_1)
pass
LocalTimer
map(mtc:httpPort,system:httpTestSystemPort)
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Content
• Overview on TTCN-3• TTCN-3 applied to software• TTCN-3 and XML Test Execution
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
SUT
Real Test System Interface
Test Execution
Test System
Abstract Test System Interface
TC2TC1
IN OUT
OUT IN
Connected Ports
INOUTMapped Ports
INOUT
Real Test System Interface
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
TTCN-3 Execution
Test System
Test System User
TE
TM: Management
System Under Test (SUT)
CD
: C
oD
ec
CH
: C
om
po
ne
nt
Ha
nd
ling
SA: Communication PA: Timer
• Generic XML adaptor that adheres to the mapping
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
TTCN-3 Runtime Interface
• Adaptation to the SUT
Test System User
TE
TM: Management
System Under Test (SUT)
CD
: C
oD
ec
CH
: C
om
po
ne
nt
Ha
nd
ling
SA: Communication PA: TimerTRI
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
TTCN-3 Control Interfaces
• Adaptation to the test platform/test device
Test System User
TE
TM: Management
System Under Test (SUT)
CD
: C
oD
ec
CH
: C
om
po
ne
nt
Ha
nd
ling
SA: Communication PA: Timer
TCI
ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3
Summary of TTCN-3• New version of the only standardized test notation• Modernization: Programming-like test specification with
flexible data support and various representation formats• Wider scope of application
• applicable to many kinds of test applications not just conformance (development, system, integration, interoperability, scalability …)
• applicable in the datacom domain• Harmonization
• first choice for test specifiers, implementors and users both for standardized test suites and
• as a generic solution in industrial software development• Was successfully shown to be applicable to testing of IDL
and XML interfaces, Java and C++ classes, …• Tools are available
Thank You.
Questions?
Thank You.
Questions?