Post on 27-Mar-2015
transcript
UML Testing Profile
© Telelogic AB
Graphical Testing With UMLThe UML Testing Profile
Eric Samuelssoneric.samuelsson@telelogic.com
UML Testing Profile
© Telelogic AB
Agenda
• Background
• Key concepts
• Advanced concepts
• Summary
UML Testing Profile
© Telelogic AB
Background
• Test technologies are not on par with modern design technologies such as model-driven architectures (MDA)
• There’s a need for a widespread graphical test language
• OMG realized this and issued and RFP for a graphical testing profile to UML
– Using the UML ensures widespread use across a wide range of application areas
– Numerous tool vendors support UML
– UML is a proven de facto standard
– But UML is focused on definition of system structure and behavior and has no built-in testing constructs...
UML Testing Profile
© Telelogic AB
UML Testing Profile RFP
• Issued by OMG in July 2001
• LOI’s submitted by
– Telelogic, Rational, SofTEAM, FOKUS, Ericsson, Motorola, IBM
• One joint submission
• Initial submission made in April 2002
• Revised initial submission made in June 2002
– Terminology changes
– Elaboration of Test Data
• Final submission in January 2003
UML Testing Profile
© Telelogic AB
Requirements and restrictions
• Define structural and behavioral test concepts
• Conformance testing only
– Black-box functional testing
• Based upon the UML meta-model
– UML 2.0 as proposed by the U2 Partners
– Inter-operation with existing testing technologies
– TTCN 3
– Junit
– ...
UML Testing Profile
© Telelogic AB
Current proposal
• A graphical testing language based the current UML 2 metamodel
• Reusing as many UML concepts as possible
– Recognition important for UML users
– No need to re-invent the wheel
• Designed with TTCN 3 in mind
– Semantics basically follows TTCN 3 semantics
– But caters for other technologies like JUnit as well
UML Testing Profile
© Telelogic AB
Current proposal
• Defined as a UML profile
– Will be recognized by UML users
• Basically a number of stereotypes
– Easy to implement for tool vendors
• Models will be interchangeable
– Any tool implementing XMI will be able to exchange models
«profile»Testing Profile«import»
UML
«stereotype»TestCase
«metaclass»BehavioralFeature
UML Testing Profile
© Telelogic AB
Key Concepts
• Test Architecture
– Test Context, Test Configuration
– Test Component, SUT
– Arbiter
• Test Behavior
– Test Case
– Verdict, Validation Action
– Defaults
• Test Data
– Logical partitions, Pattern matching
• Test Deployment
UML Testing Profile
© Telelogic AB
Test Objective
• An informal description of what to test in one test case
• Commonly expressed using natural language
• Can also be expressed using other diagrams, such asuse case diagrams and sequence diagrams
”Verify that if a user inserts and authorizes a valid card correctly, he is able to withdraw money if he has sufficient funds.”
Client ClientWithdrawal
InvalidPinTester
<<include>>
UML Testing Profile
© Telelogic AB
Test Case
• A specification of one test, including
– Test objective
– Test behavior
• Defined as a BehavioralFeature in UML
• It’s method is the behavior of the test case
«testCase» +validWithdrawal() : Verdict
«testContext»ATMContext
«testCase»+validWithdrawal() : Verdict+invalidPIN() : Verdict-authorizeCard() : Verdict
UML Testing Profile
© Telelogic AB
Test Behavior
• Describes the expected behavior of a test case
• Defined using any type of UML behavior
– Interactions, statemachines and activities
• Facilitates specification of arbitrarily complex behavior
sd ATMContext
invalidPIN()refref
[verdict == fail][verdict == pass]
validWithdrawal()refref
storeCardData(currentCard)
isPinCorrect(enteredPIN)
true
display(”Enter PIN”)
«sut»atm
hwe
Idle
WaitCode
UML Testing Profile
© Telelogic AB
Test Architecture
• A complete specification of all types used in the tests
– An ordinary UML package
– Contains Test Contexts, Test Components, Ports, Interfaces, etc.
«testArchitecture»
ATMTest
UML Testing Profile
© Telelogic AB
Test Architecture Example
«testArchitecture»
ATMTest
«testContext»ATMContext
«testCase»+validWithdrawal() : Verdict+invalidPIN() : Verdict-authorizeCard() : Verdict
Account
balance : Integernumber : String
credit(a : Integer)debit(a : Integer)
* accounts
ATM::CardData
pinCode : Integernumber : String
isPinCorrect(c: Integer) : Boolean
*
cards
«testComponent»BankEmulator
IAccount
bePort-pinOk : Boolean-enteredPIN : String-message : String
«testComponent»HWEmulator
hwCom
IScreen, ICardMachine,IMoneyBox
IATM
UML Testing Profile
© Telelogic AB
Test Component
• Implements (parts of ) the behavior of one or more test cases
• Each test case is normally executed by several components
• Ordinary classes or components
-pinOk : Boolean-enteredPIN : String-message : String
«testComponent»HWEmulator
hwCom
ATM::CardData
pinCode : Integernumber : String
isPinCorrect(c: Integer) : Boolean
0..1 -currentCard
IScreen, ICardMachine, IMoneyBox
IATM
UML Testing Profile
© Telelogic AB
Test Context
• A classifier that contains
– a number of test cases and their behavior and objectives
– default behavior used by the test cases
– configuration of test components
– test control behavior
• Can be a class, a component or a collaboration«testContext»ATMContext
«testCase»+validWithdrawal() : Verdict+invalidPIN() : Verdict-authorizeCard() : Verdict
UML Testing Profile
© Telelogic AB
Test Configuration
• A specification of the structural aspects of test cases
• Defines the parts participating in the test case and their interconnections
– Test Components, SUT, Ports, Interfaces and Connectors
• Defined as the internal structure of the Test Context
«sut»atm : BankATM
hwe : HWEmulator
be : BankEmulator
atmPort
hwCom
bePortnetCom
accnts : Account [0..*]
UML Testing Profile
© Telelogic AB
Test Control
• Test execution can be controlled is two ways
– By defining the classifier behavior of a test context
– Or by invoking the test cases from outside of the context
UML Testing Profile
© Telelogic AB
Test Control by ClassifierBehavior
sd ATMContext
invalidPIN()ref
[verdict == fail][verdict == pass]
validWithdrawal()ref
«testContext»ATMContext
«testCase»+validWithdrawal() : Verdict+invalidPIN() : Verdict-authorizeCard() : Verdict
UML Testing Profile
© Telelogic AB
Test Control by Invocation
*
-context
MyClass
-verdict : Verdict
«testContext»MyTestContext
«testCase»+TC1() : Verdict +TC2() : Verdict -TC3() : Verdict
ad myOp
verdict := context.TC1()
[else][verdict == pass]
verdict := context.TC2()
+myOp()
UML Testing Profile
© Telelogic AB
Test Data
• Parameters, default values, return values are reused from UML without changes
• Logical partitions
• Pattern matching
• TDB
UML Testing Profile
© Telelogic AB
Test Deployment
• Test Deployment is important, but there is probably no difference between deploying test systems compared to ordinary UML systems.
• We will try to reuse the existing UML deployment concepts without changes
• Under investigation
UML Testing Profile
© Telelogic AB
Advanced Concepts
• Test Evaluation
– Verdict
– Validation action
– Arbiter
• Default behavior
• Structural and behavioral reuse
UML Testing Profile
© Telelogic AB
Test Evaluation
• A number of concepts are used to evaluate test results
– Verdicts
– Validation actions
– Arbiter
UML Testing Profile
© Telelogic AB
Verdicts and Validation Actions
• Each test component is associated with a verdict
• Validation Actions are used to set the verdict
• A Validation Action sets the verdict of a test component and reports the verdict to the Arbiter
Client«sut»ATM
storeCardData(cardData)
isPinCorrect(pin)
true
display(”Enter PIN”)
alt
false
setverdict(fail)
UML Testing Profile
© Telelogic AB
Arbiter
• A test component that knows how to evaluate the test behavior
• Separates test evaluation from test case specification
– One behavior can yield different results
• Each test context has a default arbiter
– Implicit connections to all test components
– TTCN 3 semantics
• Advanced arbiters can be defined
– Caters for more advanced semantics
«arbiter»myArbiter
setVerdict(v : Verdict)
UML Testing Profile
© Telelogic AB
Default behavior
• Defines behavior for unexpected observations
• Can be expressed using any UML behavior
• Can be applied to several different levels:
– Test component
– Message reception
– Alternative statement
– ...Client
«sut»ATM
storeCardData(cardData)
isPinCorrect(pin)
true
display(”Enter PIN”)
sd authorizeCard
enterPinDefault
UML Testing Profile
© Telelogic AB
Default expressed as interaction...
sd enterPinDefault
self
alt getStatus
statusOk
ejectCard()
display(”Connection lost”)
setverdict(pass)
setverdict(inconc)
setverdict(fail)
UML Testing Profile
© Telelogic AB
...and as statemachine
The representations are semantically equivalent
enterPinDefault
*
getStatus / ^statusOk, setverdict(pass);
display(”Connection lost”) /setverdict(inconc)
ejectCard /setverdict(fail)
UML Testing Profile
© Telelogic AB
Meta Default
• Each test component has a meta default to cater for differences in semantics
• Can be redefined
MetaDefault
*
* / defer
UML Testing Profile
© Telelogic AB
Reuse of Test Structure and Behavior
UML Testing Profile
© Telelogic AB
«testContext»ATMContext
«sut»atm : BankATM
hwe : HWEmulator
be : BankEmulator
atmPort
hwCom
bePortnetComaccnts : Account [0..*]
ATM
Client
ac: AuthorizationContext
UML Testing Profile
© Telelogic AB
sd ValidWithdrawal
hwe«sut»atm be
refac.authorizeCard()
selectOperation(withdrawal)
true
withdraw(amount)
netCom
findAccount(cardData)
account
display(”Take cash”)
deliverMoney(amount)
true
debitAccount(account, amount)
true
atmPort
UML Testing Profile
© Telelogic AB
Summary
• The testing profile enables complete graphical specification of tests for conformance testing of UML models
– Defined as a UML profile
– Reuses existing UML concepts whenever possible
– Designed for inter-operation with TTCN-3 and other technologies
• Due date
– January 6th, 2003
• More information
– Testing Profile Consortium
– OMG’s Testing Profile page