Post on 03-Jan-2016
description
transcript
Multi-Paradigm Models as Source for Automatic Test Construction
Victor Kuliamin
ISP RAS, Moscow
Why Multiple Models?
Requirements
Functionality ReliabilityEfficiency Usability
0,000
0,200
0,400
0,600
0,800
1,000
1,200
1,400
1,600
1,800
2,000
1 2 3 5 10 15 20
f1()
f2()
accept()
read()
modifyAll()
Testing
?
Modeling Techniques
OperationalCan be executed by virtual machine
ContractPre- and postconditions, data integrity constraints
History-basedConstraints on possible traces
AlgebraicEquivalence between different execution histories
(C)(E)FSM, LTS, PN, CSP, ASMSDL, LOTOS, Lustre, VDM, Murphi, Simulink
Z, B, ADL, JML, Eiffel, VDM, RSLLarch-C++
TL, MSC
Larch, ML, OBJ
Tasks of Testing
Software under Test
Evaluate Correctness
Organize Bundle of
Test Inputs
Construct
Single Test Input
Evaluate Testing Quality
Test Results
Transform Test Inputs and Responses
Gather Responses
Modeling Techniques Comparison
Operational Contract History-based Algebraic
Behavior Evaluation Closeness to Requirements
Low-level Coverage
High-level Coverage
Test Sequence Construction
Single Input Construction
Scalability
Concurrency
Comparison Results
There is no the best technique No one technique is good for everything May be a mix of different approaches can
fit more needs?
UniTesK Technology
Model-based testing technology
Developed in 2000 – 2002 in
ISP RAS
UniTesK Solutions
Contract specifications of behavior
FSM and LTS testing models
Contract Specifications
Preconditions and postconditionsof interface operations andasynchronous reactions
Data integrity constraints
Close to requirements Suitable for oracle generation Provide low-level coverage
criteria
Functional Requirements
Contract Specifications
FSM and LTS Testing Models
Define states and admissible inputactions
More abstract than originalspecifications
Guarantee some low-level coverage
Suitable for test sequence construction
Provide high-level coverage criteria
Contract Specifications
!Coverage Requirements
?
!
?!
!
?
?
?
Relation between Models
states
parameters operation domain
1
2
3coverage goals
Whole Picture I
Software under Test
Model of Behavior Testing Model
Coverage Model
Test OracleTest Sequence Construction
Whole Picture II
Software under Test Model of Behavior Testing Model
Coverage Model
Operation
Data
Event
Operation
Operation
pre post
pre post
Eventpre post
invariantsData model
Operation
StateCalculation
Scenario method
Scenario method
Tool Demo
Set of Integers – Scenario I
0
1 2
312
7
5
States of behavior model
3
5
States of FSM model
Mapping Abstract Call to Specific
1
2
3
current state
parameters
states
Set of Integers – Scenario II
0
1 2
312
7
5
States of FSM model = States of behavior model
Failure
{ -2147483648, 2147483647 }
Add ( -715827883 ) / false
References
1. V. Kuliamin, A. Petrenko, N. Pakoulin, I. Bourdonov, and A. Kossatchev. Integration of Functional and Timed Testing of Real-time and Concurrent Systems. Proc. of PSI 2003. LNCS, Springer-Verlag, 2003.
2. V. Kuliamin, A. Petrenko, I. Bourdonov, and A. Kossatchev. UniTesK Test Suite Architecture. Proc. of FME 2002. LNCS 2391, pp. 77-88, Springer-Verlag, 2002.
3. A. K. Petrenko, I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Experiences in using testing tools and technology in real-life applications. Proceedings of SETT’01, India, Pune, 2001
4. I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Using Finite State Machines in Program Testing. "Programmirovanije", 2000, No. 2 (in Russian). Programming and Computer Software, Vol. 26, No. 2, 2000, pp. 61-73 (English version)
5. I. Bourdonov, A. Kossatchev, A. Petrenko, and D. Galter. KVEST: Automated Generation of Test Suites from Formal Specifications. Proceedings of World Congress of Formal Methods, Toulouse, France, LNCS, No. 1708, 1999, pp. 608-621
6. http://www.ispras.ru/groups/rv/rv.html
Contact
Victor V. Kuliamin
E-mail: kuliamin@ispras.ru
109004, B. Kommunisticheskaya, 25
Moscow, Russia
Web: http://www.ispras.ru/groups/rv/rv.html
Phone: 007-095-9125317
Fax: 007-095-9121524
Thank you!