Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | ronald-gillman |
View: | 213 times |
Download: | 0 times |
Technology ofTechnology ofTest Case GenerationTest Case Generation
Levi LúcioUniversity of Geneva
Marko SamerVienna University of Technology
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
OverviewOverview
Introduction
Model Checking
Symbolic Execution
Theorem Proving
Conclusions
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
IntroductionIntroduction
• In this presentation three technologies for test case generation are introduced;
– Symbolic Execution;– Theorem Proving;– Model Checking (presentation not available);
• All the models used by these methods can be reduced to state machines;
• Test case generation can be seen as state space exploration for traces obeying a criteria.
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
OverviewOverview
Introduction
Model Checking
Symbolic Execution
Theorem Proving
Conclusions
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
OverviewOverview
Introduction
Model Checking
Symbolic Execution
Theorem Proving
Conclusions
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Symbolic ExecutionSymbolic ExecutionTalk OverviewTalk Overview
• The technique– How symbolic execution works;
– Issues related to the approach;
– Symbolic execution of abstract models;
• Test case generation– A classification of test case generation
frameworks using symbolic execution;
– Abstract model based test case generation;
– Code based test case generation.
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Symbolic ExecutionSymbolic Execution
• Technique invented in the 1970s for verifying the consistency of code;
• The main principle consists of executing the code with symbolic inputs…
• … in which a symbolic input consists of replacing a « real input » (number, string, structure, object) with a symbol;
• For example, if the program asks a number to the user, « » would be passed to the program as a representation of all possible numbers.
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
The techniqueThe technique
• Allows exploring the possible control paths in an program. Calculates the path condition for each control path, composed of symbolic equations.
Int foo (int a, int b) {1 a++;2 if (a>b)3 a=a-b;4 else5 a=b-a;6 if (a<=-1)7 a=-a;8 return a }
Control paths for foo
1,2,3,6,7,8
1,2,3,6,8
1,2,5,6,7,8
1,2,5,6,8
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
The technique (cont)The technique (cont)a: b:PC: True
a: +1b:PC: True
(1)
a: b:PC:
(3)
(8)
a: b:PC:
a: b:PC:
a: b:PC:
(2 -true) (2 - false)
…
a: b:PC:
a: b:PC:
(6 - true) (6 - false)
No solution for PC equations:Infeasible Control Path
(search backtracks)
Int foo (int a, int b) {1 a++;2 if (a>b)3 a=a-b;4 else5 a=b-a;6 if (a<=-1)7 a=-a;8 return a }
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
The technique (cont)The technique (cont)
• From the symbolic execution of foo, one can retrieve the following information:
Control Path Path Condition
1,2,3,6,7,8 Not feasible
1,2,3,6,8
1,2,5,6,7,8 Not feasible
1,2,5,6,8
• Each path condition fully documents the decisions taken during the execution of the code.
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Test case generationTest case generation
• It is possible to generate test cases by finding values that satisfy the path condition equations:
Control Path Path Condition Example test case
1,2,5,6,8
• Test case (a=1, b=3) would force the execution of foo to follow the control path 1,2,5,6,8;
• Test cases are thus created as by-products of the activity of symbolic execution;
• Symbolic execution uncovered that in the case of foo instruction 7 is never reached...
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Issues related to the approachIssues related to the approach
• Dealing with loops
– Loops generate infinite control paths (unless the number of iterations is known). Solution?
• Prompt the user at each loop iteration;• Establish an upper bound for the number of iterations;• Try to automatically find a fixed point to the loop (costly).
• Solving path condition equations
– Necessary for: • understanding when a path is infeasible;• generating test cases.
– Algorithms heavy on computing resources.
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Symbolic execution of abstract Symbolic execution of abstract modelsmodels
• Up until now we have considered symbolic execution only as a “white box” verification technique…
• However, nothing prevents from applying the same technique to an abstract model (state machine)
CodeAbstract Model(state machine)
Symbolic Execution
Test Cases
• State space of a model can be symbolically searched for interesting control paths;
• Symbolic execution reduces state space explosion by associating classes of inputs.
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Symbolic execution for test case Symbolic execution for test case generationgeneration
• In the text several frameworks for test case generation from different models of the application are studied;
• We classify the frameworks according to two axis:
Application model
Synergies with otherverification techniques
Model Checking
Theorem proving
Abstract (B, AUTOFOCUS, CO-OPN), Code (Java)
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Abstract model based test case Abstract model based test case generationgeneration
• In the text three frameworks are described that have their starting point on different abstract models:– B (framework from Legeard, Peureux et al)– AUTOFOCUS (framework from Pretschner et al)– CO-OPN (framework from Buchs et al)
• The models can be considered as state machines and the technique for test case generation is similar:
Abstract Model(state machine)
LogicProgramming
ModelTest Cases
Translationstep
Symbolicstate space search
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Abstract model based test case Abstract model based test case generation (cont)generation (cont)
• The step of translation into a logic programming language (theorem prover) is necessary to:
– Animate the (static) abstract specification;
– Search the state space;
– Perform symbolic execution.
• Two kinds of logic programming languages may be used:
– Pure Prolog;
– Constraint Logic Programming (CLP) Languages: enabled to deal with numeric constraints at the unification level.
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Code based test case generationCode based test case generation
• Why take a detour from model based test case generation (title of the section)?
• In the text we present a framework from Khurshid et al that generates test cases from Java code. It uses:
– A model checker (Java PathFinder) to overcome some difficulties in symbolic execution;
– The symbolic execution technique to overcome some difficulties in model checking.
• We think an insight on the framework is enriching for the discussion…
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Test generation from codeTest generation from code
ExampleExample
• Symbolic execution helps dealing with state space explosion by dealing with inputs as classes of values;
• Model checking deals automatically with loops, recursion or method invocation…
ModelChecker
LogicFormulas
StateSpace
WitnessTrace
Counter-exampleTrace
Formulas hold Formulas don’t hold
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
Test generation from codeTest generation from code
Example (cont)Example (cont)
• The code is instrumented to deal with symbolic values and accumulate path conditions (via Java libraries):
Java CodeInstrumented
Java codeTest case
traces
Java SymbolicExecution libraries
Modelchecking
• The algorithm that performs symbolic execution knows how to deal with complex dynamic structures (objects)!
• A phase of test case instantiation is required…
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
OverviewOverview
Introduction
Model Checking
Symbolic Execution
Theorem Proving
Conclusions
H. S. Hong, L. Lúcio, M. Samer: Technology of Test Case Generation
OverviewOverview
Introduction
Model Checking
Symbolic Execution
Theorem Proving
Conclusions