Framework for Constructing Context-Specific Migration Methods for Test Cases
Presented by Ivan Jovanovikj
© All rights reserved
2 © All rights reserved
1 Challenges in Test Case Reusein Migration Projects
2 Construction of Context-Specific Migration Methods
Agenda
3 Outlook and Conclusion
Problem Statement
4 © All rights reserved
SystemMigratedSystem
test
Restructuring
ReverseEngineering
ForwardEngineering
MigratedModel
Model
testTest CasesTest Cases
Test the desired functionality of
the source system
Test Case Designis costly and time
consuming
Test Requirements must be considered
Changes from thesoftware migrationmust be considered
Problem StatementHow to systematically
reuse existing test cases in a software migration scenario?
SoftwareMigration
Solution Approach
5 © All rights reserved
Solution ApproachDevelopment of aContext-Specific
Test Case Migration Method
In this talk:Framework for
Constructing Context-Specific Migration Methods
for Test Cases
SystemMigratedSystem
test testMigratedTest Cases
Test Cases
SoftwareMigration
Approach Overview
7 © All rights reserved
Migration Context
Characterization
Tool Implementation
Migration Method Tailoring
Migration Method
Enactment
ReferenceMigrationMethod
ContextModel
AdaptedMigrationMethod
Context-specific
Tools
Migration Validation
Pre-Migration Migration Post-Migration
Migration Context Characterization
8 © All rights reserved
System Migration Context
Testing Context
1. What was migrated?
2. How was it migrated?
3. What is the current state of the test cases?
4. What are the testing requirements?
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Migration Context Factors
9 © All rights reserved
Migration ContextFactors
Source Environment
Characteristics
Language
Framework
Architecture
TransformationCharacteristics
Degree of Automation
Degree ofFormality
Process Support
Change Type
Target Environment
Characteristics
Language
Framework
Architecture
Source Testing Environment
Characteristics
Testing Framework
Test Case Type
Test Case Anatomy
Test Case Quality
Target Testing Environment
Characteristics
Testing Framework
Test Case Type
Test Case Anatomy
Test Case Quality
System MigrationContext Factors
TestingContext Factors
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Migration Context Characterization
Eclipse Modeling Project
System Migration Context - Case Study
10 © All rights reserved
Modelling Framework
Ahead-of-TimeCompilation
(AOT)
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Migration Context Characterization
&
Eclipse Modelling Framework
EMF.CodeGen
EMF EcoreEMF.Edit
Model Development Tools
BPMN2
MoDisco
…
OCL
Just-in-TimeCompilation
(JIT)
ecore
OCL
Testing Context - Case Study
11 © All rights reserved
OCL Test Framework
Testing Framework(JUnit)
Language(Java)
Test Cases
OCL Test Framework
Testing Framework(NUnit)
Language(C#)
Test Cases
Source Testing EnvironmentCharacteristics
Target Testing EnvironmentCharacteristics
1Migration of
TestOCL
2Migration of
Test Cases
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Migration Context Characterization
Reference Migration Method
12 © All rights reserved
Language Metamodel
Test Framework Metamodel
UTP Meta-Model
instance of
instance of
instance of
Textual Artifact
Model
Model to Model Transformation
Text to Text Transformation
Text to Model / Model to Text
Transformation
Exogeneous Tranformation
Endogeneous Tranformation
Activities
Artifacts
MetaModel
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
ModelDiscovery
CodeGeneration
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
TestAbstraction
TestConcretization
Restructuring
Platform SpecificTest Model(Migrated)
Platform IndependentTest Model
Model of the Code
Language Transformation
Model of the Migrated Code
Test CaseUnderstanding
TestConcretization
FrameworkTransformation
MigratedTestCode
Reductive Tailoring
Method Tailoring
13 © All rights reserved
Remove Activity
Remove Artifact
Migration Context Characterization
Tool Implementation
Migration Method Enactment
Migration Validation
Migration Method Tailoring
ModelDiscovery
CodeGeneration
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
TestAbstraction
TestConcretization
Restructuring
Platform SpecificTest Model(Migrated)
Platform IndependentTest Model
Model of the Code
Language Transformation
Model of the Migrated Code
Test CaseUnderstanding
TestConcretization
FrameworkTransformation
MigratedTestCode
Method Tailoring
14 © All rights reserved
…
Context C4 specific Context C5 specific Context C6 specific
Context C8 specific Context C9 specific Context Cn specificContext C10 specific
Tailoring
Migration Context Characterization
Tool Implementation
Migration Method Enactment
Migration Validation
Migration Method Tailoring
Context C1 specific Context C2 specific Context C3 specific
Context C7 specific
ModelDiscovery
CodeGeneration
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
TestAbstraction
TestConcretization
Restructuring
Platform SpecificTest Model(Migrated)
Platform IndependentTest Model
Model of the Code
Language Transformation
Model of the Migrated Code
Test CaseUnderstanding
TestConcretization
FrameworkTransformation
MigratedTestCode
Method TailoringTest Case Migration Method
15 © All rights reserved
Remove Activity
Remove Artifact
Reductive Tailoring
Migration Context Characterization
Tool Implementation
Migration Method Enactment
Migration Validation
Migration Method Tailoring
ModelDiscovery
CodeGeneration
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
TestAbstraction
TestConcretization
Restructuring
Platform SpecificTest Model(Migrated)
Platform IndependentTest Model
Model of the Code
Language Transformation
Model of the Migrated Code
Test CaseUnderstanding
TestConcretization
FrameworkTransformation
MigratedTestCode
Method Tailoring: Test Case Migration Method
16 © All rights reserved
Remove Activity
Remove Artifact
Reductive Tailoring
Migration Context Characterization
Tool Implementation
Migration Method Enactment
Migration Validation
Migration Method Tailoring
Code
Generation
TestConcretization
+
ModelDiscovery
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
Platform SpecificTest Model(Migrated)
Model of the Code
Test CaseUnderstanding
FrameworkTransformation
MigratedTestCode
Method Tailoring: TestOCL Migration Method
17 © All rights reserved
Remove Activity
Remove Artifact
Reductive Tailoring
Migration Context Characterization
Tool Implementation
Migration Method Enactment
Migration Validation
Migration Method Tailoring
ModelDiscovery
CodeGeneration
Platform SpecificTest Model
(Source)
TestOCLCode
Reimplementation
TestAbstraction
TestConcretization
Restructuring
Platform SpecificTest Model(Migrated)
Platform IndependentTest Model
Model of the Code
Language Transformation
Model of the Migrated Code
Test CaseUnderstanding
TestConcretization
FrameworkTransformation
MigratedTestOCL
Code
Method Tailoring: TestOCL Migration Method
18 © All rights reserved
Remove Activity
Remove Artifact
Reductive Tailoring
Migration Context Characterization
Tool Implementation
Migration Method Enactment
Migration Validation
Migration Method Tailoring
TestOCLCode
Reimplementation MigratedTestOCL
Code
Method Tailoring
19 © All rights reserved
2
1 TestOCLMigration Method
Test CaseMigration Method
Migration Context Characterization
Tool Implementation
Migration Method Enactment
Migration Validation
Migration Method Tailoring
TestOCLCode
Reimplementation MigratedTestOCL
Code
Code
Generation
TestConcretization
+
ModelDiscovery
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
Platform SpecificTest Model(Migrated)
Model of the Code
Test CaseUnderstanding
FrameworkTransformation
MigratedTestCode
Tool Implementation
20 © All rights reserved
TestCase2TestModel
plugin
JDT Core
TestModel2TesCase
plugin
supports
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Code
Generation
TestConcretization
+
ModelDiscovery
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
Platform SpecificTest Model(Migrated)
Model of the Code
Test CaseUnderstanding
FrameworkTransformation
MigratedTestCode
Migration Method Enactment
21 © All rights reserved
ToolsMethod Specification
2 Test Case Migration
1 TestOCL Migration
TestOCLCode
Reimplementation MigratedTestOCL
Code
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
CodeGeneration
TestConcretization
+
ModelDiscovery
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
Platform SpecificTest Model(Migrated)
Model of the Code
Test CaseUnderstanding
FrameworkTransformation
MigratedTestCode
TestCase2TestModel
plugin
TestModel2TesCase
plugin
Migration Method Enactment
22 © All rights reserved
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Code
Generation
TestConcretization
+
ModelDiscovery
Platform SpecificTest Model
(Source)
TestCode
Reimplementation
Platform SpecificTest Model(Migrated)
Model of the Code
Test CaseUnderstanding
FrameworkTransformation
MigratedTestCode
TestCase2TestModel
plugin
JDT Core
TestModel2TesCase
plugin
AST
ModelDiscovery
Test CaseUnderstanding
Migration Method Enactment
23 © All rights reserved
xUnit model xUnit model
Reimplementation
Code
Generation
TestConcretization
+
FrameworkTransformation
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
TestCase2TestModel
plugin
JDT Core
TestModel2TesCase
plugin
Test CaseUnderstanding
ModelDiscovery
AST
Migration Method Enactment
24 © All rights reserved
TestModel2TesCase
TestCase2TestModel
plugin
JDT Core
plugin
xUnit model xUnit model
Reimplementation
Code
Generation
TestConcretization
+
FrameworkTransformation
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Test CaseUnderstanding
xUnit model
Migration Method Enactment
25 © All rights reserved
TestModel2TesCase
TestCase2TestModel
plugin
JDT Core
plugin
xUnit model
Reimplementation
Code
Generation
TestConcretization
+
ModelDiscovery
FrameworkTransformation
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
AST
Test CaseUnderstanding
Migration Method Enactment
26 © All rights reserved
TestModel2TesCase
TestCase2TestModel
plugin
JDT Core
plugin
xUnit model
Reimplementation
Code
Generation
TestConcretization
+
ModelDiscovery
FrameworkTransformation
xUnit model
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
AST
Migration Method Enactment
27 © All rights reserved
TestModel2TesCase
TestCase2TestModel
plugin
JDT Core
plugin
xUnit model
xUnit model
Reimplementation
Code
Generation
TestConcretization
+
ModelDiscovery
Test CaseUnderstanding
FrameworkTransformation
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
AST
Migration Method Enactment
28 © All rights reserved
TestModel2TesCase
TestCase2TestModel
plugin
JDT Core
plugin
xUnit model xUnit model
Reimplementation
Code
Generation
TestConcretization
+
ModelDiscovery
Test CaseUnderstanding
FrameworkTransformation
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
AST
Migration Validation
29 © All rights reserved
How to validate a test case migration?
What is a valid test case migration?
Test case migration is a process of transferring test cases into new environmentswithout changing their functionality, i.e., without changing what they test.
How to ensure behavioral equivalence in test case migration ?Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Migration Validation
30 © All rights reserved
TestReport
Migration
Migrated systemworks as expectedErrors in the
migrated system
TestCases
MigratedTest
Cases
Execution
PassedTest
Cases(negatives)
FailedTest
Cases(positives) False
positives
Truepositives
Falsenegatives
Truenegatives
Problem: Detects non-existing errorsConsequence: Waste of time by looking for non-existing error
Problem: Hides potential errors Consequence: System malfunctioning/crashes
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Migration Validation
31 © All rights reserved
How to avoid/detect false positives and false negatives?
How to ensure behavioral equivalence in test case migration?
Constructive approaches
Analytical Approaches
Bi-simulation Mutation Testing
Cost ofTest Case Migration
Cost ofValidation of Test Case Migration
Cost ofDeveloping New
Test Cases+ <=
Bottom Line
Migration Context Characterization
Tool Implementation
Migration Method Tailoring
Migration Method Enactment
Migration Validation
Overview and Conclusion
33 © All rights reserved
Migration Domain
Conceptualsolution
Comprehensive discussion on the
practical application
SystemMigratedSystem
test testMigratedTest Cases
Test Cases
Software Migration
Test the desiredfunctionality of
the source system
Test Case Designis costly and
time consuming
Changes from thesoftware migrationmust be considered
Problem StatementHow to systematically reuse
existing test cases in a software migration scenario?
Test Case Migration
Test Requirements must be considered
MigrationContext
Characterisation
ToolImplementation
MigrationMethodTailoring
MigrationMethod
Enactment
ReferenceMigrationMethod
ContextModel
AdaptedMigrationMethod
Context-specificTools
MigrationValidation
Pre-Migration Migration Post-Migration
plugin
Tools
TestModel2TesCase
TestCase2TestModel
plugin
Method Specification
ModelDiscovery
CodeGeneration
Platform SpecificTest Model
(Source)
TestCode
Reimplementation MigratedTestCode
Platform SpecificTest Model(Migrated)
Model of the Code
Test CaseUnderstanding Test
Concretization
FrameworkTransformation
+
Eclipse Modeling Project
Model Development ToolsEclipse Modelling Framework
EMF.CodeGen
EMF Ecore EMF.Edit
BPMN2
MoDisco
…
Modelling Framework
ecore
OCL
Just-in-TimeCompilation
(JIT)
Ahead-of-TimeCompilation
(AOT)
OCL
&
System Migration Context
OCL Test Framework
Testing Framework(JUnit)
Language(Java)
Test Cases
OCL Test Framework
Testing Framework(NUnit)
Language(C#)
Test Cases
Source Testing EnvironmentCharacteristics
Target Testing EnvironmentCharacteristics
2
1
Testing Context
34 © All rights reserved
Thank you for your attention
s-lab – Software Quality Lab
Paderborn University
Zukunftsmeile 1
33102 Paderborn
Ivan Jovanovikj
Tel.: (05251) 54 65-216
http://s-lab.upb.de