www.pegasusprojekt.de TEST CASE GENERATION, TEST SPACE COVERAGE AND TEST EFFORT REDUCTION Basics for Testing – Stand 13 Coupling Environment Simulation and Database to Test Automation for Generation, Execution and Evaluation of Concrete Scenarios with Representative Coverage of the Test Space Test Automation • Interpretation of logical scenarios • Application of parameter-space specifications • Control of environment simulation Parameter Variation • Combinatorial Methods • Evolutionary Methods Test Automation Test Derivation Stochastics Module Test Control Scenario Database * Logical Scenario * Parameter Space * Evaluation Criteria Simulation Tool Models Driving function Sensor models Overall Results Result Evaluation Concrete Scenario OSI or OEM Connection OSI or OEM Connection Test Result Result Evaluation • Computation of quality criteria such as criticality • Presentation of test results and test space coverage Contribution to Proof of Safety • Identification of critical scenarios • Representative coverage of the entire test space, statistically and with respect to the test concept Overview
Transcript
www.pegasusprojekt.de
TEST CASE GENERATION, TEST SPACE COVERAGEAND TEST EFFORT REDUCTION
Basics for Testing – Stand 13
Coupling Environment Simulation and Database to Test Automation for Generation, Execution and Evaluation of Concrete Scenarios with Representative Coverage of the Test Space
Test Automation• Interpretation of logical scenarios• Application of parameter-space
Result Evaluation• Computation of quality criteria such
as criticality• Presentation of test results and test
space coverage
Contribution to Proof of Safety• Identification of critical scenarios• Representative coverage of the entire
test space, statistically and with respect to the test concept
Overview
www.pegasusprojekt.de
TEST CASE GENERATION, TEST SPACE COVERAGEAND TEST EFFORT REDUCTION
Basics for Testing – Stand 13
Coupling Environment Simulation and Database to Test Automation for Generation, Execution and Evaluation of Concrete Scenarios with Representative Coverage of the Test Space
The PEGASUS Database provides a parameterizablelogical scenario and thecorresponding parameterspace.Additionally it includes thecriticality metric for resultevaluation (e.g. TimeToCollision).
QTronic TestWeaver, TraceTronic ECU-TEST or IfRStochastics Module takes a parameter set from theparameter space and generatesa concrete scenario in theformat OpenSCENARIO. Theparameter selection can utilizethe results from previoussimulations (reactive process).
The concrete scenario isexecuted via simulation usingIPG CarMaker or dSPACEModelDesk. For evaluation, thesimulation result is fed back into TestWeaver or ECU-TEST.
Logical Scenario & Parameter Space Metrics
Parameter Set Result Evaluation
OpenSCENARIO Simulation Result
OpenSCENARIO
Internal Data Model
Simulation Result
Simulation
Database
Test Automation
www.pegasusprojekt.de
TEST CASE GENERATION, TEST SPACE COVERAGEAND TEST EFFORT REDUCTION
Basics for Testing – Stand 13
Coupling Environment Simulation and Database to Test Automation for Generation, Execution and Evaluation of Concrete Scenarios with Representative Coverage of the Test Space
Parameter Variation
The parameter space of a logical scenario cannot be tested exhaustively. Thus, for the purpose of testing, a representative sample with respect to the defined application area of the driving function has to be derived.
During the first half of the project, the stochastics modules focused on the definition of interfaces and the conversion of formats. Currently, the stochastics modules provide the following functionality:• Import of logical scenarios• Visualization of modeled
distributions• User interface for manual
selection of discretization levels
• Algorithm for combining discretization levels
• Combinatorial and evoluationary methods for deriving optimal value combinations wrt. given metrics
• Export of concrete scenarios in OpenSCENARIO format.
Using the stochastics modules, logical scenarios are imported, parameter sets to be tested are derived from probabilities of occurence p and the resulting concrete scenarios are exported in OpenSCENARIO format.
Scenario Concretization
Speed Limit
Velocity Actor
Lane Widthp
Initial Distance Ego-Actor
Sheer-off Point in Time
p
p
p
p
2,5 m 3,75 m
80 km/h 130 km/h
60 km/h 140 km/h
30 m 150 m
1 s 10 s
Concrete Scenario 1Concrete Scenario 2
Concrete Scenario 3Parameters:
Logical Scenario
Concrete Scenarios
Import of a Logical Scenario
Export of concrete scenarios
www.pegasusprojekt.de
TEST CASE GENERATION, TEST SPACE COVERAGEAND TEST EFFORT REDUCTION
Basics for Testing – Stand 13
Coupling Environment Simulation and Database to Test Automation for Generation, Execution and Evaluation of Concrete Scenarios with Representative Coverage of the Test Space
Result presentationusing 2D heatmaps
Detailed analysis of selected scenarios using signal plots
Criticality of concrete scenarios with respect to multiple parameters. The colors range from green (uncritical) to red (critical).
