Post on 04-Jul-2020
transcript
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Tim Edwards
Senior Consultant, CAV Technologies. HORIBA MIRA
Verification and validation challenges for Connected Autonomous Vehicles (CAVs)
Verification Futures 2019
13th June 2019. Reading, UK
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
What is an Autonomous Vehicle?
2
SAE J3016
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
What is a Connected Autonomous Vehicle (CAV)?
• Real-time information
• e.g. Rich mapsConnect
• Peer-to-peer communication & negotiation
• e.g. Traffic signal prioritisation
• e.g. C-ACC / platooning
Cooperate
• External management intelligence
• e.g. Remote automated parking
• e.g. Fleet supervision
Control
3
Requirements: Integrity, latency, availability
■ Different definitions have very different requirements and suit different technologies
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Why develop CAVs?
4
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Jekyll & Hyde
5
What is all this fuss
about CAV verification?
Don’t worry, we can
apply existing practices
from automotive and
other sectors
Ahh no… this is
unprecedented we need
completely new tools and
approaches
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Challenges of autonomy
Complexity
Integrity
CAV
Diversity
6
• Complex distributed software
• High availability requirement
• Introduction of Machine Learning
• Wireless communications and external dependencies
• Scenario definition & parameterisation
• Road networks
• Weather
• National / international variations
• Other road users
• Acceptance criteria
• Test coverage
• Cost / time / recursion
• High severity failure modes
• Consideration of Safety of the
Intended Functionality (SOTIF)
• Misuse cases
• Maintenance & in-service support
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Systems engineering lifecycle
7
Transport System
Requirements
Vehicle
Requirements
AD System
Requirements
AD System
Verification
Full Vehicle
Verification
Operational
System
Linking both sides
using advanced
verification tools
and DVPs
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Scenario based testing
■ Scenarios are a mechanism for mapping
use cases or requirements to test cases
■ Operational Design Domain (ODD) bounds
the intended scope of the system operation
and informs the environmental
requirements
■ Functional decomposition is recommended
to constrain and target the scenarios
8
Requirements Scenarios TestCases
User
requirements
ODD
Behavioural requirements
Environmental requirements
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Test distribution
9
Test Gaps
We can’t test all the requirements at this level:
■ Test equipment not capable
■ System integration level
Correlation
Some element of the system or environment was
approximated - we devise tests to carry out at later
stages to seek correlation
PG
HIL
SIL
Motivation:
(1) Test early (Reduce rework cost) (2) Test virtually (Reduce test cost)
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Integrated CAV Test Eco-System
10
■ Public environments
■ Virtual environments
■ Controlled environments
■ Cyber-physical environments
SaM
UKCITE & Midlands
Future Mobility
TIC-IT
Off Highway Driving
City Circuit
Park-IT
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Public environment
*N. Kalra, S. Paddock; “Driving to Safety. How Many Miles of Driving Would
It Take to Demonstrate Autonomous Vehicle Reliability?”, RAND (2016) 11
■ Capture real scenarios and failure modes
- Particularly for perception systems
■ Highly automated driving trials (SAE L4)
already live – under UK Code of Practice
■ Not a viable means for safety assessment
- Billions of miles to make a statistical
argument about system failure rates*
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Virtual
12
■ Required to address the volume of test scenarios
■ Accuracy and realism of models can be a limiting factor
■ Different levels of abstraction for different test activities
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Controlled environments
■ Characterise models, and demonstrate correlation of simulation results
■ Validate system integration and demonstrate functionality in controlled conditions
13
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Proving ground developments (Park-IT and TIC-IT)
14
Urban, interurban, highway and controllability scenarios
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019 15
Proving ground test automation
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019 16
Proving ground test automation
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
A hybrid approach: Vehicle-in-the-loop
17
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
Acknowledgements
18
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019 19
Summary
■ V&V forms part of a robust, requirements driven, systems engineering methodology
■ Challenging to adequately define scenarios that reflect the real-world
- … and to define acceptable levels for test coverage
■ All test activities have limitations and dependencies, which need to be addressed collectively
- Simulation is critical - only practical way to address scale - but still just one piece of the puzzle
- Development of test tools, assets, and facilities to exercise CAV technologies at component,
system and vehicle level – in simulation, laboratory and proving ground conditions
■ Here to engage with experts in other sectors and learn from previous experiences
© HORIBA MIRA Ltd. 2019© HORIBA MIRA Ltd. 2019
HORIBA MIRA Ltd.
Watling Street,
Nuneaton, Warwickshire,
CV10 0TU, UK
T: +44 (0)24 7635 5000
F: +44 (0)24 7635 8000
www.horiba-mira.com
Firstname SecondnameQualifications / Affiliations
Job Title
Direct T: +44 (0)24 7635 5xxx
M: +44 (0)7xxx xxxxxx
E: firstname.secondname@horiba-mira.com
Contact Details
20
HORIBA MIRA Ltd.
Watling Street,
Nuneaton, Warwickshire,
CV10 0TU, UK
T: +44 (0)24 7635 5000
F: +44 (0)24 7635 8000
www.horiba-mira.com
Tim EdwardsBEng (Hons), MPhil, CEng, MIET
Senior Consultant
Connected and Autonomous Vehicles
Direct T: +44 (0)24 7635 5484
M: +44 (0)7787 280164
E: tim.edwards@horiba-mira.com