Post on 22-May-2018
transcript
Jack Weast
Principal Engineer, Chief Systems Engineer
Automated Driving Group, Intel
2
From the Intel Newsroom…
3
Levels of Automated Driving
Courtesy SAE InternationalRef: J3061
Data Center
4
Captured Sensor Data Analytics
In-Vehicle
Sensor Processing
Automated Driving Functions
Vehicle Simulation & Validation
Simplified End-to-End Automated Vehicle Architecture
Sensor Capture
Data
Vehicle Endpoint Management
What Makes an Automated Vehicle Work?
SensingSensor Fusion
Environment Modeling
HD-Map Correlation
Sensor Processing
ManeuveringPath
PlanningTrajectory
EnumerationLocalization
Decision Making
What is Happening in the Data Center?
Development of Automated Vehicles *starts* in the Data Center
There you store Data and Lots of it!!!
The first step towards developing your algorithms for autonomy
Example Data: Data
• Raw Sensor Data• Vehicle Bus Traffic• Environmental Conditions• Scenario Under Capture• Driver Name / Date / Time• More…
How Does the Data get There?
Test Fleet Vehicles produce an incredible amount of data
PetaBytes per hour!
Physical Ingestion is really the only option
Commercially Deployed Vehicles produce all the same amount of data
But it is used inside the vehicle
The trick is how to know when to capture “interesting” data for later cloud processing
Sneaker Net!
Intelligent AnomalyDetection
What else is Happening in the Data Center?
All of that uploaded data is used to trainour Deep Neural Networks
DNN’s support key use cases like
Pedestrian Detection
Traffic Sign Detection
Visual Object Detection
And, emerging use cases like
Driving Policy
Captured Sensor Data Analytics
Model TrainingMulti-node / IA-optimized Frameworks
Neural Network Design for Target Hardware, and usage
(Vision, Data Driven, etc.)
Big Data Analytics
Statistical Trends
Intel® Architecture (IA)
Simulation>than Real Time
Model Simulation & Verification
What is else is Happening in the Data Center?
All of that uploaded data is also used for ‘big data’ style statistical analysis
Statistical Analysis can provide:
Longitudinal usage patterns
Insight into vehicle performance
Personalization
The real value here is the intersection of Big Data and Deep Learning
Captured Sensor Data Analytics
Model TrainingMulti-node / IA-optimized Frameworks
Neural Network Design for Target Hardware, and usage
(Vision, Data Driven, etc.)
Big Data Analytics
Statistical Trends
Intel® Architecture (IA)
Simulation>than Real Time
Model Simulation & Verification
What else is Happening in the Data Center?
But how do I know if the algorithm works?
You need to test it on new, labeled data that was not previously used for training
Simulation >> faster than Real Time mechanism to test the proposed vehicle implementation
Crucial to do before deploying to the real world!
Simulation>than Real Time
Model Simulation & Verification
Intel® Architecture (IA)
Captured Sensor Data Analytics
Model TrainingMulti-node / IA-optimized Frameworks
Neural Network Design for Target Hardware, and usage
(Vision, Data Driven, etc.)
Big Data Analytics
Statistical Trends
Introducing Intel® Deep Learning SDKIntel DL Deployment Tool
• IMPORT Trained Model (trained on Intel or 3rd Party HW)
• COMPRESS Model for Scoring on Target Intel HW
• GENERATE Scoring HW-Specific Code (OpenCL*, C/C++)
• INTEGRATE with System SW / Application Stack & TUNE
• EVALUATE Results and ITERATE
configure_nn(fpga/cve,…)
allocate_buffer(…)
fpga_conv(input,output);
fpga_conv(…);
mkl_SoftMax(…);
mkl_SoftMax(…);
…
Optimized libraries & run-times (MKL-DNN, OpenVX, OpenCL)Data acquisition (sensors) and acceleration HW
Target Scoring Hardware Platform (physical or simulated)
MKL-DNN Optimized Machine Learning Frameworks
Intel DL Training Tool
• INSTALL / SELECT IA-Optimized Frameworks
• PREPARE / CREATE Dataset with Ground-truth
• DESIGN / TRAIN Model(s) with IA-Opt. Hyper-Parameters
• MONITOR Training Progress across Candidate Models
• EVALUATE Results and ITERATE
Intel® Xeon® Workstation or Cluster (local or cloud)
Intel® Architecture (IA)https://software.intel.com/en-us/deep-learning-sdk
Endpoint Management & Deployment
Connected Vehicles represent a significant Endpoint Management Challenge!
Our Goal is to deploy early and often, over the air, updated models and other driving policy preferences
Model Compression is the process of reducing the size (layers) of the network while retaining accuracy
Other methods could include some amount of re-training within the vehicle…
Vehicle EndpointManagement
Compression
Putting it All together: An End-to-End Architecture
Data Storage
Dataset Management
and Traceability
Data Center
13
Model TrainingMulti-nodeoptimized
Frameworks
Neural Network / Algorithm Design
In-Vehicle
Real Time Environment ModelingLocalization
Sensor Processing and FusionObject ID & Classification
Automated Driving FunctionsTrajectory Enumeration, Path
Planning, Selection & ManeuveringDriving Policy, Path Selection
CompressedDL Model
Captured Sensor Data
Real Time HD Map Updates
Data Formatting, Annotation
Annotation Tools
OTA SW /FW Updates
5G
Simulation>than Real Time
Model Simulation & Verification
An
om
aly
De
tectio
n
Endpoint ManagementGeographical Tracking, OTA Updates
Sneaker Net – Test Fleets
Big Data and Statistical Analytics
Introducing Intel® GO™ Automated Driving Solution
15
But what does all this mean for the In-Vehicle System Architecture?
Key Challenges in Enabling Autonomous Vehicles
Computational Performance and Power Efficiency
Data Center Compute in a limited power envelope
Real-time processing for vehicle control / collision avoidance
Safety and Securityfor increased levels of automation
CHALLENGES
Flexible & ScalablePlatform that scales across vehicle models from standard to luxury (price & performance)
17
The ROI Conundrum for Developing Custom Logic
1 2 3 4 5 6 7 8 9 10
$175$262
$439
$782
$1,605
* Development Cost Source: IBS
$ Millions
65nm 40nm 28nm 20nm 14nm
Revenue ROI Requiredto maintain R&D costs at 20% of Revenue
$35 $52 $88$156
$321
IC Development Costs*:IC Hardware
Software
Validation
18
FPGAs are Multi-function Accelerator Platforms
• Hardware accelerators provide breakthrough compute performance
• Re-configurable to adapt to a wide variety of workloads
• Performance-per-Watt gains with highly paralleled architecture
CPU
FPGA
Pre-distortion
Machine Learning
Sensor Fusion
Pedestrian Detection
Ra
dio
Au
tom
oti
ve
Da
ta C
en
ter
Accelerator
Accelerator
FPGA: Automated Driving Flexible Accelerator
Performance / Watt
Real-Time Processing
Flexible Accelerator
FPGA Value
FPGAAttributes
FPGAWorkload
Deep Learning
Grid Fusion
Path Planning
Sensor Fusion
Object Classification
DeterministicArchitecture
Hard FloatingPoint
EmbeddedMemory & DSP
ADHeterogeneous
Architecture
The MissingPuzzle Piece
Safety and SecuritySecurityArchitecture ∞
The Only Timeline That Matters Is When People Are Ready for Automated Driving
Trust means we feel:
• Safe
• Comfortable
• Confident
• “in control”
Without Trust, Automated Vehicle Adoption Won’t Happen
• Good listening
• Open dialogue
• Prompt action
How Do AVs Gain Our Trust?
Dr. Nicholas Epley
“The Mind in the
Machine” To the Car: A Name, Gender, and Voice.
So what are the key system elements of trust?
Visual, auditory, and motion sensors help the system “listen actively”—both inside and outside the vehicle.
SENSING
Touchscreens, voice controls, communication screens, and lighting give the vehicle multiple ways to communicate in multiple directions with passengers.
COMMUNICATING
Automated Vehicles must react quickly and effectively in response to driving conditions and routing requests.
RESPONDING TO CHANGES
• Key interactions and capabilities
• Technical implications
• Understanding what to optimize
We Are Building a
Trustworthy Platform.
Summary
Automated Driving is an End to End Architecture and Intel GO™ Automated Driving Solutions has you covered
Key Technical Challenges in Automated Vehicles can be solved through use of FPGAs as Flexible HW Accelerators for evolving workloads
Human afforded Trust is what will decide when we as humans are ready for automated vehicles
Incredibly scalable in-vehicle computing
• Wide range of reliable, available, and secure compute
• Collaboration with Wind River and others to deliver functionally safe OS, software, tools
• Layered security from chip to cloud with features rooted in hardware and support for secure over-the-air updates
Ideal combination of sequential and parallel computing
• Powerful and efficient Intel® Atom™ and Intel® Xeon® processors for sequential computing
• Arria® FPGAs for a powerful, cost-effective, scalable design platform
• Hardware acceleration technology for computer vision and machine/deep learning
• Suite of tools for automated driving software application developers
• Support for in-vehicle code and cloud development
• Machine learning/deep learning, computer vision algorithm development on Intel® architecture and accelerators
• Acceleration libraries, compilers, debuggers and IDE
Deep Learning, Computer Vision, and Sensor Fusion ToolsSensor Data Tool [NEW]
DL Training and Deployment ToolsIntel-optimized DL Frameworks
OpenVX* Kernel Library and Graph Builder
In-Vehicle Platform ToolsYocto* Recipe
Board Flashing Utility
FPGA & Heterogeneous Programming ToolsFPGA OpenCL
Full Stack Optimization Tools and LibrariesCompiler (ICC), JTAG debugger
Intel® Vtune™ Amplifier, Thread Check, Thread ProfilerPerformance and Threading Libraries (Intel® MKL, IPP, TBB)
• Industry’s first 5G-ready automotive platform
• Rapid development and testing of 5G solutions
• Target use cases:
• HD map downloads in real time
• HD content for in-vehicle infotainment
• Over-the-air updates
• Sensor uploads from vehicle, for machine learning
• Safety, smart intersections, cooperative driving
Introducing Intel® GO™ Automated Driving Solutions
Unmatched scalability and performance per power
Available 1H’17
1st 5G ready platform for automotive development
Available Feb 2017
Automotive Software Development Kit (SDK)
Intel Data Center Solutions
Automotive 5GPlatform
In-Vehicle Development
Platform for Automated Driving
>97% of servers deployed form machine learning workloads powered by Intel