Post on 18-Sep-2020
transcript
Author: Marita Andersson Date: 2018-02-12 Projekt inom Delprogrammets namn alt strategisk satsning exempelvis Trafiksäkerhet och automatiserade fordon
VPRP
Vehicle Positioning and Route Prediction
Public report
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 2
Contents
1 Summary ....................................................................................................... 3
2 Executive summary in Swedish .................................................................. 4
2.1 System design och verifiering .......................................................................................... 4
2.2 Lokal karta ........................................................................................................................ 4
2.3 Kartpositionering .............................................................................................................. 5
2.4 Kooperativ positionering ................................................................................................... 5
2.5 Global positionering ......................................................................................................... 6
2.6 Ruttprediktering ................................................................................................................ 6
2.7 Global kartuppbyggnad .................................................................................................... 6
3 Background .................................................................................................. 7
4 Purpose, research questions and method ................................................. 9
5 Objective ..................................................................................................... 11
6 Results and deliverables ........................................................................... 11
6.1 Result per work packages .............................................................................................. 11
6.2 FFI objectives and project contributions ........................................................................ 18
7 Dissemination and Publications ............................................................... 19
7.1 Knowledge and dissemination of results ........................................................................ 19
7.2 Publications .................................................................................................................... 19
8 Conclusions and future research ............................................................. 22
9 Participating parties and contact persons ............................................... 22
Kort om FFI
FFI är ett samarbete mellan staten och fordonsindustrin om att gemensamt finansiera forsknings- och
innovationsaktviteter med fokus på områdena Klimat & Miljö samt Trafiksäkerhet. Satsningen innebär verksamhet
för ca 1 miljard kr per år varav de offentliga medlen utgör drygt 400 Mkr.
För närvarande finns fem delprogram; Energi & Miljö, Trafiksäkerhet och automatiserade fordon, Elektronik,
mjukvara och kommunikation, Hållbar produktion och Effektiva och uppkopplade transportsystem. Läs mer på
www.vinnova.se/ffi.
http://www.vinnova.se/ffihttp://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 3
Summary
The VPRP is a project coordinated by Volvo Cars cooperation where Volvo Technology cooperation
and Chalmers are contributing parties. The project budget is 24,6 MSEK with finance of 12,3 MSEK
from Vinnova.
When moving from Advanced Driving Assistant System to more Autonomous systems, this puts a
higher requirement on accuracy for positioning and to move the car safely forward and prevent from
accidents.
This project aims at developing and verifying algorithms to accurately estimate the position of the ego
vehicle and predict the most likely route the vehicle will take in the road network. Insufficient
positioning accuracy is one of the factors which today prevent new safety and convenience systems
from being developed.
We have strived to answer questions like How can exact map position be obtained by matching the
information obtained from sensors mounted on the ego vehicle (e.g. radars and cameras) with a static
map? How the relative positioning accuracy between cooperative nodes can be improved by
communicating raw GNSS measurement data between the nodes? How the current position accuracy
of state of the art GNSS systems can be improved using e.g. more advanced GNSS receiver techniques
(carrier phase detection) or using position information from other sources (LTE, infrastructure)? Is it
possible to establish a driver route prediction based on earlier driver route statistics to enable reduced
fuel consumptions or similar solutions?
To answer the questions, we have used methods like exploit available information sources to create a
map and then developed algorithms to improve positioning and route prediction at present time.
New methods and algorithms for clustering driver destination is one examples of results during the
project time.
27 publications in combination with internal reports shows that we have succeeded with our target of
the project, to both achieve knowledge in positioning and route prediction system, but also develop
algorithms for predict future routes and methods to achieve high positioning of the ego vehicle,
something that is crucial for taking the step towards autonomous driving.
Better control of detailed traffic situations as for example lane-departure and reducing road transport
environmental impact by for example better optimisation of car control and route predicion, are some
examples of that we have achieved the FFI-program goals.
Beside of gained knowledge within the companies, the interest of positioning and route prediction has
also grown within Academia, which is shown by the majority of new courses.
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 4
1 Executive summary in Swedish
Nedan redovisas resultat från varje arbetspaket, med undantag från projektledning. Vissa av
leveranserna går in i flera arbetspaket, då till exempelvis karta och positionering i vissa metoder ligger
väldigt nära varandra.
1.1 System design och verifiering
Två generationer testbilar har utrustats med surround radarsensorer, lidarsensor, visionssensor
(kamera) och ett referenssystem av sensorer. Programvara för efterbehandling av dessa data har
utvecklats för att synkronisera den samlade datan och att beräkna en exakt
referenspositionsuppskattning. Referenspositionen har använts både för att bygga kartor och för att
utvärdera positioneringsprestandan. I detta arbetspaket har metoder för att utvärdera både kartan och
metoder för positionering tagits varm som används kontinuerligt på den insamlade datan. Arkitekturen
för första generationens bilar, ses på bild nedan [Figure 2].
WP3 Local map
1.2 Lokal karta
Inom VCC/Zenuity har man inom området tagit fram ett ramverk genom att sätta ihop
linjemarkeringars information från framåttittande kamera med objektpositioner från bakåttittande
radar som ett första steg. Som en förbättring för uppskattning av väggeometri har man också
inkluderat kartinformation vilket förlänger uppskattningshorisonten samt också utvecklat en metod för
att beräkna i vilken körbana värdbilen kör i [25].
För att stödja värdbilens nuvarande syn på trafiksituationen har akademien bidragit till att förbättra
placeringen av andra fordon och fotgängare genom att använda nya objektspårningsalgoritmer [1] - [4]
samt utvecklade grundläggande spårningsramar [5] - [8] och prestanda metriker för dessa scenarier
[9]. Några av dessa objekt är dynamiska (som fotgängare och bilar) [1] - [4] medan andra objekt är
statiska (t.ex. lampposter och trafikskyltar) och genom att placera alla statiska objekt får vi en karta
över vår miljö [16], [17].
Dessutom har metoder utvecklats för att beskriva vägen framför värdbilen, både med hjälp av radar
och kamera [10] - [12], men också med hjälp av lidarsensorer [13] skapat en djupinlärningsmetod för
Figure 2: Architecture for the first generation of test vehicles
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 5
att utföra vägdetektering där man från ett ostrukturerat punktmoln genererat toppbilder som kodar för
flera grundläggande statistiker, t.ex. genomsnittlig höjd och densitet. Genom att överväga en
toppbildsrepresentation reduceras vägdetektering till ett enkelskaligt problem som kan åtgärdas med
ett enkelt och snabbt fullt konvolutionellt neuralt nätverk (FCN).
I samband med väguppskattning har även akademien utvecklat algoritmer för att uppskatta vägens
form före ett värdfordon. I [10] - [12] är målet att noggrant beskriva väggeometrin (i
motorvägsscenarier) upp till 200 m framför ett värdfordon, som är utrustad med en kamera, en radar
och fordonsinterna sensorer. Detta syfte uppnås genom att härleda en exakt Klotoidbaserad vägmodell
för vilken vi designar en Bayesian fusionsram. Med hjälp av detta ramverk uppskattas väggeometrin
med hjälp av sensorobservationer baserat på formen av banmarkeringarna, rubriken på ledande fordon
och positionen av vägarnas radarreflektorer. Utvärderingen av sensordata visar att de föreslagna
algoritmerna kan fånga vägens form väl, även i utmanande bergsvägar.
I [13] har en djup inlärningsmetod utvecklats för att utföra vägdetektering med endast LIDAR-data.
Detta innebär att från ett ostrukturerat punktmoln genereras toppbildsbilder som kodar för
grundläggande statistik, t.ex. genomsnittlig höjd och densitet. Genom att överväga en
toppbildsrepresentation reduceras vägdetektering till ett enkelskaligt problem som kan åtgärdas med
ett neuralt nätverk (FCN).
1.3 Kartpositionering
I detta arbetspaket har vi inom VPRP bland annat tittat på hur man genom enkla sensorer kan uppnå
positionsnoggrannhet.
Undersökningen i [14] utgick man från att bara använda sensorer som är vanligtvis tillgängliga på
bilarna vid den tiden, t.ex. enkla GPS (Global Positioning System), tröghetssensorer och
framåtblickande radar och kamera, tillsammans med en landmärkesbaserad karta. Vår slutsats, även
om vi uppnådde tillräcklig noggrannhet under gynnsamma förhållanden, var lösningen inte tillräckligt
robust för att hantera förlust av en informationskälla (till exempel saknade körfältmarkeringar) eller
förändringar i miljön (väder, säsong, ombyggnad). Dessutom fann vi att de absoluta (globala)
positionsmätningarna från typiska GPS-mottagare var inte tillräckligt noga.
Med hjälp av detta arbete utredde vi olika sätt att förbättra noggrannheten och robustheten hos
positioneringsalgoritmen. I [15] utvecklar vi en kostnadseffektiv lösning för att använda GPS för att
noggrant mäta odometri (rörelsen hos värdbilen) istället för att ge oss felaktiga globala positioner.
Som sådan kan vi, om vi kan placera bilen i en karta, använda GPS-enheten för att utföra automatisk
dödräkning av bilen under mycket längre tid än om vi bara använde hjulhastighet och
tröghetssensorer. Dessutom har vi utvecklat metoder för hur man mer exakt bygger de underliggande
kartorna baserat på landmärken [16], [17] men också hur man både bygger dessa kartor samtidigt som
vi försöker lokalisera oss i dem [18] (SLAM).
1.4 Kooperativ positionering
Det arbete som utförts i VPRP har visat att det finns stor potential inom kooperativ GNSS (Global
Navigation Satellit System) för att öka relativ positionsnoggrannhet och att stödja kooperativa ITS-
applikationer och aktiv säkerhet även inom tung lastbilssegment. Grundidén bakom kooperativ GNSS-
positionering är att fordon utbyter rå GNSS-mätning, data / korrigering eller motsvarande. Med hjälp
av denna data kan differentieringstekniker tillämpas för att beräkna den relativa positionen mellan
fordonen.
Den enklaste formen av kooperativ GNSS skulle vara att byta GNSS (kod) position från fordonet
ombord GNSS mottagare. Eftersom de nuvarande GNSS-mottagarna för fordon uppnår en absolut
positioneringsnoggrannhet på 2 meter i bästa fall (med tanke på RTK-system (Real Time Kinematic))
är denna information dock mer eller mindre användbar för kooperativ positionering.
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 6
I [31] beskriver vi som nästa steg utbytet av råa satellitavståndsmätningar mellan fordon. Genom att
dubbeldifferensiera dessa mätningar kan man avlägsnas atmosfäriska, satellitklockfel och
mottagarklockfel. Detta gör att positionens noggrannhet förbättras väsentligt. Det mätfel som kvarstår
kommer antingen från sk. local multipath dvs. där signalen reflekterats en eller flera gånger innan den
når mottagaren, eller mätfel i mottagarna. I tester med dyrare mottagare och antenner hittas enbart
små precisionsfel och precisionen av dubbeldifferensierade lösningarna är ofta mindre än 50 cm.
Samma test med billigare mottagare visar sämre resultat [31]. En ytterligare förbättring av den relativa positionen i kooperativ GNSS kan uppnås genom att utbyta och använda råa bärvågsmätningar.
1.5 Global positionering
I detta arbetspaket fokuserade man sig på att testa och utvärdera ett urval av GNSS system, för att se
hur väl noggrannheten på systemet kunde upprätthållas när satellitbilden förloras, vilket presenteras i
en teknisk rapport [32]. Slutsatsen blev att med bra dödräkning, så kan låsning av bärvågen erhållas
nästan direkt när satellitbilden åter finns tillgänglig.
1.6 Ruttprediktering
En av huvuddelarna i detta arbetspaket har varit att undersöka hur man använder sig av det repetitiva
mönstret i de flesta människors dagliga pendling (hem till jobbet, jobba för att lagra, lagra hemma etc.)
för att förutsäga resans slutmål när den börjar. Ett bidrag till detta arbetspaket har varit att ta fram en
metod för att klustra fordonets slutposition från många resor till en uppsättning destinationer [27]. En
annan metod som tagits fram är att göra effektiva kortsiktiga förutsägelser under en resa [26], vilket
har visat sig ha fördelar som
Att föreslå en rimlig väg för värdfordonet är en viktig uppgift för alla självkörande fordon. En
grundläggande strategi för att göra det är att uppskatta positionen i en karta och sedan föreslå en väg
som håller fordonet på vägen. Den strategin bygger emellertid på mycket exakt positionsinformation,
en detaljerad karta och frånvaron av fordon och hinder på vägen. Med kraftfulla
maskininlärningsstrategier är det istället intressant att försöka utnyttja all tillgänglig sensorinformation
och försöka styra ett fordon på ett mer mänskligt sätt.
I [21] har ett nytt inlärningsbaserat tillvägagångssätt utvecklats för att generera körbanor genom att
integrera lidarpunkts moln, GPS-IMU-information och Googles körriktningar. Systemet är baserat på
ett fullständigt sk. convolutional neuralt nätverk som gemensamt lär sig att förstå väg och omgivning
från verkliga körsekvenser och som utbildas med hjälp av automatiskt genererade träningsexempel.
Resultaten i detta arbete indikerar att det föreslagna systemet kan bidra till att fylla gapet mellan
lågnivå-parsing och beteende-reflex-tillvägagångssätt genom att generera resultat som ligger nära
kontroll av bilen och samtidigt är mänskligt tolkningsbara.
1.7 Global kartuppbyggnad
För att stödja positioneringsutvecklingen i WP 4 har en exakt och detaljerad karta, kallad HD-kartan,
utvecklats inom VCC, vilket vi också använt oss för att utvärdera våra positionsalgoritmer. Den
fokuserade testvägen för VPRP har anpassats till den avsedda rutten i Drive Me, vilket visas i bild
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 7
nedan [Figur 1].
Den utvecklade HD-kartan innehåller en riklig beskrivning av exakta geometrin av körbanan som
beskrivs i Zenuity internrapport [28], vilket också visar hur kartan är segmenterad i karthorisonter som
möjliggör effektiv kommunikation i realtidssystem.
Ett annat sätt att jobba med karta för tunga fordon, har varit att i detta projekt tittat på metoder för att
simultant jobba med lokalisering och kartläggning (SLAM). Det innebär att upptäcka och kartlägga
landmärken och objekt i förhållande till den globala referensramen och använda dessa sedan för att
lokalisera fordonet. Bland annat beskriver en publikation [29], denna kartläggning och
landmärkningsbaserad lokaliseringsmetoden.
2 Background
The project has been running from Q3 2013 until Q4 2017 with a budget of 24.6 MSEK of which 12.3
MSEK from VINNOVA. Volvo Car Corporation (VCC) has been main responsible for this project.
Participants in this project are:
Volvo Car Corporation (VCC) (556074-3089)
Volvo Technology Corporation (VTEC) (556542-4321)
Chalmers tekniska högskola (CTH) (556479-5598)
Zenuity to work as consultants for VCC from 2017-05-01.
Moving from active safety systems to self-driving vehicles requires greater accuracy in positioning.
However, the next generation of active security systems can also benefit from improved positioning.
Today's system has the requirement to estimate on which road the vehicle is driving, which is not
safety-critical. But many features could be improved if also a lane level positioning accuracy could be
achieved, i.e. determining in which lane the vehicle is driving. Examples of active safety system that
could be improved with a lane level accuracy are lane keeping aid and adaptive cruise functions.
When it comes to autonomously move the car, the requirements of positioning become safety-critical.
You must be able to position the vehicle in a detailed HD map with very high accuracy (
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 8
move the car safely forward, but also to prevent accidents for examples in intersections or crossing
scenarios.
The applications are often based solely on positions provided by GNSS (Global Navigation Satellite
System) systems. Existing GNNS systems are usually quite error- prone, especially in urban areas
where the amount of satellites is limited and the GNSS signals are subject to multi- path effects.
However, the accuracy is usually sufficient for this the navigation systems, but we see a clear demand
of high precision positioning systems in the near future.
Route planning applications has been utilized in vehicles for over a decade; the most common use is
traditional navigation systems, where the driver is presented with a route between two geographical
positions.
This project aims at developing and verifying algorithms to accurately estimate the position of the ego
vehicle and predict the most likely route the vehicle will take in the road network. Insufficient
positioning accuracy is one of the factors which today prevent new safety and convenience systems
from being developed. Examples of such systems are:
Autonomous vehicles. These types of systems require a detailed description of the environment surrounding the ego vehicle and accurate position estimates of the ego vehicle
and all other objects relative to that map. Affordable sensors available on the market today,
such as radars, cameras and digital maps, can provide accurate estimates of the relative
position of surrounding objects, but not a detailed enough description of the static
environment or the position of the moving objects relative to the environment.
Active safety systems. Most of the active safety systems today intervene in accidents caused by something blocking the path of the ego vehicle when the ego vehicle is moving straight
ahead. Rear end collisions or pedestrians walking out in front of the ego vehicle are examples
of such accidents which can be prevented. However, for more complicated scenarios, such as
accidents in intersections, these systems need a detailed map, accurate position estimates of
the objects and reliable predictions of the route the objects are intending to drive.
Engine optimization systems. Several aspects of engine optimization could be improved using accurate ego position estimates and route predictions. One example is in hybrid
vehicles, where knowledge of the upcoming route could have a great impact of brake
regeneration optimization. Similarly, for diesel particle filter regeneration, knowledge about
the ego vehicle final destination, or more precisely an estimate of the driving time and driving
style to reach the final destination, could prevent incomplete regeneration cycles from
occurring.
Cooperative ITS-systems. Intelligent transportation systems (ITS) are transportation systems in which information and communication technologies are used to provide new and innovative
services. In cooperative ITS-systems are information shared between the road users using
vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2I) communication. These systems
would benefit greatly from a more accurate positioning and route prediction of the road users,
since that would enable the systems to put the shared information into a context on how it
would affect the ego vehicle and decide which appropriate measures should be taken.
The level of position accuracy needed varies between these types of systems, but the ambition in this
project is to develop methods supporting all of them. The most demanding position accuracy
requirements will be posed by autonomous vehicle systems as internal studies by the industrial
partners show that these systems need to position objects relative to a detailed map with an accuracy in
the order of magnitude of 0.1 m to 0.2 m. As is discussed in the state of the art section at the end of
this document, this level of accuracy cannot be obtained by any commercially viable systems on the
market today.
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 9
The accuracy of the route prediction can be improved by storing sensor information in a separate map
layer. The stored information is related to driving statistics such as common destinations (as a function
of day and time), common routes and driving style. Using statistics on how and where the driver
usually drives together with accurate ego vehicle position information reliable route predictions can be
obtained.
3 Purpose, research questions and method
In many relevant ADAS solutions, low-accuracy positioning is important to position a car on the
correct road segment. Such solutions include slippery road-detection for example. However, high
precision positioning has a big potential to both create new more advanced ADAS solutions and
improve existing ones. Further, for future AD (Autonomous Drive) features, high precision
positioning is crucial.
Being able to tell which lane a car is in enables functionalities such as automatic lane-changes,
automatic mapping, detection of road abnormalities (accidents, road deterioration etc). It also enables
prediction of probable route choices, which enables improved possibility to optimize engine functions
for example. Even higher precision positioning enables possibility to do autonomous drive through
intersections, better mapping and increased safety in detailed traffic situations.
All these possible solutions lead to lowered environmental impact and increased safety, both which are
core values for VCC and Zenuity.
The research question for this project is divided into following areas:
Positioning
Map positioning How can exact map position be obtained by matching the information obtained from
sensors mounted on the ego vehicle (e.g. radars and cameras) with a static map?
Both currently available and the next generation high precision digital maps will be
evaluated for this purpose. We also foresee that the ability to improve the static map
with observations from the on-board sensors will be important.
Cooperative positioning How the relative positioning accuracy between cooperative nodes can be improved by
communicating raw GNSS measurement data between the nodes?
Global positioning How the current position accuracy of state of the art GNSS systems can be improved
using e.g. more advanced GNSS receiver techniques (carrier phase detection) or using
position information from other sources (LTE, infrastructure)?
Route prediction
Is it possible to establish a driver route prediction based on earlier driver route statistics
to enable reduced fuel consumptions or similar solutions?
To find answers to the above questions, the following methods has been considered:
Jointly exploit all available information sources. The project considers an ego vehicle equipped with a host of on-board sensors such as radars, camera, V2V/V2I communication,
enhanced map databases, GNSS and internal sensors. All these on-board sensors deliver
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 10
imperfect observations of the current local traffic situation.
Utilize statistical historic data to improve the positioning and route prediction at present time. For example, knowing that the driver typically takes a specific exit surely will help us to
improve local route prediction. As such, we collect the information regarding the local traffic
situation as observed by the on-board sensors in a local map and the historic information in a
global map.
Provide a framework (fusion engine) to jointly estimate parameters of interest and provide a convenient description of the surrounding traffic environment to support local route
prediction.
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 11
4 Objective
The objective for this project has been:
To find methods to accurately position the ego vehicle (globally, relative to the map and relative to other vehicles).
Precisely predict the future route both on a local (will we take this exit?) and a global (what is the final destination?) scale.
5 Results and deliverables
Expected results from the project:
Build the necessary competence in this field such that a positioning and route prediction system can be designed which can help realize the new active safety functions and
autonomous vehicle systems.
Present result in academic publications which describe both the mathematical derivations of the developed algorithms and performance evaluations using real data.
Knowledge about what level of positioning accuracy can be obtained using existing sensor technologies and how much this can be improved in the future.
5.1 Result per work packages
The results will be presented below for each work package except work package 1, project
management. Some of the deliveries goes over more than one work package due to that methods in
different combinations can give input to more than one work package. Overview of all work packages
in this project is shown in the picture below [Figure 1].
Figure 1: Work packages
WP1Projectm
anagem
ent
WP2Systemdesign
WP3Localmap
WP6Globalmap
WP4Vehicleposi oning
WP5Routepredic on
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 12
WP2 System design and verification
At VCC/Zenuity, two generations of test vehicles have been equipped [Figure 2] with surround radar
sensors, a lidar sensor, vision sensors and reference sensors. Software for post-processing of this data
has been developed to time synchronize the collected data and to compute an accurate reference
position estimate. The reference position has been used both for building maps and to evaluate the
positioning performance. Methods for evaluating both the map and the positioning methods have been
developed and are continuously used on the collected data. Currently a third generation of test vehicles
is being developed, which seeks to improve on the many lessons learned in this project.
WP3 Local map
WP3 Local map
In creating a map of the local surroundings, the level of detail can be augmented by aggregating static
3D detections from different times, where detections from for example a camera system or Lidar
sensors can be applied. A prerequisite for map aggregation is to precisely know the sensors’
movements. For truck applications this is very challenging when sensors are mounted on the cab, due
to the complicated dynamics of a cab that is suspended on the vehicle chassis, which in turn is
suspended on the wheels. For this purpose, a cab motion model was developed and applied for
filtering the relative trajectory of the vehicle. The filter uses relative motion of cab and chassis,
measured by low-cost IMU sensors, combine with low-cost GNSS carrier phase measurements for
accurate relative position measurements. Results show that very precise trajectories can be estimated,
providing a good base for a short horizon (local) mapping system without using expensive IMU
sensors. The results are detailed in the master’s thesis [30] that was carried out within VPRP.
Within VCC/Zenuity, several contributions to the local map development has been done during the
project within. Prior to the project we had a road estimation framework which fused lane marking
information from a forward-looking camera with object positions from a forward-looking radar. In this
project, we have improved the framework by also including map information which extends the
estimation horizon. We have also developed a method for estimating in which lane the ego vehicle is
driving, referred to as ego lane assignment. This lane assignment module is an important component
for making map-based path predictions. Both the improved road geometry estimation and the ego lane
assignment method is part of the deliverables to the Drive Me project with Volvo Cars. The ego lane
assignment algorithm has been published in [25].
Figure 2: Architecture for the first generation of test vehicles
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 13
To support the host vehicle’s current view of its traffic situation, academia have made contributions to
improve the positioning of other vehicles and pedestrians using novel object tracking algorithms [1]–
[4] as well as developed basic tracking frameworks [5]–[8] and performance metrics for these
scenarios [9]. In addition, they have developed methods to describe the road in front of the host
vehicle, both using radar and camera [10]–[12], and using lidar sensors [13].
In WP3, the objective is to create a local map and to understand the current traffic situation. We
formulate this as an extended target tracking (ETT) problem, where we seek to position all objects in
our vicinity. Some of these objects are dynamic (such as pedestrians and cars) [1]–[4] whereas other
objects are static (such as lamp posts and traffic signs), and by positioning all static objects we obtain
a map of our environment [16], [17]. We have studied a number of different techniques to efficiently
handle the ETT problem and developed new algorithms that we argue are state of the art for the
sensors that we considered.
One of the main problems that we faced while developing new tracking algorithms (see above), is that
there is a lack of benchmark scenarios and useful performance metrics. To develop these benchmark
scenarios, we need to be able to extract accurate trajectory estimates from data, which is a very
challenging problem. As part our work on ETT, we developed new different performance metrics
(among which one received a best paper award [9]) for trajectory estimates.
Related to road estimation academia have developed algorithms for estimating the shape of the road
ahead of a host vehicle. In [10]–[12], the aim is to accurately describe the road geometry up to 200 m
ahead of a host vehicle equipped with a camera, a radar, and vehicle internal sensors, in highway
scenarios. This purpose is accomplished by deriving a precise clothoid-based road model for which we
design a Bayesian fusion framework. Using this framework, the road geometry is estimated using
sensor observations on the shape of the lane markings, the heading of leading vehicles, and the
position of roadside radar reflectors. The evaluation on sensor data shows that the proposed algorithms
can capture the shape of the road well, even in challenging mountainous highways.
In [13], a deep learning approach has been developed to carry out road detection using only LIDAR
data. Starting from an unstructured point cloud, top-view images encoding several basic statistics such
as mean elevation and density are generated. By considering a top-view representation, road detection
is reduced to a single-scale problem that can be addressed with a simple and fast fully convolutional
neural network (FCN). The FCN is specifically designed for the task of pixel-wise semantic
segmentation by combining a large receptive field with high-resolution feature maps. The proposed
system achieved excellent performance and it is among the top-performing algorithms on the KITTI
road benchmark. Its fast inference makes it particularly suitable for real-time applications.
While intelligent transportation systems come in many shapes and sizes, arguably the most
transformational realization will be the autonomous vehicle. As such vehicles become commercially
available in the coming years, first on dedicated roads and under specific conditions, and later on all
public roads at all times, a phase transition will occur. Once a sufficient number of autonomous
vehicles is deployed, the opportunity for explicit coordination appears. In [23], [24] we (Chalmers)
treat this challenging network control problem, which lies at the intersection of control theory, signal
processing, and wireless communication. We provide an overview of the state of the art, while at the
same time highlighting key research directions for the coming decades.
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 14
WP 4.1 Map positioning
For heavy trucks, the possibility of using perception-based sensors to map the environment around the
vehicle for path planning was investigated. Most of the focus has been on lidar technology and
different positioning and prediction algorithms for trajectory and localization estimation. When using
lidar, a highly accurate map can be created and annotated. This can then be used together with
landmark recognition algorithms for localization of the ego vehicle as well as objects detected. One
thesis work [29] has been published about using lidars for map generation and localization.
The use of cheaper sensors for localization of trucks was also investigated [31]. With the use of a
combination of radar, camera, imu and GNSS together with high definition maps, an accurate lane
level positioning was achievable under favorable conditions. In-vehicle test showed that using this
type of localization is not robust enough for autonomous vehicles on its own. Different dead reckoning
Kalman filter implementations were investigated to improve the robustness of the system [31].
Related to vehicle estimation VCC have developed a generic fusion framework both in Matlab and
C++, in which we can implement and evaluate concepts for accurate map positioning using different
filtering techniques and using data from different types of sensors (radars, lidar and vision). The most
promising of these concepts has been evaluated in a real-time environment in a test vehicle. Based on
data from the sensors available, we have for instance been able to evaluate:
Different types of sensor maps and how they affect the positioning
What level of positioning accuracy that has been possible to achieve
How different traffic scenarios affect the positioning performance
Some of these conclusions have been presented in the following publications [14], [15]. One example
was the investigation [14] we did of the position accuracy where you expect just to use sensors
typically available on the cars at that time, e.g. simple GPS, inertial sensors and forward-looking radar
and camera, together with a landmark based map. Our conclusion, although we achieved sufficient
accuracy during favorable conditions, the solution was not robust enough to handle loss of one
information source (for example missing lane markings) or changes in the environment (weather,
season, rebuilding). Additionally, we found that the absolute (global) position measurements from
typical GPS receivers where not nearly accurate enough.
Using this work as a basis, we explored different ways of improving the accuracy and robustness of
the positioning algorithm. In [15], we develop a cost efficient solution for using the GPS to accurately
measure odometry (the movement of the host car) instead of giving us inaccurate global position. As
such, if we can position the car in a map, we can use the GPS to accurately dead recon the cars
position for a much longer time than if we just used wheel speed and inertial sensors. Additionally, we
have developed methods for how to more accurately build the underlying landmarked based maps
[16], [17] as well as how to build these maps simultaneously as we try to localize in them [18]
(SLAM).
Robust cross-seasonal localization is one of the major challenges in long-term visual navigation of
autonomous vehicles. In [19], Chalmers and VCC Ph D exploit recent advances in semantic
segmentation of images, i.e., where each pixel is assigned a label related to the type of object it
represents, to solve the problem of long-term visual localization. We show that semantically labeled
3D point maps of the environment, together with semantically segmented images, can be efficiently
used for vehicle localization without the need for detailed feature descriptors (SIFT, SURF, etc.).
Thus, instead of depending on hand-crafted feature descriptors, we rely on the training of an image
segmenter. The resulting map takes up much less storage space compared to a traditional descriptor-
based map. A particle filter based semantic localization solution is compared to one based on SIFT-
features, and even with large seasonal variations over the year we perform on par with the larger and
more descriptive SIFT-features and can localize with an error below 1 m most of the time.
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 15
During this work, we found that there are no good datasets to evaluate localization performance during
these varying conditions, and thus we have developed our own benchmark suite in collaboration with a
few other universities in Europe and Japan [20].
Practical visual localization approaches need to be robust to a wide variety of viewing condition,
including day-night changes, as well as weather and seasonal variations, while providing highly
accurate 6 degree-of-freedom (6DOF) camera pose estimates. In [20], we (Chalmers) introduce the
first benchmark datasets specifically designed for analyzing the impact of such factors on visual
localization. Using carefully created ground truth poses for query images taken under a wide variety of
conditions, we evaluate the impact of various factors on 6DOF camera pose estimation accuracy
through extensive experiments with state-of-the-art localization approaches. Based on our results, we
draw conclusions about the difficulty of different conditions, showing that long-term localization is far
from solved, and propose promising avenues for future work, including sequence-based localization
approaches and the need for better local features. We will make our benchmark publicly available.
WP 4.2 Cooperative positioning
The work conducted in VPRP has shown that there is great potential in the area of cooperative GNSS
to increase relative position accuracy and to aid cooperative ITS applications and active safety also in
the heavy truck segment. The basic idea behind cooperative GNSS positioning is that vehicles
exchange raw GNSS measurement, data/correction or equivalent. Using this data, differencing
techniques can be applied to calculate the relative position between the vehicles. That is, it will not tell
us where we are, but where we are in relation to each other.
The simplest form of cooperative GNSS would be to exchange GNSS (code) position from the vehicle
onboard GNSS receiver. However, since the currently used GNSS receivers for vehicle achieves an
absolute positioning accuracy of 2 meters at best (disregarding RTK systems), this information is more
or less useless for cooperative positioning. The next step, which has been implemented within the
VPRP project, is to exchange raw GNSS pseudo range measurements between vehicles [31]. By
double differencing these measurements, atmospheric, satellite clock errors, and receiver clock errors
are removed, substantially improving the position accuracy. The remaining errors are mostly from the
local multipath and from measurement errors of the receivers. Tests [31] have shown that these errors
differ between different receivers and antennas. From test series with more expensive receivers and
antennas only small errors are found and the precision of the double differenced solutions are often
less than 50 cm. The same tests for cheaper receivers show worse results [31].
A further improvement of the relative position in cooperative GNSS can be achieved by exchanging
and using raw GNSS carrier phase measurements. The main problem in using carrier phase
measurements is solving for the integer ambiguity parameters in the positioning solution. The carrier
phase measurements are very sensitive to disturbances when tracking the carrier phase of the satellite
signal. Thus, an object blocking the line of sight between the antenna and the satellite will cause the
receiver to lose track (lose fix) and new ambiguity parameters need to be estimated. A possible
solution to overcome short signal outages was investigated in VPRP [31] and showed that it could be
possible to recover the fix (recover the satellite tracking) if the correct algorithms were applied.
WP 4.3 Global positioning
In VPRP, for commercial trucks a state of the art GNSS investigation was carried out. It showed that
many suppliers and technologies are available and constantly improving. A selection of GNSS systems
were tested and evaluated [32] for accuracy together with investigations into maintaining the accuracy
when the view of the satellites is lost. It was understood that with good dead reckoning, carrier phase
lock could be obtained almost immediately once satellite view is restored.
WP 5 Route prediction
The main idea in this work package is to explore how to make use of the repetitive pattern in most
people’s daily commutes (home to work, work to store, store to home, etc.) to predict the destination
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 16
of a trip when it starts. We (VCC) have made several contributions in this area. Some related to the
problem of clustering the end position of the vehicle from many trips into a set of destinations [27].
Other contributions have been in making efficient short-term predictions during a trip [26].
To summarize above, the problem of short range (next link) prediction has been studied [figure 3] and
we have developed a new method [27] which has two main benefits:
Low real time computational complexity
The prediction model can be sequentially updated for each new trip
Pictures below gives a basic visual overview of the new method from VCC for clustering driving
destinations [Figure 4].
[Figure 3: Shot range prediction]
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 17
Suggesting a reasonable path for the host vehicle, is an important task for all self-driving vehicles. A
basic strategy to do so, is to estimate the position in a map and then suggest a path that keeps the
vehicle on the road. However, that strategy relies on highly accurate position information, a detailed
map, and the absence of vehicles and obstacles on the road. With the advent of powerful machine
learning strategies, it is interesting to instead try to leverage on all the available sensor information and
try to control a vehicle in a more human-like manner.
In [21], a novel learning-based approach has been developed to generate driving paths by integrating
lidar point clouds, GPS-IMU information, and Google driving directions. The system is based on a
fully convolutional neural network that jointly learns to carry out perception and path generation from
real-world driving sequences and that is trained using automatically generated training examples. The
results obtained in this work indicate that the proposed system may help fill the gap between low-level
scene parsing and behavior-reflex approaches by generating outputs that are close to vehicle control
and at the same time human-interpretable.
WP 6 Global map building
To support the positioning development in WP 4, an accurate and detailed map, referred to as the HD-
map, has been developed within VCC. The focused test route for VPRP has been aligned with the
[Figure 4: Clustering driving destinations]
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 18
intended route in Drive Me, as is shown in Fig. 1. For this particular route, we (VCC) have both
constructed detailed HD-maps and evaluated our positioning algorithms.
The developed HD-map contains a rich description of the exact lane level geometries as is described in
[28], which also presents how the map is segmented into map horizons allowing for an efficient
communication in real-time systems.
From VTEC, the concept of building a map database or populate and enrich an existing map database
with object data captured by vehicles is previously proven for example in the FFI Learning Fleet
project. In this project and work package, simultaneous localization and mapping (SLAM) methods
have been explored for detecting and mapping landmark objects relative the global reference frame
and use these landmarks for localizing the vehicle. At VTEC, two master thesis workers have been
supervised, in their work on this topic, resulting in a publication [29], which demonstrates efficient
landmark extraction/mapping and landmark-based vehicle localization methods.
5.2 FFI objectives and project contributions
The project focused on two out of six program targets (see bullet list below with OK) but will act as an
enabler and enhance future projects addressing additional target areas such as unprotected road users.
The vehicle and traffic safety program targets:
Intelligent safety systems: OK
Human cognition and tolerance
Crash worthiness (passive and active)
Field studies (e.g. real-life safety): OK
Unprotected road users
Security (e.g. personal protection/integrity)
Figure 1: The selected test route in Dive Me
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 19
Path prediction and high accuracy positioning are important supporting technologies in the future
development of safety and efficiency systems, which can provide the partners with a competitive
advantage on the global market.
The contribution of the VRPR projects results to the FFI-program goals can be summarized as follow:
Reducing road transports environmental impact - Through increased knowledge and results on route prediction, a better optimization of car control was enabled, which reduces
environmental impact. (More effective engine-control).”
Reduce number of casualties in traffic - Better control of detailed traffic situations such as lane-departures, going through intersections, entering and exiting freeways reduces the
probability of accidents, since manoeuvres can be better planned and executed.
Strengthen global competitiveness – Increased safety is an important topic for Volvo which strengthens competitiveness. Increased positional accuracy also enables convenience-
functions which attracts end-customers.
6 Dissemination and Publications
6.1 Knowledge and dissemination of results
Hur har/planeras projektresultatet att
användas och spridas?
Markera
med X
Kommentar
Öka kunskapen inom området X Novel methods for global positioning and route
prediction.
Föras vidare till andra avancerade
tekniska utvecklingsprojekt
X This result will be used in Drive Me project and
further developed and used by Zenuity.
Föras vidare till
produktutvecklingsprojekt
X Result will be a part of the product portfolio at
Zenuity.
Introduceras på marknaden
Användas i utredningar/regelverk/
tillståndsärenden/ politiska beslut
6.2 Publications
[1] K. Granström, S. Renter, M. F.-… S. (IV), undefined 2017, and undefined 2017,
“Pedestrian tracking using Velodyne data—Stochastic optimization for
extended object tracking,” ieeexplore.ieee.org
Chalmers
[2]
K. Granstrom, L. Svensson, S. Reuter, Y. Xia, and M. Fatemi, “Likelihood-based
data association for extended object tracking using sampling methods,”
IEEE Trans. Intell. Veh., pp. 1–1, 2018.
Chalmers
[3]
K. Granström and M. Baum, “Extended Object Tracking: Introduction, Overview
and Applications,” arXiv, pp. 1–17, 2016.
Chalmers
[4]
M. Beard, S. Reuter, K. Granström, B. T. Vo, B. N. Vo, and A. Scheel, “Multiple
Extended Target Tracking With Labeled Random Finite Sets,” IEEE
Trans. Signal Process., vol. 64, no. 7, pp. 1638–1653, 2016.
Chalmers
[5] Y. Xia, K. Granstrcom, L. Svensson, and A. F. Garcia-Fernandez, “Performance Chalmers
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 20
evaluation of multi-bernoulli conjugate priors for multi-target filtering,” in
20th International Conference on Information Fusion, Fusion 2017 -
Proceedings, 2017.
[6] Á. F. Garc’\ia-Fernández, L. Svensson, and M. R. Morelande, “Iterated statistical
linear regression for Bayesian updates,” in Information Fusion (FUSION),
2014 17th International Conference on, 2014, pp. 1–8.
Chalmers
[7]
L. Svensson and M. Morelande, “Target tracking based on estimation of sets of
trajectories,” in Information Fusion (FUSION), 2014 17th International
Conference on, 2014, pp. 1–8.
Chalmers
[8]
Á. F. Garc’\ia-Fernández, L. Svensson, M. R. Morelande, and S. Särkkä,
“Posterior linearization filter: Principles and implementation using sigma
points,” IEEE Trans. signal Process., vol. 63, no. 20, pp. 5561–5573,
2015.
Chalmers
[9] A. S. Rahmathullah, A. F. Garcia-Fernandez, and L. Svensson, “Generalized
optimal sub-pattern assignment metric,” in 20th International Conference
on Information Fusion, Fusion 2017 - Proceedings, 2017.
Chalmers
[10]
L. Hammarstrand, M. Fatemi, Á. F. García-Fernández, and L. Svensson, “Long-
Range Road Geometry Estimation Using Moving Vehicles and Roadside
Observations,” IEEE Trans. Intell. Transp. Syst., vol. 17, no. 8, 2016
Chalmers
[11]
M. Fatemi, L. Hammarstrand, L. Svensson, and A. F. Garcia-Fernandez, “Road
geometry estimation using a precise clothoid road model and
observations of moving vehicles,” in 2014 17th IEEE International
Conference on Intelligent Transportation Systems, ITSC 2014, 2014.
Chalmers
[12] A. F. Garcia-Fernandez, L. Hammarstrand, M. Fatemi, and L. Svensson,
“Bayesian road estimation using onboard sensors,” IEEE Trans. Intell.
Transp. Syst., vol. 15, no. 4, 2014.
Chalmers
[13] L. Caltagirone, S. Scheidegger, L. Svensson, and M. Wahde, “Fast LIDAR-based
Road Detection Using Fully Convolutional Neural Networks,” 2017 IEEE
Intell. Veh. Symp., pp. 1019–1024, Jun. 2017.
Chalmers
Zenuity PhD
[14]
M. Lundgren, E. Stenborg, L. Svensson, and L. Hammarstrand, “Vehicle self-
localization using off-the-shelf sensors and a detailed map,” in IEEE
Intelligent Vehicles Symposium, Proceedings, 2014.
Chalmers
VCC PhD
[15]
E. Stenborg and L. Hammarstrand, “Using a single band GNSS receiver to
improve relative positioning in autonomous cars,” in IEEE Intelligent
Vehicles Symposium, Proceedings, 2016, vol. 2016–Augus.
Chalmers
VCC PhD
[16]
M. Fatemi, K. Granström, L. Svensson, F. J. R. Ruiz, and L. Hammarstrand,
“Poisson Multi-Bernoulli Mapping Using Gibbs Sampling,” IEEE Trans.
Signal Process., vol. 65, no. 11, 2017.
Chalmers
[17]
M. Lundgren, L. Svensson, and L. Hammarstrand, “Variational Bayesian
Expectation Maximization for Radar Map Estimation,” IEEE Trans. Signal
Process., vol. 64, no. 6, 2016.
Chalmers
[18]
M. Fatemi, L. Svensson, L. Hammarstrand, and M. Lundgren, “Variational
Bayesian em for SLAM,” in 2015 IEEE 6th International Workshop on
Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP
2015, 2016.
Chalmers
[19]
E. Stenborg, C. Toft, and L. Hammarstrand, “Long-term Visual Localization using
Semantically Segmented Images,” Jan. 2018.
Chalmers
VCC PhD
[20]
T. Sattler, W. Maddern, A. Torii, J. Sivic, T. Pajdla, M. Pollefeys, and M. Okutomi,
“Benchmarking 6DOF Urban Visual Localization in Changing Conditions,”
arXiv Prepr. arXiv1707.09092, 2017.
Chalmers
[21] L. Caltagirone, M. Bellone, L. Svensson, and M. Wahde, “LIDAR-based Driving
Path Generation Using Fully Convolutional Neural Networks,” Mar. 2017.
Chalmers
[22]
A. Garcia-Fernandez, L. Svensson, and S. Sarkka, “Cooperative localisation
using posterior linearisation belief propagation,” IEEE Transactions on
Chalmers
http://www.vinnova.se/ffi
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 21
Vehicular Technology, 2017.
[23]
H. Wymeersch, G. R. De Campos, P. Falcone, L. Svensson, and E. G. Ström,
“Challenges for cooperative ITS: Improving road safety through the
integration of wireless communications, control, and positioning,” in 2015
International Conference on Computing, Networking and
Communications, ICNC 2015, 2015, pp. 573–578.
Chalmers
[24]
R. Hult, G. R. Campos, E. Steinmetz, L. Hammarstrand, P. Falcone, and H.
Wymeersch
“Coordination of Cooperative Autonomous Vehicles: Toward safer and more
efficient road transportation,” IEEE Signal Process. Mag., vol. 33, no. 6, 2016.
Chalmers
[25] Svensson, Daniel, and Joakim Sörstedt.
"Ego lane estimation using vehicle observations and map information." Intelligent
Vehicles Symposium (IV), 2016 IEEE. IEEE, 2016.
VCC
[26] Panahandeh, Ghazaleh, Nasser Mohammadiha, and Joakim Lin Sorstedt
“Method and system for utilizing a trip history." U.S. Patent Application No.
15/441,689.
VCC
[27] Panahandeh, Ghazaleh. "Driver route and destination prediction
" Intelligent Vehicles Symposium (IV)”, 2017 IEEE. IEEE, 2017.
VCC
[28] Sörstedt, Joakim
“HD-map interface specification”, Internal Zenuity report
VCC
[29] Lindholm, Robin, Pålsson, Carl-.Johan
“Simultaneous Localization and Mapping for Vehicle Localization using LIDAR
Sensors”, http://publications.lib.chalmers.se/records/fulltext/219126/219126.pdf
VTEC
[30] Agnas, Edvin, Jerenvik, Marcus
“Estimation of position and orientation of truck kinematic frames”,
http://publications.lib.chalmers.se/records/fulltext/238492/238492.pdf
VTEC
[31] Jimmy Hammenstedt, Edvin Valtersson, Sudha Vana
“Vehicle Position and Route Prediction”
VTEC
http://www.vinnova.se/ffihttp://publications.lib.chalmers.se/records/fulltext/219126/219126.pdfhttp://publications.lib.chalmers.se/records/fulltext/238492/238492.pdf
FFI Fordonsstrategisk Forskning och Innovation | www.vinnova.se/ffi 22
7 Conclusions and future research Increased security and accessibility with the help of automation is more relevant now than when the
project was started. As the result section shows, we have successfully met the goals set up for the project and have as a
result gained important knowledge related to vehicle positioning and route prediction. Our approach
has been to first develop detailed maps using specific mapping vehicles equipped with high accuracy
reference sensors, and then in a second step develop algorithms for estimating the vehicle position in
this map using affordable automotive grade sensors. Future research in this area could focus on how to
probe source the map using simpler sensors and instead build the map sequentially from multiple trips.
Also, in the academic world, interest in this knowledge grows, as has been demonstrated by the
majority of new courses.
There are still a lot of challenges to make a robust and scalable positioning solution for the
autonomous vehicle. It is clear that a combination of positioning technologies is needed as there is no
standalone system that can satisfy all the requirements. As for routing it is shown that high definition
maps can really help with path and behavioural planning. Further research into more robust and
reliable localization is needed to solve the autonomous needs going forward.
8 Participating parties and contact persons
VCC (Zenuity as consultant from 2017-05)
Marita Andersson, Research lead New Technology, Zenuity
Joakim Lin-Sörstedt, Product owner Localization & Road estimation, Zenuity
Mikael Edvardsson, 94401 Strategy, Concepts and Advanced Engineering Active Safety, Volvo Cars
Cooperation
Chalmers University of Technology
Department of Electrical Engineering
Tomas McKelvey, Professor Signal Processing
Lars Hammarstrand, Post-Doctoral Research Fellow in the Signal processing
lars.hammarstrand@chalmers.se
Volvo Group Trucks Technology
Jimmy Hammenstedt, Vehicle Automation Environment Perception
Hans Deragården, Vehicle Automation Environment Perception
http://www.vinnova.se/ffimailto:lars.hammarstrand@chalmers.se