RAVON - Platform for Outdoor
Field Robotics
Prof. Dr. Karsten Berns
Robotics Research Lab
Department of Computer Sciences
University of Kaiserslautern
Application Areas for Off-road Robots
• Service tasks
– agriculture, forestry, logistics,
environmental monitoring
• Assistance in disaster areas
– equipment transport, scouting
• Security tasks
– site / border patrol, demining
Off-road Robots for Different Applications
Timberjack
© John Deere
Workpartner
© Helsinki University of Technology
Telerob tEODor
© www.telerob.de
Remotec Wheelbarrow Revolution
Off-road Vehicles
Stanley (Stanford) RTS-Dora (Hannover)
Demo III (U.S. Army)
Navlab (CMU)
RoboScout (Base 10)
Sojourner (JPL) Amor
(Uni Siegen) MuCar
(UniBW München)
RAVON Project
Goal: Development of vehicle that can fully autonomously operate in
highly vegetated, rough terrain
Operational Environment
RAVON
• 4WD with separate motors
• Front and rear axes can be steered
independently
• Max. velocity: 10 km/h
• Max. slope: 100% at 7 km/h
• Energy source: 8 Lead batteries
• Runtime: about 4 hours
• LWH: 2.35x1.4x1.8 m
(highest point: GPS antenna)
• Weight: 750 kg
• Ground clearance: 0.3 m
Control System Architecture
Drive
Commands
Hardware Interface
Sensor Processing
Sector
Map
Creation
Mediation
Global Navigation
Short-term
Memory
Raw
Sensor Data
Se
cto
r M
ap
s
Status
Status Target
Poses
Target
Poses
Safety System
Point Access
Local Path Planning
Local Navigation
External Sensor Systems
Long Range Color Stereo System Large Scale Terrain Traversability Estimation
Short Range Color Stereo System Obstacle Detection
Rotating 2D Laser Scanner Obstacle Detection / 3D Local Memory
Planar 2D Laser Scanner Obstacle Detection / Safety System
Spring-Mounted Bumper Tactile Vegetation Discrimination
Instrumented Safety Bumper
Design and construction of the bumper
Tactile Mapping of the environment
Titel des Vortrags
3D Laser Scanner Concept
Sensor Processing
Main sensor: panning laser range finder
• Yields position, shape, and type of hazards in 3D space, e.g.:
Solid Object Penetrable Object
• Penetration technique for evaluating an obstacle’s rigidity
( Gras or rock?)
h
d h h
d
Positive Obstacle Negative Obstacle Overhanging Obstacle Water
Titel des Vortrags
beneath ground
ground
near ground
far ground
sky
shortest ground
distance
Y
X
• Extract ground representatives
(blue)
• From ground level classify:
• Pts beneath ground
• Associated ground pts (purple)
represent the load-bearing
surface
• Near ground pts
(surmountable)
• Far ground pts (positive or
overhanging obstacles)
• Sky points
Evaluation of Individual Scans
Titel des Vortrags
Solid Entities
Vegetation
Classification – Examples
Example of Content of Grid Map Cells
• General information: Properties
• Can also store height or organize cell space in 3D
Titel des Vortrags
Vision-Based Obstacle Detection
• Stereo-vision-based 3D scene analysis
• Tracking of obstacles over time
-> 4th Dimension
• Filter artefacts
• Building local maps
• Determination of colour
and texture
Water Detection based on Polarisation
Detection Process
AquaVisor C
(H&S Robotic Solutions, KL)
Compact design
90mm x 90mm x 130mm
Weight: 600g
Onboard sensor data
evaluation
Extension modules for
Vegetation detection
3D-enhanced Vision
Grid Map as Local Obstacle Memory
• Robot centric, orientation-fixed local grid map
• Scrolls in x- and y-direction
• Serves as local obstacle memory
– Robot “remembers” obstacles
– Collisions with obstacles not in direct sensor range can be
avoided
Sensor Data Representation
• Data of different sensor systems is entered into
grid maps
• Grid maps used as “local obstacle memory”
Important as sensor systems cannot
cover complete area around robot
• Sector maps as abstract and uniform
data representation
• Extracted from grid map
• Each sector contains
information about area
it covers Cartesian Sector Maps Polar Sector Map
Grid Maps
Extraction
Sector Maps around RAVON
Local Navigation (Pilot) and Drive Modes
• Task: drive robot to next target location while avoiding collisions with
obstacles
• Fast driving: requires long-range obstacle detection and the
evaluation of all detected obstacles
• Moderate driving:
– Uses penetration evaluation for vegetation density determination
– Maximum velocity reduced as only close-range area can be
monitored
• Tactile creep:
– Evaluates deflection of the bumper system
– Maximum velocity is limited to very low value
Pilot can adopt robot’s basic driving behaviour to situation
Behavior-based Control Approach
Approach: Decompose task into relatively simple independent
components
• Incremental implementation with subsequent evaluation
• Distributed representation of environment, no central world model
Problems to be solved:
• Find suitable decomposition
• Coordination of contradictory commands
• Validation of system behavior
Behavior-Based Architecture iB2C
• Uniform behavior modules
• Behavior B defined as B=(r,a,F) with
– Input vector e and output vector u sensor/control data
– Stimulation s in [0,1] gradually activating the behavior
– Inhibition i in [0,1] gradually deactivating the behavior
– Activation ι=s∙(1-i)
– Activity a in [0,1] indicating how much B is doing
– Target rating r in [0,1] assessing current situation
– Transfer function F(e,s,i)=u implementing the functionality
Basic behavior module
Behavioral Groups
Behavioural Group for
Fast Driving
Structure of the Pilot
Emergency Stop
Drive Commands
Point Access
Behavioural Group for
Moderate Driving
Behavioural Group for
Tactile Creep
State Switching and
Behavioural Group Stimulation
Stimulation
Drive Commands
Drive Commands
Fusion
Drive Commands
Human
Operator
Control Chains
Slow Down Behaviors
Safety Behaviors
Keep Distance Behaviors
Evasion Behaviors
Structure of the Guardian
Intermediate Layer - Passage Driving
• Use of passage realised by behaviour-based network
• Robot is drawn towards PEMP
=> does not drive into indentation
• Robot reaches PEMP with orientation of passage
=> is driving smoothly into the passage
Start
Passage Entry
Draw towards Passage
Target
Passage Orientation
Global Robot Navigation
• How can a robot navigate efficiently to
a distant location?
• Requires a world model (a map)
– quantify terrain traversability
→map cost measure
• Traversability estimation is hard
– ground properties
– flexible/rigid vegetation
?
Proposed Combined Approach
• Formulate a global (off-road) navigation strategy which
– scales to very large environments, but
– allows to reason about terrain traversability
• Solution strategy:
– use hybrid world representation
• global topological navigation layer
• local metrical pilot layer
– extend topological world model to handle traversability costs
– transfer local cost experience to global level
Abstract Views - Example
• Sensors
Physical
Short-Term Memory
Slowly Traversable Terrain Quickly Traversable Terrain
Sensors
Virtual
Sensors
Pilot
• Receives target from topological navigator
• Uses sensors to build metrical (grid) short-term memory
• Relevant memory content is transformed into standardized ‘abstract views’
• Behavior based control system uses abstract views for steering and collision avoidance
Local Traversability Maps (LTMs)
• Obstacle information is
relevant for later
topological map
extension
• Data is transferred from
abstract views into local
traversability maps
(LTMs)
• LTMs store abstracted
cost modifiers around a
topological node
Local Traversability Maps
A posteriori Cost Learning
• Observation of
behaviors yields cost
information after each
edge traversal
• Spatio-temporal
integration
• Transfers real-world
cost experience from
local to global scope
Topological Exploration
• Generate multiple new
path hypotheses
between two
unconnected nodes
• Predict likely costs
• Add cheapest route
• How to predict costs for
unknown edges ?
Edge Cost Calculation
• Three types of edge costs: risk, effort, and familiarity
• Risk and effort can be learnt during edge traversing by monitoring
distinct behaviours of pilot
No need to analyse the terrain extensively and construct a highly
detailed world model
• Different methods to predict costs of edges that have not been
traversed yet
Costs for driving in partly unknown terrain can be estimated
Vision Based Terrain Analysis
Image Feature Extraction
stereo camera system
color image pairs
texture area descriptors
HSV
VAR
• Highly parallelizable operators extract independent information
– Local Binary Pattern, VARiance, Hue Saturation Value
• Descriptors support efficient area union: →
contrast
color
LBP
BA
)( Adi
)()()( BdAdBAdiii
A
Unsupervised Segmentation
• Construct image regions with similar apperance → Fewer, more stable objects to classify
• Unsupervised Split & Merge segmentation – Jenson-Shannon divergence as pseudo-metric
– Threaded neighborhood lists for efficient merging
Self-Supervised Learning Procedure
1. Stereo image capture
2. Edge traversal
3. Experience projection
4. Region correlation
5. Prototype addition
Experiment: Self-Supervised Learning
Offroad Path Detection
• Not only look for free traversable space
• Find structures that help reaching the horizon
Long Range Stereo Reconstruction
Finding Ground and Horizon
• “Side-view“ Analysis
• Filter by density
• Apply RANSAC and least squares fitting
• Classify horizon and ground region
Obstacle Analysis
• Front-to-back Analysis
• Adopt obstruction to back
• Extract upper ground edge
• Fit polynomial to ground
• Use derivative for traversability
classification
• Take declination of vehicle into
account
.
.
.
Traversability Map
• Apply perspective back-transformation
• Extraction and filtering of traversable regions
Path Classification via Particle Filter
• Apply path primitives (polynomials of deg. 2)
• Primitives are managed as particles
• Overlapping of traversability map and primitive
form confidence
+ =
Place Recognition
• Machine-readable description needed
• Descriptors must be comparable
• Common known fact: Single features do
no generally work on their own
• Redundancy compensates for single
feature failure
• What is a place?
– Relevance for navigation decisions
– Descriptor is prominent in local area
Point Features (SURF)
• Salient points in scale-space (similar to SIFT)
• Gradient in these point forms feature descriptor
• Set of features forms place descriptor
Comparison of Point Features
• Use RANSAC filter for affine transformed correspondence sets
• 500 random sets are feasible for shifted and scaled images
Area Features (LBP)
• Partition image and compute LBP histgram for each tile
• Image descriptor is the list of histograms
Comparison of Area Features
• Shift images and compare histograms
• Calculate normalized sum of absolute differences
• Example for not matching images
Comparison of Area Features (Matching)
• Minimal value for each pair relates to best shift
• Low value means high similarity
• Not robust against roll rotation
Results: Correct Decision due to SURF
• SURF (blue) outweighs uncertain and wrong LBP (orange)
• Image pairs showing the same place usually have a great number
(>>10) of SURF matches after RANSAC filtering
Results: Correct Decision due to LBP
• In some situations SURF failes
• LBP does not dominate in one place but can compensate
• Still correct with wron SURF (blue) and correct LBP (orange)
Control System Architecture
Travel through Palantine Forest
Thank you for your attention!
http://agrosy.informatik.uni-kl.de
References
• [Wünsche 07] LIDAR-based Perception for Offroad Navigation M.
Himmelsbach, F. Hundelshausen, H-J. Wünsche
• [Siegwart 07] Path Planning, Replanning, and Execution for
Autonomous Driving in Urban and Offroad Environments R.
Philippsen, S. Kolski, K. Macek, R. Siegwart
• [Peterson 05] Red Team Too, Darpa Grand Challenge 05, Technical
Paper, K. Peterson et. al.
• [Urmson 05] High Speed Navigation of Unrehearsed Terrain, C. Urmson
et. al.
• [Pfaff 08] Probabilistic Models for Autonomous Systems, P. Pfaff
References 2
• [Thrun 95] Map Learning and High Speed Navigation in RHINO,
S. Thrun et. al.
• [Vale 95] Mobile Robot Navigation in Outdoor Environments: A
Topological Approach, A. Vale
• [Tapus 05] A Cognitive Modeling of Space using Fingerprints of
Places for Mobile Robot Navigation, A. Tapus et. al.
• [Madrigal 99] Mobile robot path planning: a multicriteria
approach, J. Fernandeza-Madrigal et. al.
Related Work
• Most offroad robots use metrical grid maps
• SmartTer (ETH Zuerich) [Siegwart 07]
– global planning: multi-level-surface map, field D* algorithm
– local planning: grid map, driving arcs
• Red Team (Sandstorm, CMU) [Urmson 05]
– global: A* on high-resolution elevation map
– local: laser scan evaluation, path adjustment
• Red Team Too (CMU) [Peterson 05]
– global: A* on high-resolution elevation map
– local: obstacle + traversability cost map, A*
• MuCar (Uni BW München) [Wünsche 07]
– local: traversability cost map, driving ‚tentacles‘
Related Work
• Topological Maps are mostly used indoors or in urban areas
• RHINO (Uni Bonn) [Thrun 95]
– focus on fast path planning (dijkstra), abstraction (voronoi)
– distance based cost measure
• ATRV-Jr (Uni Lisbon) [Vale 05]
– Focus on localization (POMDP), map building (EM)
– constant cost measure
• Tapus (ETH Zürich) [Tapus 05]
– focus on localization (POMDP), map building (fingerprints)
– constant cost measure
• Ram-2 (Uni Malaga) [Madrigal 99]
– focus on multicriteria path planning
– hierarchical constraint satisfaction, pre-learned costs
LADAR based Classification