Advanced Robotics: Autonomous Mobile Robots
Arshad Jamal,
Scientist, Intelligent Systems and Robotics Division Centre for AI & Robotics, DRDO
Bengaluru
1. Motivation 2. History and Current Scenario 3. Technologies for Autonomous Mobile
Robots 4. Capability requirements 5. Robotic Systems developed by CAIR
a. Systems with different mobility b. Semi-autonomous mobile robots c. Autonomous mobile robots d. GPS-less autonomous navigation
i. SLAM algorithms ii. Autonomous search robot
Outline
Why Autonomous Robotics Systems?
1. First response, Surveillance and Reconnaissance, Patrolling
2. IED handling, UXO handling, Mine laying and breaching
3. Communication relays, Logistics transport 4. Convoy protection, Road clearance 5. Target Identification and Tracking, Remotely
operated weapons 6. Disaster management
History of Autonomous Vehicles 1977: Tsukuba Mechanical Engineering Lab in Japan creates the first autonomous,
intelligent, vehicle. It tracked white street markers and achieved speeds up to 30 km/h.
1987-1995: The pan-European Prometheus project, also known as the EUREKA
Prometheus Project, the largest autonomous vehicle project so far, is funded by the European Commission.
1997: Demo '97 in San Diego, California, in which about 20 automated vehicles,
including cars, buses, and trucks, were demonstrated 2000s: Several DARPA challenges made significant contributions in development of
autonomous vehicles. Google starts work on its driverless car in 2009 2010s: All major automobile companies are working on driverless autonomous
cars
Major Research Programme for Autonomous Vehicle
• DARPA Grand Challenge 2005 – Autonomous driving through 150 miles of
desert terrain in less than 10 hours – Winner: Stanford Racing Team (6:54 hrs),
Stanford University
• DARPA Urban Challenge 2007 – Autonomous driving through 60 miles in an
urban environment in less than 6 hours – Winner: Tartan Racing (4:10 hrs), Carnegie
Mellon University
• DARPA Robotics Challenge 2014 – Humanoid robots in disaster management
• Enter, drive and exit a vehicle, Clear obstacles and open a door, Climb a ladder, Turn off a valve, Attach a hose
Global Scenario - Military • Light weight man-portable to heavy duty systems
– Scouting operations, intelligence, surveillance and reconnaissance, IED handling, logistics support, mine clearance
Dragon runner Viper PackBot Talon
MULE Gaurdium Big Dog Panther
Technologies for Autonomous Mobile Robots
Mobility
Power Communication
Human-Robot Interaction
Localization
Perception Planning
Navigation
Learning/ Adaptation
Enabling Technologies
Autonomous Behaviour
1. Perception a. Multi-modal environment sensing and data fusion
• Vision (EO, IR), ranging (LIDAR, RADAR, SONAR), tactile (haptic) b. Highly influenced by operational environment and platform
embodiment 2. Knowledge representation
a. Ontologies • Describes entities and their relationships
b. Enables abstraction of concepts and inference c. Provides the bridge between perception and machine reasoning
3. Reasoning a. ‘What-If’ scenario modeling - Projection of actions and consequences
into the future in a given context b. Highly dependent on embodiment c. Enables high level behaviours
• Collaboration
Key Capabilities for Unmanned Systems
4. Planning a. Decomposition of a mission specification into specific tasks
• Considers available platforms and capabilities vis-à-vis mission requirements – non-trivial!
b. Requires semantic characterization of capabilities c. Requires reasoning to infer that a composition of capabilities can
meet a mission requirement d. Scheduling, monitoring and re-planning in context e. Key enabler for effective collaboration
5. Learning a. The defining principle of intelligence! b. Enables high-level behaviours
• improvisation, adaptation, cunning, strategy 6. Self-monitoring for effective immunity from external infections
a. Guard against internal corruption in the system b. Immunization against adversarial take-over c. Identify the right set of security requirements starting from policy to
architecture to models to mechanisms; more than just encryption!
Key Capabilities of Unmanned Systems
1. Consists of three repeated steps a. Sense your
environment b. Plan what to do next
by building a world model through sensor fusion and taking all goals (both short term and long term) into account
c. Execute the plan through actuators
Key Capabilities: Sense-Plan-Act Paradigm for Autonomous Systems
Sensory Inputs
Actuation
Intelligent Robotics Systems Developed at CAIR
Technology Focus Areas Mobility
Leg & Wheel Legged Serpentine Wall Climbing Tracked
Autonomous Navigation
Robot Sentry Wheeled Vehicle Tracked Vehicle
Flapping Quad-rotor
Manipulation
Hot slug handling, NMRL
Inspection of Composites, HAL
Steam generator inspection, NPCIL
Educational Manipulators arms
Mobile Manipulator
Perception
Change Detection Fusion Tracking SLAM
1. Suspension with linkage mechanism a. Six actuated wheels b. Parallel bogies in center c. Fork suspension in front d. Step climbing capability upto
1.5 times wheel diameter
2. Suspension with Spring-Damper a. Six actuated wheels b. Lower vibration c. Suspension in both roll and
pitch
Wheeled Locomotion- Passive Suspension
1. Requirements a. Stair climbing b. Self righting c. High ground clearance
2. Multi-segment tracked robot (MiniUGV) a. Main tracks in center b. Tracked flippers in
front and rear with endless rotation
c. Camera in front and rear
d. Remote operation with 160m NLOS range
e. 3 hrs of endurance f. 50 Kg payload
capacity on flat terrain
Tracked Locomotion
1. Hexapod a. Cockroach type
i. 6 legs with 2 DoF b. Crab type
i. 6 legs with 3 DoF ii. Joint level leg control (Angles)
c. Omni-hex i. 6 legs with 3 DoF ii. Cartesian level leg control (X, Y, Z) iii. Ultrasonic Sensors for obstacle
avoidance
Legged Locomotion
1. Quadruped a. No static stability b. Lateral movement of
body can shift the CG to stable location
c. 3-DoF Leg design with extra Hip joint
d. Gaits i. Crawl: one leg at a
time with body sway ii. Trot: diagonal legs
at a time at higher speed
Legged Locomotion
1. Design a. 2-DoF Joints with alternate
horizontal and vertical joints b. Free wheels to emulate differential
friction 2. Combination of travelling waves
along horizontal and vertical plane generate
a. Lateral undulation b. Caterpillar gait c. Side-winding gait d. Rolling gait
Serpentine Locomotion
Transverse wave propagates along the body Differential Friction results in forward motion
Wave travel
Body displacement
1) Low pressure adhesion 1. Impeller to generate vacuum for
sticking to wall a. Suction force large enough
to offset the robot weight b. Suction force Small enough
to enable locomotion 2. Differential tracks for motion 3. Camera on 2-DoF arm
2) Electro-adhesion based - Adhesion due to electrostatic
force - Noiseless operation - High endurance
Wall Climbing
Semi-autonomous Mobile Robot
1. Autonomous Navigation a. with a-priori map
2. Continuous Video Feed a. via Pan-Tilt-Zoom Network
Color Camera 3. GPS and Stabilized Digital
Compass a. for Localization
4. Scanning Laser Range Finder a. for Obstacle Avoidance
5. WiFi Link a. for Command & Control
Semi-Autonomous Navigation – Sentry Robot
Current Pos/Head Desired Pos Laser range data
Autonomous Mobile Robot
1. Unstructured environment 2. No predefined map is available 3. Only coarse waypoints may be available 4. Day and night operation 5. Modular, scalable and extensible hardware and
software architecture
Autonomous Tracked Platform - Scope
Hardware Architecture
• Drive-by-wire system interface
• Vehicle speed and heading controller
Steering actuation Throttle actuation
Topological Architecture
21-03-2016 24
Goal Specification
Vehicle Drive-by-Wire
Control
GPS
INS
Odometer
Localization Data Fusion
LOCALIZATION PERCEPTION
Monocular Camera
Stereo Camera LIDAR RADAR
Perception Data Fusion
Global path-planning
Local path-planning PATH PLANNING
Localization
50 Hz
Obstacle map
Occupancy map
Perception
10 Hz 15 Hz 4 Hz
5 Hz
Goal specification
Global path
planning
Local path planning
Path planning
0.5 Hz 10 Hz
Vehicle control
Navigation 20 Hz
Autonomous Navigation System
Goal specification
Global path
planning
Local path planning
Path planning
0.5 Hz 10 Hz
Localization
50 Hz
Obstacle map
Occupancy map
Perception
10 Hz 15 Hz 4 Hz
5 Hz
Vehicle control
Navigation 20 Hz
Sensor interface for data acquisition Vehicle pose (position and attitude) Vehicle velocities
Coordinate frame transforms for localizing sensor data with respect to the vehicle
Localization
Goal specification
Global path
planning
Local path planning
Path planning
0.5 Hz 10 Hz
Localization
50 Hz
Obstacle map
Occupancy map
Perception
10 Hz 15 Hz 4 Hz
5 Hz
Vehicle control
Navigation 20 Hz
Generation of 3D point cloud from LIDAR data Generation of occupancy map from 3D point cloud Fusion of occupancy maps to create obstacle map
Perception
Range data
Intrinsic parameters
Point Cloud (in sensor frame)
Point Cloud (in vehicle frame)
Extrinsic parameters
Point Cloud (in world frame)
Vehicle pose
New occupancy map Existing occupancy map Fused occupancy map (log odds formulation)
Fused occupancy map (weighted sum formulation)
Obstacle map (thresholded occupancy map)
Occupancy map
Occupancy map
Path planning
Range data Point Cloud (in vehicle frame)
Extrinsic parameters
Point Cloud (in world frame)
Vehicle pose
New occupancy map Existing occupancy map Fused occupancy map (log odds formulation)
Nodder pose
+ + ∫
Goal specification
Global path
planning
Local path planning
Path planning
0.5 Hz 10 Hz
Localization
50 Hz
Obstacle map
Occupancy map
Perception
10 Hz 15 Hz 4 Hz
5 Hz
Vehicle control
Navigation 20 Hz
Integration of ARA* search-based lattice planner for global path planning
Integration of Trajectory Rollout for local path planning
Path Planning
Obstacle map Inflated Obstacle map Global plan
Local plan
Goal specification
Motion primitives
Newtonian mechanics model
Desired vehicle speed and turning Vehicle control
Vehicle pose
Path Planning
Obstacle
Vehicle position and heading
Occupancy map
Path Planning
Goal pose
Vehicle pose
Obstacle inflation
Path Planning
Replanning
Localization
50 Hz
Obstacle map
Occupancy map
Perception
10 Hz 15 Hz 100 Hz
5 Hz
Goal specification
Global path
planning
Local path planning
Path planning
0.5 Hz 10 Hz
Vehicle control
Navigation 20 Hz
Implementation of entire workflow using open-source framework ROS 12 nodes configurable to run
across multiple computation units
Configurability of key parameters for adaptability to different platforms Vehicle dimensions Motion model Map dimensions and resolution Obstacle definitions Sensor mounting configuration Sensor interface configuration
Software Architecture
1. Web-based a. Can be run from any laptop
connected to vehicle WiFi 2. Configuration of all software
modules and parameters 3. Selection of modules and
customization of parameters for a particular run
4. Selective power-on of systems and sensors
5. Live feed of vehicle position and image data, if available
GUI
• Autonomous navigation • With online cost-map generation • In unstructured terrain
• Drive-by-Wire conversion of a COTS tracked vehicle Terex ST50
• Localization system • High accuracy integrated GPS-INS
• Perception system comprising – 3D LIDAR for long range mapping – 2D LIDAR with nodding for short
range mapping – Color camera for feedback
• Onboard embedded computational platform comprising
– Networked box PCs • Wireless link for remote
monitoring
Autonomous Navigation – Tracked Vehicle
Autonomous Navigation – BMP
Towards GPS-less Autonomous Navigation
Localization Where am I? The problem of estimating the pose (position &
orientation) of the robot relative to a map. Mapping
What does the world look like? Mapping is the problem of integrating the information
gathered with the robot's sensors into a given representation.
Simultaneous Localization And Mapping Defined as the problem of building a map while at the
same time localizing the robot within that map.
Simultaneous Localization And Mapping
Figure: A classification of various SLAM algorithms
Classification of SLAM algorithms
1. Kalman Filter and variants – EKF, UKF based feature based SLAM
2. Particle Filter (PF) based – Rao Blackwelliesd Particle Filter based
3. Information Filter based – Sparse extended information filters
Filtering Based SLAM Approaches
• Particle Filter based Algorithm – GMapping – Rao-
Blackwellised PF based algorithm.
Filtering based SLAM: Algorithm Choice
Video: Sentry Indoor Navigation
• ORB-SLAM – Output in CAIR Outdoor campus – Output in CAIR indoor campus
• LSD-SLAM – Output in CAIR outdoor campus – Output in CAIR indoor campus
Monocular SLAM
RGBD SLAM
RGB images Frame to
frame motion estimation
Robot Location
Depth images
Sensor processing module for computation of visual odometry is aided with observations from 2D LiDAR
RGBD sensor data
To “Build Pose Graph”
2D LiDAR
RGBD SLAM
Pose Graph
Bag of Words loop closures
Spatial proximity loop closures
View Point based loop closures
Frame Matching
Loop Closures
Loop closure detection is driven by Bag-of-words approach.
Yes
No
RGBD SLAM Results
Video: RGBD SLAM in System Block, CAIR
Simulation video of Multi-robot exploration and mapping [speed-up 12x]
Simulation Video
1. Two robots interacting with each using ROS Multi-master a. Both the robot are controlled by the same
ROS console. b. Share of data between the robots and the
operator console e.g. online created maps from both robots.
c. Display of robot data on the operator console.
d. Autonomous navigation by the robots. e. Video: Demonstration of ROS Multi-master
Two robot communicating using Multi-master
Technologies
at work
Artificial Intelligence
Perception
Robotics
1. Environment: Unknown. Urban indoor terrain.
i. User asks the robot to search for a particular color or object in the environment.
ii. Robot enters the space, searches for objects and sends them back to user’s device.
iii. Map of explored areas and its current position is continuously updated on the user screen.
iv. Robot returns to its initial position after the search operation is completed.
Autonomous search robot
1. Exploration Algorithm a. Finds new frontiers as goals, monitors updated map for new
frontiers 2. Path Planning
a. Plans a minimum cost path to the goal 3. Obstacle Avoidance
a. Avoids obstacles in the path and steers robot away 4. Simultaneous Localization and Mapping
a. Creates map of environment and finds its own location 5. Object detection
a. Detects user desired objects, captures images and marks its location in the environment map
6. Visualization and User Interface a. Renders the environment and displays the detected objects
interactively
Autonomous Search Robot
Results for Autonomous Search Robot
Visual Perception Technologies
Visual Perception Technologies
Camera Calibration
Feature Detection and matching
Sparse 3D estimation
Time-to-collision
Motion estimation
Stereo Vision
Dense 3D estimation
Visual Odometry and
SLAM
Object, Place, Scene
Saliency Analysis
1. Significant advances in every aspect of the field
2. CAIR has developed many advanced robotics system
3. Currently working on Multi-Agent Robotics System to manage scale of the problem
4. Long Term Autonomy and Never Ending Learning paradigms are needed
5. Huge potential for collaboration between DRDO, Industry and Academia
Conclusion