Hyun Myung, Ph.D.Professor, School of Electrical Engineering

Head, KAIST Robotics Programhmyung@kaist.ac.kr

http://urobot.kaist.ac.kr

Urban Robotics Lab“IT & Robotics for smart cities”

6-DoF SHM robot system

Vision / LiDAR / Magnetic field / RF-based 3D SLAM

JEROS (Jellyfish Elimination RObotic Swarm)

Low-cost self-driving

Oil spill protection robot

CAROS (Climbing Aerial Robot System)

Green algae removal robot

UAV

Green Algae

FAROS (Fireproof Aerial Robot System)

Robot Navigation

Structural Health Monitoring

Environmental Robotics

Autonomous valet parking Mole-bot

AI & Robot IntelligenceAutonomous robot navigation Vision, LiDAR, Magnetic field-based SLAM (Simultaneous Localization And Mapping) Indoor, Outdoor (Ground, Underwater, Underground, Aerial) SLAM

Object recognition UAV image-based jellyfish recognition, green algae recognition, underwater jellyfish recognition Object recognition using low-resolution aerial images

Human behavior recognition for HRI (Human Robot Interaction) Human pose estimation Sequence-based human behavior recognition

Developed robot system in our lab JEROS (Jellyfish removal robot), ARROS (Green algae removal robot) CAROS, FAROS (Wall climbing drones)

2

Autonomous Robot Navigation

Mapping•Feature map•Grid map

Localization•Localization•Relocation

Motion Control•Path Planning•Reactive Control

ExplorationActive Localization

SLAMIntegrated Approach•Active SLAM

(Source: Makarenko et al. ’02)

Core technologies of autonomous navigation

Graph-based sensor fusion for dynamic and feature-poor environment False constraints removal in dynamic environment (low + high dynamic) SLAM performance improvement using low-cost sensor fusion

• ex1) 2D LiDAR + 2D camera = 3D matching for robustness• ex2) Magnetic feature + 2D LiDAR = minimize planar localization error

KAIST’s technology

5

GP-SLAM for low-dynamic environmentGrouping nodes

Pruning false constraints

Normalized covariance values of the odometry edges. Result obtained after grouping the nodes

Error metric (average Mahalanobis distances of the edges) based on

the grouped nodes.Optimized trajectory after pruning false constraints

Edges btw. groups 2 & 6 can be pruned

6

GP-SLAM for low-dynamic environmentVisual odometry

Conventional SLAM

GP SLAM

Donghwa Lee, Hyun Myung, "Solution to the SLAM Problem in Low Dynamic Environments Using a Pose Graph and an RGB-D Sensor," Sensors, vol.14, no.7, pp.12467-12496, [Link], July 2014.

7

SLAM in dynamic environment (LiDAR sensor)Real-time recognition of high dynamic (human) and low dynamic objectsGraph-based SLAM using 3D feature matching & LRF scan matchingTested on KAIST Geocentrifuge center building (10 × 10m) with 2 low dynamic and 5 high dynamic objects

8

3D SLAM for autonomous driving

Stereo camera GPS

3D point cloud map

Odometry

Robust localization for autonomous driving

Hyungjin Kim, Bingbing Liu, Chi Yuan Goh, Serin Lee, Hyun Myung, "Robust Vehicle Localization using Entropy-weighted Particle Filter-based Data Fusion of Vertical and Road Intensity Information for a Large Scale Urban Area," IEEE RA-L (Robotics and Automation Letters), vol.2, no.3, pp.1518-1524, [DOI], July 2017.

Velodyne, GPS, IMU, and wheel odometry

Mean error ≤ 0.1 m10

3D SLAM using a tilted 2D LiDAR sensor

3D localization and mapping (SLAM) using a tilted 2D LiDAR sensorImproving accuracy through hierarchical graph optimization

Before global optimization

Global optimization result

Super node generation

Super node optimization

Sensor system

Hierarchical graph optimization

Robot motion

Tilted LiDAR

Local map

graph

Local map

graph

Local map

graph

Local map

graph

Local map

graph

Local map

graph

Global map graph

Local point cloud maps

3D SLAM using a tilted 2D LiDAR sensor

3D SLAM using a tilted 2D LiDAR sensor

Earth’s magnetic field-based SLAM

Motivation Feature-poor (Vision/LiDAR)

environment Anomaly of earth’s magnetic field

(20~80 μT) is used as features

<3-axis magnetic field distribution>

14

Magnetic sequence-based graph SLAM

Jongdae Jung, Taekjun Oh, and Hyun Myung, "Magnetic field constraints and sequence-based matching for indoor pose graph SLAM," Robotics and Autonomous Systems, vol.70, pp.92-105, [DOI], Aug. 2015

Information matrix Num of constraints

Robot localization

- Cyan: reference (SICK Gmapping)- Magenta: wheel odometry- Blue: optimized pose graph

Localization (KI building) Magnetic field map

15

Mole-bot

Directional drilling

Directional drilling for shale gas extraction requires: Underground localization RSS (Rotary Steerable System) mechanism Control algorithm

17

source: Katie Mazerov , “New directional drilling system combines improved mud motor, MWD technologies”, Drilling Contractor (2011), Hallibuton Hompage

Mole-bot

• Embedded directional drilling

• Underground localization

Underground comm. & monitoring system

As-is To-be

18

Mole-bot

Digging mechanism

Source: NY Times (https://youtu.be/toARdZKs-IE)

19Scapula as a lever

20

Mole-bot

Front part Rear part

21

Mole-bot

x15 x10

Forward drilling Directional drilling

Underground localization using Graph SLAM

The magnetic sensor can measure similar features againDesigned for loop closing at both direction movement

Forward movement Backward movement

Concurrent normalized cross-correlation

22

Underground localization using Graph SLAM

Results Comparison of RMSE

OrientationPosition

Dead reckoning 77.8cm

12.5cm

10.2º

Proposed algorithm [1] 2.4º84.0%↓ 76.5%↓

[1] Byeolteo Park and Hyun Myung, "Resilient underground localization using magnetic field anomalies for drilling environment," IEEE Trans. Industrial Electronics, [DOI], Feb. 2018.

23

Structural Health Monitoring

SLAM in aerial environment

Bridge inspection using UAVs

25

UAV SLAM Framework

GPS + Camera + IMU + 3D LiDAR

Camera + IMU

GPS

3D LiDAR

Visual Inertial

OdometryNode

generationNormal

Distribution Transform

Graph optimization

Poseestimation

pose

Pose (only if it is available)

Generalized ICP

with voxels

Constraint

node

pose

node

3D point cloud (2.6s/frame)

Voxel of 3D point cloud (0.7s/frame)

NDT3D point cloud

UAV SLAMDownward camera

Upward camera

CAROS: Wall-climbing drone (robust to wind)CAROS (BBC News, May 2015)

Tilt-rotor-based low-impact perching

Significant decrease of impact in perching

5o/s

87o/s

Applications of a wall-climbing droneFAROS (MBC News, Apr. 2016)

Environmental Robotics

JEROS (Jellyfish Elimination RObotic Swarm)

USV(unmanned surface vehicle) + Jellyfish shredding device Autonomous navigation Formation control: leader-follower Vision-based jellyfish detection &

distribution recognition Jellyfish removal performance: 216

kg/hour

Damage by jellyfish in South Korea: over 300M USD/year (fishery: 230M, power plants: 70M)

PROBLEM

DEVELOPMENT

JEROS (BBC News, 2015)

Formation control test

Jellyfish distribution recognitionJellyfish images using drones

Jellyfish distribution recognition

Deep learning GPS/IMU aided image-based localization

Jellyfish images

Localized Jellyfish distribution

JEROS formation control

Jellyfish removal

Coverage path planning ASV launching and docking

Jellyfish distribution recognition using DNN

Proposed LeNet-5 AlexNet GoogLeNetAccuracy [%] 94.06 95.11 95.60 98.37

Average frame rate[Hz] 8.23 4.89 0.01 0.02

Proposed network

Hanguen Kim, Jungmo Koo, Donghoon Kim, Sungwook Jung, Jae-Uk Shin, Serin Lee, and Hyun Myung, "Image-Based Monitoring of Jellyfish using Deep Learning Architecture," IEEE Sensors, vol.16, no.8, pp.2215-2216, Apr. 2016.

ARROS (Algal bloom Removal RObot System)ECF (electrocoagulation and floatation) reactor

Sungwook Jung, Hoon Cho, Donghoon Kim, Kyukwang Kim, Jong-In Han, and Hyun Myung, "Development of Algal Bloom Removal System Using Unmanned Aerial Vehicle and Surface Vehicle," IEEE Access, Dec. 2017.

hmyung@kaist.ac.kr http://urobot.kaist.ac.kr