Jürgen ‘Juxi’ Leitner

Reactive Reaching and Grasping on a Humanoid Robot

Dalle Molle Institute for AI (IDSIA)

#ICINCO 2014

humanoidour iCub

object manipulationtowards learning

http://robotics.idsia.ch/

projectIM-CLeVeR

http://robotics.idsia.ch/im-clever/

object manipulationtowards learning

http://robotics.idsia.ch/

!

perceptionvisual

thanks to G. Metta and IIT for this picture

!

objectsdetecting

Harding et al., 2013Leitner et al., ICDL 2012, ARS 2012, BICA 2012, CEC 2013

cartesian genetic programming

+ min dilate avg INP INP INP

[Leitner et al, 2012a/b, Harding et al., 2013]

learningapproach

detection

! icImage GreenTeaBoxDetector::runFilter() { icImage node0 = InputImages[6]; icImage node1 = InputImages[1]; icImage node2 = node0.absdiff(node1); icImage node5 = node2.SmoothBilateral(11); icImage node12 = InputImages[0]; icImage node16 = node12.Sqrt(); icImage node33 = node16.erode(6); icImage node34 = node33.log(); icImage node36 = node34.min(node5); icImage node49 = node36.Normalize(); ! //cleanup ... icImage out = node49.threshold(230.7218f); return out; }

detect

detect

detection

approachcgp

visualhand detection

[Leitner et al, 2013]

handsdetecting

approachsupervised learning

BUT

segmentationfeature

saliencymap

collaboration FIAS

presegmentation

approachcombined

!

transferringspatial perception

setuplearning

trainingset

9DOF

iCubbounding box

6 per eye Carte

sian

Coor

dinate

s

.

.

.

spatial perception neural network

. . .

9DO

F iC

ubbo

undi

ng b

ox

6 pe

r eye

Cart

esian

Coor

dina

tes

!fu

lly c

onne

cted

!fu

lly c

onne

cted

. . .

object manipulationtowards learning

http://robotics.idsia.ch/

MoBeEframework Frank et al., ICINCO, 2012.

MoBeEFrank et al., ICINCO, 2012.

Frank et al., ICINCO, 2012.

motion

robot[Stollenga et al, IROS 2013]

generationmotionStollenga et al, 2013

Shak

ey 2

013

Win

ner

MoBeEv2[Frank et al., 2011,2012, 2013]

hand/armop-space forcing

CSWorld

CSHand

CSR/CSL

[Leitner et al, in prep]

-

object manipulationtowards learning

http://robotics.idsia.ch/

coordinationhand-eye

model

http://robotics.idsia.ch/

manipulation for improved perception

manipulation actions

extracting information

improveddetection

detection

! icImage* BlueCupFilter::runFilter() { icImage* node43 = InputImages[4]; icImage* node49 = node43->LocalAvg(15); ! icImage* out = node49->threshold(81.532f); return out; }

detection

! icImage* BlueCupFilter::runFilter() { icImage* node0 = InputImages[4].Exp(); icImage* node5 = InputImages[0]; icImage* node16 = node0->Gabor(-8,14,1,13); icImage* node17 = InputImages[4]->LocalAvg(6); icImage* node18 = node16->Laplace(5); icImage* node19 = node5->Sobel(13,9); icImage* node24 = node17->Erode(5); icImage* node28 = node19->Min(node18); icImage* node29 = node28->Min(node24); icImage* node41 = node29->LocalAvg(7); icImage* node49 = node41->LocalMax(7); ! icImage* out = node49->threshold(68.032f); return out; }

resulting

operationtele

teleoperation

!

workfuture

improved eye-hand coordination

different object representations

online, continuous learning

vSLAM aerial robotics & for world model

for listeningthanks

juxi@idsia.ch http://Juxi.net/projects

http://dilbert.com/strips/comic/2013-10-24/