Proposte di Tesine/TesiLaboratorio di Sistemi Autonomi Intelligenti

IAS-Lab

Dipartimento di Ingegneria dell’Informazione

Università di Padova

IAS-Lab

IAS-LAB

Body pose estimation is the ability to detect and track human joints while a person is moving. This ability is mandatory for several applications as action recognition, health-care and long-term people re-identification. In the state-of-the-art there are plenty of algorithms for single body pose estimation. However, when multiple cameras are available, we have enough information for enhancing the final body-pose quality. This proposal is about the design and implementation of a multi-view approach for the real-time multi-people body pose estimation problem. • Final Project (ED3D):

– Test different single-view algorithms– Apply the best algorithm found with a baseline multi-view fusion algorithm

• Thesis:– Study and compare different multi-view approaches – Combine them to achieve better results

Marco Carraro

Multi-view body pose estimation from RGB-D data

IAS-LAB

REQUIREMENT: You should speak englishOpenPTrack is an open-source people detection and tracking algorithm from RGB-D data. The people are detected by classifying the different clusters of points after an initial ground plane removal phase. The objective of this project is to make the people detection work better with different conditions (e.g. people close other objects, or in unconventional poses). • Final Project (ED3D):

– Make OpenPTrack work with Ubuntu 16.04 • Write an installation guide to be published online

– Compare different people detection algorithm at the state of the art.

• Thesis:– Fuse different approaches/Create a new more robust approach for people detection

Marco Carraro, Yongheng Zhao

OpenPTrack: people detection and tracking from RGB-D data

IAS-LAB

• Re-identificazione: riconoscere una stessapersona osservata in istanti/luoghi diversi

• Feature soft biometriche: insieme/cascata di feature in grado di fornire unadescrizione dettagliata di unapersona

• Sviluppo di nuovi body model per la gestione dei dati soft biometrici

• Applicazione delle feature soft biometriche alla re-identificazione

Stefano Ghidoni

Re-identificazione di persone

IAS-LAB

• Deep learning: reti neurali profonde– Apprendono le feature,

che non devono perciò essere hard-coded

• Applicare tecniche di deep learning alla re-identificazione di persone– Creazione di nuove

feature da estrarre

• Elemento critico: dataset

Stefano Ghidoni

Deep learning per re-identificazione

IAS-LAB

Morris Antonello

Comparison of 3 SLAM algorithms

You will compare 3 state-of-the-art algorithms for reconstructing 3D sceneslike InfiniTAM, Elastic Fusion and RGBDSLAM v2.This will help our research in semantic mapping (finding scene structures andbig objects in reconstructed scenes), currently based on RGBDSLAM v2.Tesina: you will compare them on the datasets freely available athttp://cs.nyu.edu/~silberman/datasets/.Tesi: you will test them on a dataset acquired from a mobile robot. You willhave to face with real situations like a robot in front of a wall.

IAS-LAB

Morris Antonello

Detecting people lying on the floor

You will improve our algorithm for detecting people lying on the floor so as tofind a good trade-off between runtimes and detection rates.Tesina: you will find out the bottlenecks and test different combinations ofparameters to speed up the pipeline.Tesi: you will test the algorithm on a dataset we acquired in a real apartment.You will exploit new fast features calculated on RGB or infrared images withthe aim of reducing the number of false positives.

IAS-LAB

Morris Antonello

From detection to semantic segmentation

From the detection of a person lying on the floor to the semantic segmentation of a generic class

You will extend our algorithm for detecting people lying on the floor so as to semanticallysegment them. In a subsequent step, you will adapt the current algorithm so as to semanticallysegment a generic class.Tesina: you will extend our algorithm so as to semantically segment the detected people. Indeed,“detection” does not mean “semantic segmentation”. If you simply detect, you lose the actualboundary of the person.Tesi: you will train the SVMs involved in the pipeline on some other classes (walls, doors, sofas,tables…). To do so, you will use the NYU-Depth datasets freely available athttp://cs.nyu.edu/~silberman/datasets/.

IAS-LAB

Andrea Cimolato

Multiclass “Cascade” SMR-BCI Protocol

Objective:

Design and implementation of a multi-class “cascade” protocol for Brain

Computer Interface (BCI) framework based on Sensorimotor Rhythms (SMR).

You will learn to:

Design and prototype novel protocols for biological signal acquisition and robotic devices control

Interface and program BCI frameworks

Signal processing and classification

Requirements:

Good knowledge of C++, signal acquisition and processing

TESINA: (max 1 student)

Design and implementation of a multi-class

protocol based on a two classes BCI-SMR,

employing a “cascade” composition of the binary control already validated.

TESI: (max 2 students)

Integration and evaluation of the

implemented protocol for the navigation control on Humanoid Robot ‘NAO’

Integration and evaluation of the

implemented protocol in the existing tool

for autonomous navigation of an autonomous telepresence robot ‘O-Robot’

IAS-LAB

Andrea Cimolato

Manipulator Control with multiple MYO

Manipulator Control with multiple MYO Gesture Control Armband

Objective:

Design and implementation of a manipulator control with multiple MYO Gesture Control Armbands

You will learn to:

Record and analyze data from MYO.

Interfacing and programming an embedded system

EMG signal processing and classification

Requirements:

Good knowledge of C++ and Matlab, signal acquisition and processing

TESINA: (max 1 student)

Interfacing multiple MYO Gesture Armbands, employing Windows SDK and existing Python scripts under ROS frameworks.

TESI: (max 2 students)

Real-Time control implementation for a robotic manipulator

employing two MYO devices. The control has to be applied and

validated in a "center-out reaching tasks" protocol and finally compared with the control proposed in [1].

Real-Time control implementation for a robotic manipulator

employing two MYO devices. The control has to be applied and validated in a “reach-to-grasp tasks” protocol.

Both the two thesis proposal are planned in two steps: first, motion

control based on IMU data and, second, using an EMG-driven

approach. Testing phase of the two respective protocols are planned first under simulation environment and then on the real robot NAO.[1] Cimolato, A. et al. Muscle Synergies for Reliable NAO Arm Motion Control: an Online Simulation with Real-Time Constraints, SIMPAR 2016, San Francisco, California, December 13th-16th.

IAS-LAB

Roberto Bortoletto, Andrea Cimolato

Object Recognition and Grasping

Objective:

Design and implementation of a software for the object

recognition and grasping. Realization of a “reach-to-grasp

tasks” protocol throw manipulator with stereo-camera on board.

You will learn to:

Use the OpenCV library to perform object recognition

Interfacing and programming of a vision-based robotic system

Use ROS to control movements of a robotic manipulator

Requirements:

Good knowledge of C++ and OpenCV

Object Recognition and Grasping using a Robotic Manipulator

TESI: (max 1 student)

Implementation of a software for:

i. Object recognition using stereo-camera based on object shapes and color.

ii. Planning of “reach-to-grasp” motions toward the recognized object.

Testing of the implemented software before in

simulation and after on real robot (UR5 robotic arm

with the gripper and stereo-camera mounted on board).

.

IAS-LAB

Stefano Michieletto

Anticipating Human Activities

Tesina: Anticipation is usedextensively by human forinteracting with each other.The aim of this work is to test astate of the art method usingan Anticipatory TemporalConditional Random Field(ATCRF) that models the richspatial-temporal relationsthrough object affordances.Code and data are available inthe project website.

Tesi: The ATCRF method is based on 3 subtasks: Human Activity FeatureExtraction, Activity Labeling, and Activity Anticipation. The skeletal tracking forthe first task (based on NITE) will be replaced with a more reliable methodbased on deep learning. Compare the two methods and try to improve theresults obtained by using the standard method.

IAS-LAB

Stefano Michieletto

Writing with robots

Tesina: Use the codeavailable for extracthand informationwhile people write aletter. Model theletter with aGaussian MixtureModel. Use themodel to make therobot write theletter itself.

IAS-LAB

Stefano Michieletto

Obstacles Avoidance with Octmap

OctoMap is a 3D occupancy grid mapping suited for robotics. The map implementation is based on octree.

Tesina: Represent info provided by a 3D camera with Octomapand use it in MoveITfor obstacle avoidance purposes.

IAS-LAB

Francesca Stival, Stefano Michieletto

Online Dynamic Time Warping

This method allows us to find a good match between two sequencesby using a non-linear distortion with respect to an independentvariable (time).

Tesi: look for methods available at the state of the art. Extend thecurrent version of the algorithm to an online version working with 100-200 ms of available signal. Test the result by controlling humanoids androbotic hands.

IAS-LAB

Tesina: The aim of this work is simulatingthe leg rehabilitation through prosthesisor exoskeleton. The leg movement isperformed by humanoid robot. Wecompute bending angles related to hip,knee, and ankle from real human motionin order to move the robot. Part of thework consists on developing a supportstructure to allow movement of one legavoiding robot damages or falls.

Tesi: Estimate online movements forrobot walking by processing EMG dataextracted from multiple subjects. Createof a subject-independent model usingmachine learning techniques.

Francesca Stival, Stefano Michieletto

Robot walk for rehabilitation

IAS-LAB

Tesina: The pipeline above allows us to control a robot byusing sEMG signals collected from human muscles. The dataused for the study came from recorded demonstrations.The aim of this work is to move to a real device, namely aMyo armband, in order to have a real-time working demo.

Francesca Stival

Online Prosthesis Control using sEMG

Human

MovementsEMG signals

analysis

Gaussian

Mixture

Model

Gaussian

Regression

Robot

motion

Morris Antonello – morris.antonello@dei.unipd.itRoberto Bortoletto – roberto.bortoletto@dei.unipd.it

Marco Carraro – marco.carraro@dei.unipd.it

Andrea Cimolato – andrea.cimolato@dei.unipd.it

Stefano Ghidoni – stefano.ghidoni@dei.unipd.it

Emanuele Menegatti – emanuele.menegatti@dei.unipd.itStefano Michieletto – stefano.michieletto@dei.unipd.it

Michele Moro – michele.moro@dei.unipd.it

Francesca Stival – francesca.stival@dei.unipd.it

Elisa Tosello – elisa.tosello@dei.unipd.it

Yongheng Zhao – zhao@dei.unipd.it