Laser-Camera systems and algorithms to sense the environment for robotic applications. The research project: FP7-People-Marie Curie-COFUND-Trentino-post-doc 2009 - Incoming. Presenter: Dr. Ilya Afanasyev Lab. Mechatronics, University of Trento, Italy - PowerPoint PPT Presentation
Laser-Camera systems and Laser-Camera systems and algorithms to sense the algorithms to sense the environment for robotic environment for robotic applications applications Presenter: Dr. Ilya Afanasyev Lab. Mechatronics, University of Trento, Italy [email protected], [email protected]PAT mid-term review, Trento, 2011 The research project: FP7-People-Marie The research project: FP7-People-Marie Curie-COFUND-Trentino-post-doc 2009 - Curie-COFUND-Trentino-post-doc 2009 - Incoming Incoming 18/10/20 11 1/ 15
Transcript
Laser-Camera systems and algorithms Laser-Camera systems and algorithms to sense the environment for robotic to sense the environment for robotic applicationsapplications
The research project: FP7-People-Marie Curie-The research project: FP7-People-Marie Curie-COFUND-Trentino-post-doc 2009 - IncomingCOFUND-Trentino-post-doc 2009 - Incoming
18/10/2011 1/15
About the grant holderAbout the grant holder
2/15
AREAS OF EXPERTISE 1. Robotics. Computer vision. Data Treatment and Analysis. 2. Experimental Physics. Optical sensors (UV, Visible, and IR).
QUALIFICATIONS AND EDUCATION2006 Ph.D. (in Technical Sciences), Vavilov State Optical
Institute, St. Petersburg, Russia (www.npkgoi.ru) Specialization: Optical-electronic instruments
1994-2000 full-time student at the Baltic State Technical University, Russia (www.voenmeh.ru) Qualification: Engineer Specialization: Automatic Control Systems for flying apparatus
Dep., Vavilov State Optical Institute, Russia (www.npkgoi.ru)2000 – 2006 Researcher/PhD student, Aerospace Physical Optics Lab.,
Vavilov State Optical Institute, Russia (www.npkgoi.ru)
18/10/2011
Status of the ProjectStatus of the Project
The current project proposes using a Laser-Camera System (LCS) as an instrument to get 3D information of the world environment. LCS includes a scanning laser rangefinder (LRF) and RGB camera. The rotated LRF captures an object as unstructured “clouds of points” of 3D data, whereas a camera gives a 2D image of the object. The fusion of 2D and 3D sensor’s data gives full and detailed information about the object and its environment.
3/15
The 3-years project in Mechatronics has been doing since May 2010.
Description of the ProjectDescription of the Project
18/10/2011
The calibration of laser-camera systemThe calibration of laser-camera system
4/15
2D and 3D sensor fusion should be a result of a mutual sensor calibration. The calibration is important for: a compatibility of sensors measurements, and having the metrical coordinate system with good accuracy. We are using the standard camera calibration technique (Jean-Yves Bouget method).
Then joint 3D and 2D data processing is used: to recognize an object, to estimate its orientation in 3D space, and to improve LCS calibration by mathematical modeling.3D object visualization / reconstruction can be executed by math. modeling with SuperQuadrics (SQ).
http://lrv.fri.uni-lj.si/~franc/SRSbook/SRS.html
,
sin)(sin
sin)(sincos)(cos
cos)(coscos)(cos
4
54
54
3
2
1
a
aa
aa
signuma
signumsignuma
signumsignuma
z
y
x
;2/2/
.
where x,y,z - superquadric system coordinates; η, ω – spherical coordinates, where a1, a2, a3 – parameters of object scaling; a4, a5 – parameters of object shape.
The object recognition algorithm is based on using RANSAC (RANdom SAmple Consensus) method of SuperQuadrics model fitting to objects presented in 3D “cloud of points”. The solution is verified by evaluating the matching score between the SQ object model and 3D real data by a robust least square fitting.
- Building a prototype of a robotic vehicle with installed laser-camera system
7/15
The laser and the camera oriented and calibrated together have been formed the Laser-Camera system
18/10/2011
Mid-term results & achievementsMid-term results & achievements- Testing the sensitivity (laser remission effectiveness) of the Laser-Camera System to registration of the objects from different materials (metals, plastic, wood, paper, polyethylene, etc.) at the different illumination conditions.
8/15
The polyethylene is transparent for IR rangefinder, but multilayer polyethylene decreases the depth of recognition significantly.
18/10/2011
Mid-term results & achievementsMid-term results & achievements- Capturing 3D data and development of 3D object reconstruction / recognition algorithms (for human body, etc. achieved at www.ing.unitn.it/~afanasye/)
9/15
3D Human Body Pose Estimation: photo, “cloud of points”, fitting a math. model to 3D data, final math. model of pose.
18/10/2011
Future project workFuture project workAccording to the work plan, the future project work should be dealt with: installation of the developed Laser-Camera System on a robotic vehicle. research & control of robot vehicle motion. improving Laser-Camera System calibration technique. improving object recognition algorithms. implementation algorithms to a robotic vehicle. developing algorithms of relative position estimation (robot - object). verification of the algorithms with hardware facilities.
10/1518/10/2011
Possible impactsPossible impacts 3D Laser-Camera System can provide 2D/3D measurements capturing 2D/3D data of an object (with distance and color information) for indoor applications. Object recognition and pose estimation algorithms can be applied for: detection and reconstruction by math. models of different real objects (including complex objects, like human body). detection, classification and selection a definite thing in a heap of different objects for robotic manufacturing (such as a search an object in mechanical details laying in mess). different scientific projects (with a possibility of a collaboration).
11/1518/10/2011
Training experiencesTraining experiences
18/10/2011 12/15
During the project I participated: at 21th Summer School in Jyväskylä University, Finland, 8-19 August 2011 (www.jyu.fi/summerschool/). in preparation of 3 conference papers: to 21th Int. Conference: GraphiCon'2011 (Moscow, 2011, http://gc2011.graphicon.ru/); to RGB-D Workshop on 3D Perception in Robotics (Sweden, http://ias.cs.tum.edu/events/rgbd2011); to Int. Conference VISAPP-2012 (Rome, 2012, February 2012, http://visapp.visigrapp.org). Intensive course of Italian as a second language (Corso intensivo di lingua italiana per stranieri), CIAL, University of Trento, test (Livello A2), February 2011. 3D Modelling in 3ds Max. Training at St. Petersburg State Polytechnical University, 2011. Certificate #1KQYHQDCK1.
Training experiencesTraining experiences
18/10/2011 13/15
During the project I have been engaging in:- Leading a Mechatronics group (2 Master students) on measurements of remission from different materials (wood & aluminum foil & polyethylene) by IR rangefinder.- Giving a lecture on “Object Detection with Superquadrics” in “Robotics and Sensor Fusion for Mechatronics Systems” (course of Assoc. Professor Mariolino De Cecco, University of Trento, Italy). Povo, 05 April 2011. - Close collaboration with Italian colleagues for experimental and theoretical work, making seminars with presentation: www.ing.unitn.it/~afanasye/10_UniTN_reports/20_Presentations/.- Creating, discussing and sharing some algorithms in MATLAB: www.ing.unitn.it/~afanasye/.
Impacts on career developmentImpacts on career development
18/10/2011 14/15
The project helps in: developing the professional skills in computer vision and robotics, getting an experience in resource, time and money managing, and strengthening the researcher reputation, that must be valuable for future work at a research center or an industrial company.
AcknowledgementsAcknowledgements
18/10/2011 15/15
This work of Ilya Afanasyev on laser-camera system creation and algorithms of 3D object recognition, localization and reconstruction has been supported by the grant of EU\FP7-Marie Curie-COFUND - Trentino post-doc program, 2010-2013.He is very grateful to colleagues from Mechatronics Dep. of UniTN, especially supervisor - Professor Mariolino De Cecco.
