Perception-enabled KnowledgeIAS Internal Workshop
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Intelligent Autonomous Systems Group
Technische Universität München
June 14, 2012
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Outline
1. Semantic MappingSemantic Mapping
2. Perception of Objects of Daily UseInteractive SegmentationObject Categorization - Graph-PartsObject Categorization -Hough VotingODUFinderObject Recognition using Barcodes
3. Dynamic Scenes and Spatio-Temporal Memory (Nico)GPU-based PerceptionCompression of Point CloudsUnstructured Information Management applications
4. Misc
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingSemantic map representation
Abstract knowledgeabout object classes
Object instances and componenthierarchy
Poses in the environmentand their changes over time
Related: TBOX/SBOX, Galindo et al (RAS
2008)
[Pangercic IROS2012]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingSystem Version 1
������������������������������������������AB���C���D�
E�����F������D�
��A�DC���D�D����D������CD�����
�
���������A�BC�C
DCEF�C��EC�F�
�F������������D��A�
�B��B��F�����C����
���F��C��AC����BF�F�E��������EC��FA�F�����
D�E�����DC�A�AC�F����������F�
AD�����D�EE�������CAC�����D�
�F�F�C��������B����C�A�B�C�F�� !���"F�F�
�C��AC�������
!���"F�F�
�F���������������C�F��
C�A�#�$��F�
*
[Blodow IROS2011]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingAcquisition of Sensor Data
360deg Scans plus ICP optimizations:• overlaping regions• zeroed roll and pitch
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingNext Best View - Visibility Kernel
Costmap-based approach1. max fringe points costmap, 2. occupied voxels costmap → minimumintersection
vϑ
φ
φ2
dmaxdmin
*Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingPoint Cloud Data Interpretation - Floor, Ceiling
• classifying points close to min/max along Z axis as floor and ceiling
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingPoint Cloud Data Interpretation - Walls, ROIs
• classifying planes perpendicular to X and Y axis orientation of the wholepoint cloud and close to min/max along those axes as walls
• classifying tabletops as those between 0.75m-1m heightPerceptionDejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingPoint Cloud Data Interpretation - Fixtures
*• remaining vertical surfaces and tabletops classified as areas of interest
• findind fixtures - handles and knobs
• various handle and knob detectors
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingDoor and Drawer Hypotheses
• using region growing• median intensity and median average distance
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingSystem Version 2
����������ABCD���DEAB
F�������
���D�������A���DEAB
F�������
��ECD��DEABF�������
���D���CD��E�� !�BBEB�F�����"�
��DE��!�DEAB#A$�!%���BEB�F�������
&�D��DEAB�A���!�'�BD� !�B�C
F�������
&�D��DEAB�A��(�B$!�CF�������
�)#*�#���+�!�+�BD�$�EB,BA�A-F���DEAB����
&AA���B$�&�����(.�AD/�C�CF�����"�
)-���D�&�D��DEAB�B$
��A�BEDEAB
)-���D�A��&�E!.�0C��&�D��DEAB��B$���A�BEDEABF �B����E�1��)��2���
&AA���B$�&�����(.�AD/�C�C��!E$�DEABF�����3�
�C4���!!�BD������
�����A��CCEB�
����������
A�BCDCB�E���
����������
*
[Pangercic CogSys2012, Pangercic IROS2012]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingRegistration, Reconstruction, Texture Re-projection
Testbed Kitchens Poisson surface re-construction
Blending-based tex-ture reprojection
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingLearning of Articulation Models - Arm Control
cabinet objectinteraction
observegripperpose yi
generatenext controlpoint xCEP
i
estimatearticulationmodel M̂, θ̂
[Sturm JAIR2011]
• Using impedance controller fromWillow Garage (pr2 cockpit stack)
• Initialization: gently pull thehandle backwards by moving theCartesian equilibrium pointtowards the robot
• Record the trajectory of robot’sgripper y1:n with yi ∈ SE(3)
VIDEO: *
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingLearning of Articulation Models - Model Fitting
cabinet objectinteraction
observegripperpose yi
generatenext controlpoint xCEP
i
estimatearticulationmodel M̂, θ̂
Iteratively:• (Re-)estimate the kinematic modelM ∈ {rigid, prismatic, rotational}
• Estimate model-specific parametervector θ ∈ Rd (encoding radius,rotation axis) of the articulatedobject:M̂, θ̂ = argmaxM,θ p(M, θ | y1:n)
• Fit the parameter vector of allmodel candidates using anMLESAC estimator
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingLearning of Articulation Models - Model Selection
cabinet objectinteraction
observegripperpose yi
generatenext controlpoint xCEP
i
estimatearticulationmodel M̂, θ̂
• Select the best model accordingthe Bayesian information criterion(BIC)
• Use the model to predict thecontinuation of the trajectory andto generate the next Cartesianequilibrium point xCEPn+1 .
• Finally, determine opening angle /opening distance
• Output: Kinematic model
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingDoor and Drawer Hypotheses Validation through Interaction
*PerceptionDejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingFinal Segmented Map
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingFinal KnowRob Map
Acquisition Code:bosch registration, bosch surface reconstruction,bosch texture reconstruction, mappingRepresentation Code:http://www.ros.org/wiki/knowrob
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
http://www.ros.org/wiki/knowrob
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingFuture Work
In collaboration with KTH:
• classification of rooms using shape, size and set of objects
• language for probabilistic representation and reasoning
• room and object novelty detection
• integration of planning
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Semantic MappingTutorial 1
In progress . . . - by end of June 2012.
• RGB + D + Plane-based + Edge-based Registration• Automatic segmentation of fixtures and dimensions• Automatic learning of articulation models
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Outline
1. Semantic MappingSemantic Mapping
2. Perception of Objects of Daily UseInteractive SegmentationObject Categorization - Graph-PartsObject Categorization -Hough VotingODUFinderObject Recognition using Barcodes
3. Dynamic Scenes and Spatio-Temporal Memory (Nico)GPU-based PerceptionCompression of Point CloudsUnstructured Information Management applications
4. Misc
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Interactive SegmentationSystem
��������A���BCD�E�
A���F���������F��������������B��������
A���F����
�����������������F�������������������F��
�B����D���������� !���F����
����F�����"����������F#��E
$�D%���E�����
������������F���&���������E
������������ABCABD���E��� �F�����ABC
C����CD�E�
C��������������
D����D�AB��EA����A�B
���A���A�B
�'('����
*
[Bersch IROS2012, Bersch RSS2012]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Interactive SegmentationContact Point and Push Direction
• Segmentation of cluttered scenes
• Contour estimation
• Finding concave corners
• Bisector gives a push direction
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Interactive SegmentationClustering Algorithm
t = 0
A0
A1
t = 1
t = 2
��������AB�CD
• Features u and v, depicted as redcircles, are randomly selected
• From their trajectories Su and Sv,a rigid transformation At iscalculated
• If u and v are on the same object,all other features will moveaccording the sequence of At
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Interactive SegmentationResults
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Interactive SegmentationFuture Work
• dealing with textureless and transparent objects
• real-world scenes
• integration of an arm planner
• constrained spaces
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Interactive SegmentationTutorial 2
Available on:http://ros.org/wiki/pr2_interactive_segmentation/Tutorials.
• Form 3 groups with 1 Kinect, 1 tripod and 2 objects each.
Thanks to Karol Hausman!
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
http://ros.org/wiki/pr2_interactive_segmentation/Tutorials
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization - Graph-PartsIntroduction
Figure: Overview of the process ofscanning segmenting and categorizingobjects in clutter
• Object categorization in clutteredscenes where accuratesegmentation can be difficult toachieve.
• Over-segmentation and multiplehypotheses better than relying ona single, possibly erroneoussegmentation
• Approach based on scene- orpart-graphs, using additive RGBDfeature descriptors and hashing
[Marton SC2012]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization - Graph-PartsObjectives
������������AB�C�DECF
�
�
�
FFF
A
�
��
���
���
FFF
�
FFF
FFF
�CC�A����A�D���
������D��
�
�
�
FF
A
D��D��D��D�D���D
��D��D��D�D���
��D��D��D�D���
�FF
��D��D��D�D���
D��D��D��D�D���D
��D��D��D�D���
��D��D��D�D���
�FF
��D��D��D�D���
D��D��D��D�D���D
��D��D��D�D���
��D��D��D�D���
�FF
��D��D��D�D���
FFF
FFF
FFF
FFF
FFF
FFF
FFF
• an efficient part-based objectclassification method for clutteredscenes, taking into accountrelations between parts;
• a graph-theoretic hashing methodthat allows model refinement whilehaving competitive classificationperformance;
• evaluation of geometric, color, andmulti-modal features andclassification approaches;
• exemplifying the advantages ofmultiple views, multiple segmentgroupings, and domain adaptation;
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization - Graph-PartsRadius-based Suface Descriptor (RSD)
Theory and Approximation
• Estimate minimum and maximum curvature radius from angle/distancepairs:
����
����
�
�
�
� �� �� �� �� �� �� �� � A�
�
�B�
�B�
�B�
�B�
�B�
�B�
�B�
�B
�BA
�
CDEF�����C����
�����E���������C�F������������ �CF�!�"��#����F�$�����F�������E�
%�"�������&�F'����F(����E���C��#�F'������FC
��CF������)��E���F�*��F��F(����+��,��)�CF����
Near-linear relation between distance and angle: the radius [m]
d(α) =√2r√
1− cos(α) ⇒ d(α) = rα+rα3
24+O(α5) ⇒ d = r · α
[Marton IROS2010:]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization - Graph-PartsRadius-based Surface Descriptor (RSD)
Principle Radii
• Local variation of normal angles by distance (similar to PFH and “spin images with
normals”):
Synthetic Data
plane sphere sphere side corner cylinder cylinder top cylinder side edge handle
Real Data
small cylinder medium cylinder big cylinder handle1 handle2 handle3
The tilt angles of the lines starting from bottom left corner correspond to the physical radii:
smallest tilt that still covers occupied cells to the min. radius, while the biggest to the max.
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization - Graph-PartsStatic Scenes
(a) Scene 1
(b) Scene 2
(c) Scene 3
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization - Graph-PartsScenes From Multiple Views
��� ��� ��� ��� ��� ���
��
��
��
��
��
�
ABCDEF
A�F
�������
������
Figure: As the camera is moved (left), multiple frames can be captured thatcover different parts of the objects in the scene (right).
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization -Hough VotingPart-Based Object Detection using Hough Voting and Model Fitting
Table Chair Sideboard
[RAM2011]
• Same task as before, with larger objects:furniture pieces have to be identified forwhich (similar, but not exactly matching)CAD models are available.
• Here pose estimation is added by modelmatching and geometric verification.
• Parts are considered separately, and only3DOF are possible, so more descriptivefeatures can be used (stattistics about thedistribution of the 3D points).
• Integration is achieved by parts beingcategorized (unsuperwised) and voting forlikely positions.
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization -Hough VotingPart Codebook of Example CAD Models
• Realistic scan simulation, unsupervised segmentation, building acodebook of parts, learning of spatial relations
• Probabilistic Hough voting to obtain likely object locations and aweighted list of their parts
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization -Hough VotingModel Fitting for Verification and 3DOF Pose
Assigning parts/points to objects and rejecting false positives
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization -Hough VotingReal-world Results: Office
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization -Hough VotingReal-world Results: Seminar Room
• remaining false matches due to high occlusionsPerception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization -Hough VotingImproving Results by Taking Multiple Scans
• Decreasing occlusion, increasing number of parts• Similarly as for small objects: merging the votes from multiple scans
• The model fitting and verification steps also do not assume oneviewpoint or one segmentation per scan
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization -Hough VotingFuture Work
• test on objects of daily use
• include color features
• explore the “power-of-data”
• integrate online building of models and sharing (e.g. via RoboEarth)
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Categorization -Hough VotingTutorial 3
Available on: http://www.ros.org/wiki/furniture_classification.Thanks to Vlad Usenko!
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
http://www.ros.org/wiki/furniture_classification
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
ODUFinderObject Recognition with ODUFinder - Idea
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
ODUFinderSystem
��������AB�CDE�AEF
�DBA���A������ACECE�
�E��AEF�����B��DF��CE�
�ABDD����C��
�ED ��F���A��
�CE�B�!DCE��B�D�F�AEF�C"A��
�ABDF��#DBA�C$A�CDE
AEF%�BDFCE�
��"AE�CB�E&D"A�CDE
'!!�AAEB��DF���
[Pangercic IROS2011, Pangercic IROS2012]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
ODUFinderVocabulary Tree and SIFT
+ + TF-IDF+Metric Score (e.g. L1)
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
ODUFinderOver-Segmentation-Based Object Candidate Detection
r2(x) = (r2max − r2min)(K(1− logsig(x−A))) + r2min (1)
where logsig is defined as:
logsig(x) =1
1 + e−x, (2)
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
ODUFinderOver-Segmentation-Based Object Candidate Detection - cont.
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
ODUFinderSemantic Map Prior
*Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
ODUFinderTopic-based Integration with KnowRob
• Automatically created ontology of>7500 objects from the online shopgermandeli.com
• Class hierarchy from categories +perishability, weight, price, origin,...
• Code:http://www.ros.org/wiki/
objects_of_daily_use_finder,KnowRob: comp germandeli
[Tenorth, RAM2011a]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
http://www.ros.org/wiki/objects_of_daily_use_finderhttp://www.ros.org/wiki/objects_of_daily_use_finder
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Recognition using BarcodesIdea
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Recognition using BarcodesSystem
��������AB�CDE�AEF
�DBA���A������ACECE�
�E��AEF�����B��DF��CE�
�ABDD����C��
�ED ��F���A��
�CE�B�!DCE��B�D�F�AEF�C"A��
�ABDF��#DBA�C$A�CDE
AEF%�BDFCE�
��"AE�CB�E&D"A�CDE
'!!�AAEB��DF���
[Pangercic IROS2012]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Recognition using BarcodesBarcode Decode
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Recognition using BarcodesServer Communication
• Request: http://www.barcoo.com/api/get_product_complete?pi=4101530002475&pins=ean&format=xml&source=ias-tum
• Response:
• Information parsed: image, category, product name, brandPerception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
http://www.barcoo.com/api/get_product_complete?pi=4101530002475&pins=ean&format=xml&source=ias-tumhttp://www.barcoo.com/api/get_product_complete?pi=4101530002475&pins=ean&format=xml&source=ias-tum
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Recognition using BarcodesIntegration with KnowRob
• OWL file-based
• Topic-based using json server(TBD)
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Object Recognition using BarcodesTutorial 4
Available on: http://www.ros.org/wiki/objects_of_daily_use_finder/Tutorials/Barcode-based%20object%20recognition.For this tutorial you will need to connect to the “lab” wireless network(pw: artificialintelligence) and disable your wired connection.Thanks to Nacer Khalil.
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
http://www.ros.org/wiki/objects_of_daily_use_finder/Tutorials/Barcode-based%20object%20recognitionhttp://www.ros.org/wiki/objects_of_daily_use_finder/Tutorials/Barcode-based%20object%20recognition
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Outline
1. Semantic MappingSemantic Mapping
2. Perception of Objects of Daily UseInteractive SegmentationObject Categorization - Graph-PartsObject Categorization -Hough VotingODUFinderObject Recognition using Barcodes
3. Dynamic Scenes and Spatio-Temporal Memory (Nico)GPU-based PerceptionCompression of Point CloudsUnstructured Information Management applications
4. Misc
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Outline
1. Semantic MappingSemantic Mapping
2. Perception of Objects of Daily UseInteractive SegmentationObject Categorization - Graph-PartsObject Categorization -Hough VotingODUFinderObject Recognition using Barcodes
3. Dynamic Scenes and Spatio-Temporal Memory (Nico)GPU-based PerceptionCompression of Point CloudsUnstructured Information Management applications
4. Misc
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Misc
List of things we did not talk about . . . :
• object tracking
• http://ros.org/wiki/cop
• plethora of algorithms in PCL (pointclouds.org)
• people/hands tracking
• DAFT feature
• . . .
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
http://ros.org/wiki/coppointclouds.org
Semantic Mapping Perception of Objects of Daily Use Dynamic Scenes and Spatio-Temporal Memory (Nico) Misc
Thank You
Contact:http://ias.cs.tum.edu/people/[pangercic|marton|blodow|balintbe]
Perception
Dejan Pangercic, Nico Blodow, Zoltan-Csaba Marton, Ferenc Balint-Benczedi
http://ias.cs.tum.edu/people/
Semantic MappingSemantic Mapping
Perception of Objects of Daily UseInteractive SegmentationObject Categorization - Graph-PartsObject Categorization -Hough VotingODUFinderObject Recognition using Barcodes
Dynamic Scenes and Spatio-Temporal Memory (Nico)GPU-based PerceptionCompression of Point CloudsUnstructured Information Management applications
Misc