Object Recognition with Global Features - PCL matchingPractical exercise Training Obtain synthetic...

Overview VFH CVFH Histogram matching Practical exercise

Object Recognition with Global FeaturesSeptember 25, 2011


Outline

1. Overview

2. VFH

3. CVFH

4. Histogram matching

5. Practical exercise

Aitor Aldoma / PCL :: Recognition


Training

I World knowledge: Training dataI Different sources: Real sensors, CAD models

I Input for training: Partial views of the objects



TrainingObtain synthetic partial views of CAD models with PCL

1 std::string PLYModel = std::string(argv[1]);2 float resx = atof(argv[2]);3 float resy = resx;4 typedef pcl::PointCloud<pcl::PointXYZ> Cloud;5 std::vector <Cloud, ... > views_xyz;6 std::vector < Eigen::Matrix4f, ... > poses;7 std::vector<float> entropies;89 pcl::visualization::PCLVisualizer vis;

10 vis.addModelFromPLYFile (PLYModel, "mesh1", 0);11 vis.setRepresentationToSurfaceForAllActors ();12 vis.renderViewTesselatedSphere (resx, resy, views_xyz,13 poses, entropies);



Training

I World knowledge: Training data

I Encode geometry for efficient matching: VFH, CVFH, ...

I Histogram database: VFH, CVFH, ...



Recognition

I Real scene from the Kinect:

I Segment and cluster objects of interest (plane extraction +Euclidean Clustering).

I Encode geometry: VFH, CVFH, ...I Match against DB: L1,L2,Chi-Square,... (FLANN)

I Object + viewpointAitor Aldoma / PCL :: Recognition


VFHViewpoint Feature Histogram

I Encodes the surface of the object and the viewpointI Relative to the centroid and the average of the normals

I Efficient to compute



VFHVFH Estimation in PCL

1 using namespace pcl;2 typedef PointCloud<PointXYZ> CloudXYZ;3 typedef PointCloud<Normal> CloudNormal;4 typedef PointCloud<VFHSignature308> VFHSig;5 typedef CloudXYZ::Ptr CloudXYZPtr;6 typedef CloudNormal::Ptr CloudNormalPtr;78 CloudXYZPtr cloud(new CloudXYZ());9 CloudNormalPtr normals(new CloudNormal());

10 // Compute normals11 ...1213 // Compute VFH14 VFHEstimation<PointXYZ, Normal, VFHSignature308> vfh;15 vfh.setInputCloud (cloud);16 vfh.setInputNormals (normals);17 KdTreeFLANN<PointXYZ>::Ptr tree (new KdTreeFLANN<PointXYZ> ());18 vfh.setSearchMethod (tree);19 VFHSig::Ptr vfhs (new VFHSig ());20 // Compute the features21 vfh.compute (*vfhs);



CVFHClustered Viewpoint Feature Histogram

I Designed for CAD model matchingI Deal with different training/detection data properties

I Synthetic training views have no defects.I Kinect data defects are not uniformly distributed.



CVFH

I Based on VFH.I Semi-global feature, multiple histograms for same view.I Scale dependant.I Smooth region clustering based on XYZ and Normal.I Smooth regions point used to build CS. Angular normal

distributions relative to this CS.



CVFHCVFH Estimation in PCL

1 using namespace pcl;2 typedef PointCloud<PointXYZ> CloudXYZ;3 typedef PointCloud<Normal> CloudNormal;4 typedef PointCloud<VFHSignature308> CVFHSig;5 typedef CloudXYZ::Ptr CloudXYZPtr;6 typedef CloudNormal::Ptr CloudNormalPtr;78 CloudXYZPtr cloud(new CloudXYZ());9 CloudNormalPtr normals(new CloudNormal());

10 // Compute normals11 ...1213 // Compute VFH14 CVFHEstimation<PointXYZ, Normal, VFHSignature308> CVFH;15 CVFH.setInputCloud (cloud);16 CVFH.setInputNormals (normals);17 KdTreeFLANN<PointXYZ>::Ptr tree (new KdTreeFLANN<PointXYZ> ());18 CVFH.setSearchMethod (tree);19 CVFHSig::Ptr cvfhs (new CVFHSig ());20 // Compute the features21 CVFH.compute (*cvfhs);



VFH matching

I World knowledge encoded as VFH histograms.I Actual scene objects encoded as VFH histograms.I Which are the objects in the scene?I Solution: compare object histograms with DB histograms

— Nearest neighbour searchI How: FLANN.I Metric: L1, L2, Chi-Square, ...



And in PCL?Also easy... 1) Load training features and build FLANN structure

1 typedef PointCloud<VFHSignature308> VFHSig;2 VFHSig::Ptr train_vfhs (new VFHSig ());34 pcl::KdTreeFLANN<VFHSignature308>::Ptr kdtree_;56 //Load VFH features from disk for models7 std::vector<std::string> filenames;8 //Fill filenames content.9 ...

1011 for (size_t i = 0; filenames.size() < n; ++i)12 {13 train_vfhs += *(loadPointCloud<VFHSignature308>14 (filenames[i], ""));15 }1617 kdtree_ = pcl::KdTreeFLANN<VFHSignature308>::Ptr (18 new pcl::KdTreeFLANN<VFHSignature308>);19 kdtree_->setInputCloud (descriptors_);



And in PCL?2) Match object to the DB

1 VFHSignature308 query;2 //compute VFH for the current object3 ...45 //Retrieve first nearest neighbour6 std::vector<int> nn_index (1);7 std::vector<float> nn_sqr_distance (1);8 kdtree_->nearestKSearch (query,1,nn_index,nn_sqr_distance);9 const int & best_match = nn_index[0];



Practical exerciseStep 1 - VFH computation

I In include/feature_estimation.h, modifycomputeGlobalDescriptor to compute a VFH descriptor.

1 GlobalDescriptorsPtr2 computeGlobalDescriptor (const PointCloudPtr & points,3 const SurfaceNormalsPtr & normals)4 {5 GlobalDescriptorsPtr global_descriptor;6 //Add your code here7 return (global_descriptor);8 }



Practical exerciseStep 2 - Build kdtree representation

I Training data under data/.I In include/object_recognition.h, modify populateDatabase

to build a kdtree representing the trained views.1 void2 populateDatabase (const std::vector<std::string> & filenames)3 {4 //Add your code here5 }



Practical exerciseStep 3 - Matching

I In include/object_recognition.h, modifyrecognizeAndAlignPoints to retrieve the nearest neighbourin the VFH kdtree.

1 PointCloudPtr2 recognizeAndAlignPoints (const PointCloudPtr & query_cloud)3 {4 ObjectModel query_object;5 constructObjectModel (query_cloud, query_object);6 //Add your code here7 PointCloudPtr output = alignModelPoints8 (models_[best_match], query_object, params_);9 }



Step 4Run the test code

I Run test_object_recognition.1 //src/test_object_recognition.cpp2 ...3 // Construct the ObjectDatabase4 ObjectRecognition obj_rec (params);5 obj_rec.populateDatabase (exemplar_filenames);67 // Find the object exemplar that best matches the query8 PointCloudPtr aligned_model_points =9 obj_rec.recognizeAndAlignPoints (query);

10 ...


Date post:	07-May-2018
Category:	Documents
Upload:	hoangbao
View:	218 times
Download:	1 times

Object Recognition with Global Features - PCL matchingPractical exercise Training Obtain synthetic...

Documents