A Three-DimensionalRepresentation method for
Noisy Point Clouds based onGrowing Self-Organizing Maps
accelerated on GPUs
Author:
Supervisors:
Sergio Orts Escolano
Dr. José García Rodríguez
Dr. Miguel Ángel Cazorla Quevedo
Doctoral programme in technologies for the information society
2/79
Outline
Introduction
3D Representation using Growing Self-Organizing Maps
Improving keypoint detection from noisy 3Dobservations
GPGPU parallel implementations
Applications
Conclusions
3/79
Index Introduction
• Motivation
• Framework
• Goals
• Proposal
3D representation using growing self-organizing maps
Improving keypoint detection from noisy 3D observations
GPGPU parallel Implementations
Applications
Conclusions
4/79
Motivation Most computer vision problems require the use of an
effective way of representation
• Graphs, regions of interest (ROI), B-Splines, Octrees, histograms, …
• Key step for later processing stages: feature extraction, feature matching, classification, keypoint detection, …
Introduction Motivation
5/79
Motivation (II) 3D data captured from the real world
• Implicitly is comprised of complex structures and nonlinear models
The advent of low cost 3D sensors
• E.g. Microsoft Kinect, Asus Xtion, PrimeSense Carmine, …
• RGB and Depth (RGB-D) streams (25 frames per second (fps))
• High levels of noise and outliers
Only few works in 3D computer vision deal with real-time constraints
• 3D data processing algorithms are computationally expensive
Finding a 3D model with different features:
• Rapid adaptation
• Good quality of representation (topology preservation)
• Flexibility (non-stationary data)
• Robust to noisy data and outliers
Introduction Motivation
6/79
Motivation (III) 3D models of objects and scenes have been extensively used in
computer graphics
• Suitable structure for rendering and display
• Common graphics representations include quadric surfaces [Gotardo et al., 2004], B-spline surfaces [Gregorski et al., 2000], and subdivision surfaces
• Not general enough to handle such a variety of features: flexibility, adaption, noise-aware, … that are present in computer vision problems
(Left) Point cloud captured from a manufactured object (builder helmet). (Right) 3D mesh generated from the captured point cloud (post-processed)
Introduction Motivation
7/79
Framework Regional research project (GV/2011/034)
• Title: “Visual surveillance systems for the identification andcharacterization of anomalous behaviour” Project financed by theValencian Government in Spain
Regional research project (GRE09-16)
• Title: “Visual surveillance systems for the identification andcharacterization of anomalous behaviour in restricted environmentsunder temporal constraints” Project financed by the University ofAlicante in Spain
National research project (DPI2009-07144)
• Title: “Cooperative Simultaneous Localization and Mapping (SLAM) inlarge scale environments”
Research stay at IPAB – University of Edinburgh (BEFPI/2012/056)
• Title: “Real-time 3D feature estimation and keypoint detection ofscenes using GPGPUs”
Introduction Framework
8/79
Goals Proposal and validation of a 3D representation
model and a data fitting algorithm for noisy point clouds
• Deals with noisy data and outliers
• Flexible
• Dynamic (non stationary data)
• Topology preserving
An accelerated hardware implementation of the proposed technique
• Considerable speed-up regard CPU implementations
• Real-time frame rates
Introduction Goals
9/79
Goals (II) Validation of the proposed method on different
real computer vision problems handling 3D data:
• Robot vision: 6DoF Egomotion
• 3D Object recognition
• Computed-aided design/manufacturing (CAD/CAM)
Integration of 3D data processing algorithms in complex computer vision systems
• Filtering, downsampling, normal estimation, feature extraction, keypoint detection, matching, …
• The use of a GPU as a general purpose processor
Introduction Goals
10/79
Proposal Growing Self-Organizing Maps (GSOM) for 3D
data representation
• Low cost 3D sensors: noisy data
• Time-constrained conditions
• Applications: 3D computer vision problems
Hardware-based implementation of the proposed GSOM method
• General Purpose computing on Graphics Processing Units (GPGPU) paradigm
Integrate the entire pipeline of 3D computer vision systems using GPGPU paradigm
Introduction Proposal
11/79
Index Introduction
3D Representation using Growing Self-Organizing Maps
• Review
• 3D Growing Neural Gas network
• Experiments: Input space adaptation & normal estimation
• Extensions of the GNG algorithm
Improving keypoint detection from noisy 3D observations
GPGPU Parallel Implementations
Applications
Conclusions
12/79
Review SOMs were originally proposed for data clustering and pattern
recognition purposes [Kohonen, 1982, Vesanto and Alhoniemi, 2000, Dittenbach et al., 2001]
As the original model had some drawbacks due to the pre-established topology of the network, growing approaches were proposed in order to deal with this problem
Growing Neural Gas network has been successfully applied to the representation of 2D shapes in many computer vision problems [Stergiopoulou and Papamarkos, 2006, García-Rodríguez et al., 2010, Baena et al., 2013]
Already exist approaches that use traditional SOMs for 3D data representation: [Yu, 1999, Junior et al., 2004]
• Difficulties to correctly approximate concave structures
• High computational cost
• Synthetic data
3D Representation GSOM Review
13/79
Review (II) Moreover, there exist some limitations and unexplored
topics in the application of SOM-based methods to 3Drepresentation:
• Majority of these works do not consider the high computational cost of the learning step
• Do not guarantee response within strict time constraints
• Assumed perfect point clouds that were noise-free
• Data fusion (Geometric information + colour information) has not been considered
• Not dealing with point cloud sequences, only single-shot data
3D Representation GSOM Review
14/79
3D Growing Neural Gas Network Obtaining a reduced and compact representation of 3D data
• Self Organizing Maps – Growing Neural Gas
Growing Neural Gas Algorithm (GNG) [Fritzke, 1995]
• Incremental training algorithm
• Links between the units in the network are established through Competitive Hebbian Learning (CHL)
• Topology Preserving Graph
• Flexibility, growth, rapid adaption and good quality of representation.
GNG representation is comprised of nodes (neurons) and connections (edges)
• Wire-frame model
Initial, intermediate and final states of the GNG learning algorithm
3D Representation GSOM GNG network
15/79Highly Parallelizable
GNG algorithm Input data is defined in ℝ𝒅
• For 3D representation d=3
Adaption: Reconfiguration module
• Random patterns are presented to the network
Growth: It starts with two neurons and new neurons are inserted
Flexibility: neurons and connections may be removed during the learning stage
This process is repeated until an ending condition is fulfilled:
• Number of neurons/patterns
• Adaptation error threshold
3D Representation GSOM GNG algorithm
16/79
Experiments: Data acquisition Algorithm independent of the data source
We managed 3D data that can come from different sensors
• Laser unit, a LMS-200 Sick mounted on a sweeping unit.
o Outdoor environments. Its range is 80 metres with an error of 1millimetre per metre
• Time-of-Flight camera
o SR4000 camera. It has a range of 5-10 metres
o The accuracy varies depending on the characteristics of the observed scene, such as objects reflectivity and ambient lighting conditions
o Generation of point clouds during real time acquisition
• Range camera: structured light, Microsot Kinect device
o RGB-D information. Indoor environments. Its range is from 0.8 to 6metres
o Generation of point clouds during real time acquisition
3D Representation GSOM Experiments: Data
17/79
Experiments: Data acquisition
Mobile robots used for experiments.Left: Magellan Pro unit usedfor indoors. Right: PowerBot used for outdoors.
3D sensors used for experiments. From left to right: Sick laser unitLMS-200, Time-Of-Flight SR4000 camera and Microsoft Kinect
3D Representation GSOM Experiments: Data
18/79
Experiments: Data Sets
Some public data sets have been used to validate the proposed method:
• Well known Stanford 3D scanning repository. It contains complete models that have been previously processed (noise free)
• Dataset captured using the Kinect sensor. Released by the Computer Vision Laboratory of the University of Bologna [Tombari et al., 2010a]
• Own dataset obtained using three previously mentioned 3D sensors
3D Representation GSOM Experiments: Data
19/79
Blensor software: It allowed us to generate synthetic scenes and to obtain partial views of the generated scene as if a Kinect device was used
• It provided us with ground truth information for experiments
Experiments: Data Sets
3D Representation GSOM Experiments: Data
Simulated scene Simulated scene + Gaussian noise
20/79
Experiments
GNG method has been applied to 3D data representation
• Input space adaptation
• Noise removal properties
Extensions of the GNG based algorithm
• Colour-GNG
• Sequences management
• 3D Surface reconstruction
3D Representation GSOM Experiments
21/79
Experiments: GNG 3D representation
Applying GNG to Kinect data Applying GNG to SR4000 data
Applying GNG to laser data
3D Representation GSOM Exp: GNG 3D representation
22/79
Applying GNG to Kinect data
3D Representation GSOM Exp: GNG 3D representation
Experiments: GNG 3D representation
23/79
GNG method obtains better adaptation to the input space than other filtering methods like Voxel Grid technique
• Obtains lower adaptation (Mean Squared Error (MSE) )
• Tested on CAD models and simulated scenes
Input space adaptation MSE for different models (metres). Voxel Grid versusGNG. Numbers in bold provide the best results.
3D Representation GSOM Exp: Input space Adaptation
Experiments: Input space adaptation
Lower error
24/79
Noisy model σ = 0.4 Original CAD model
GNG representation Voxel grid representation
Filtering quality using 10,000 nodes. GNG vs Voxel Grid comparison
3D Representation GSOM Exp: Input space Adaptation
Experiments: Input space adaptation (II)
25/79
Normal estimation over noisy input data
Surface normals are important properties of a geometric surface, and are heavily used in many areas such as computer vision and computer graphics
Normal or curvature estimation can be affected by the presence of noise
Representation obtained using the GNG method allows to compute more accurate normal information
3D Representation GSOM Exp: Normal estimation
Experiments: Normal estimation
26/79
Experiments: Normal estimation (II)
Top:. Normal estimation on a filtered point cloud produced by theGNG method. Bottom: Normal estimation on a raw point cloud.
Normals are considered more stable as their distribution is smooth and also they have less abrupt changes in their directions
3D Representation GSOM Exp: Input space Adaptation
27/79
Extension: Colour-GNG
Modern 3D sensors provide colour information (e.g. Kinect, Carmine, Asus Xtion, … )
GNG is extended considering colour information during the learning step
• Input data is defined in ℝ𝒅 where d = 6
• Colour information is considered during the weight adaptation step but it was not included in the CHL (winning neurons) process
o We are still focus on topology preservation
• Winning neuron step only compute Euclidean distance using x,y,z components
• No post-processing steps are required as neurons’ colour is obtained during the learning process
3D Representation GSOM Extensions: Colour-GNG
28/79
Extension: Colour-GNG (II)
(a),(b),(c) show original point clouds. (d),(e),(f) show downsampled pointclouds using the proposed method
3D Representation GSOM Extensions: Colour-GNG
29/79
Extension: Colour-GNG (III)
Mario figure is down-sampled using the Colour-GNG method
Results are similar to those obtained with the colour interpolation post-processing step
3D Representation GSOM Extensions: Colour-GNG
30/79
Extension: Sequences management
An improved workflow to manage point cloud sequences using the GNG algorithm
Extension of the GNG for processing sequences of point clouds
It is not required to restartthe learning
It provides a speed-up in the runtime as neurons are kept between point clouds
This extension was applied in a mobile robotics application
3D Representation GSOM Extensions: Sequences
31/79
Extension: 3D Surface Reconstruction
Three-dimensional surface reconstruction is not considered in the original GNG algorithm as it only generates wire-frame models
[Holdstein and Fischer, 2008, Do Rego et al., 2010, Barhak, 2002] have already considered the creation of 3D triangular faces modifying the original GNG algorithm
• Post-processing steps are required for avoid gaps and holes in the final mesh
We extended the CHL developing a method able to produce full 3D meshes during the learning stage
• No post-processing steps are required
• A new learning scheme was developed
3D Representation GSOM Extension: 3D Reconstruction
32/79
Extension: 3D Surface Reconstruction (II)
Considered situations for edge and face creationduring the extended CHL
Avoid non-manifold andoverlapping edges
• More than 2 neighbours
• it is checked if the face tobe created already exist
A face is created whenever the already existing edges or the new ones form a triangle
The neuron insertion process was also modified
3D Representation GSOM Extension: 3D Reconstruction
33/79
Edge removal constraint based on the Tales sphere
Left: The triangle formed by these 3neurons is close to a right triangle
Right: The edge connecting s1 and ni
is removed as the angle formed
Face creation during the insertion of new neurons
Left: neuron insertion between the neuron q with highest error and its neighbour f with highest error.
Right: four new triangles and two edges are created considering r, qand f.
3D Representation GSOM Extension: 3D Reconstruction
Extension: 3D Surface Reconstruction (III)
34/79
Extension: 3D Surface Reconstruction (IV)
Post-processing steps were avoided causing gaps and holes inthe final 3D reconstructed models
3D Representation GSOM Extension: 3D Reconstruction
Different views of reconstructed models using an existing GNG-based method [Do Rego et al., 2010] for surface reconstruction
35/79
Extension: 3D Surface Reconstruction (V)
Reconstructed models using our extended GNG method for facereconstruction and without applying post-processing steps
3D Representation GSOM Extension: 3D Reconstruction
36/79
Extension: 3D Surface Reconstruction (VI)
Top: 3D model of a person(Kinect sensor).
Bottom: digitized foot. (footdigitizer)
Left: Noisy point clouds captured using the Kinect sensor.
Right: 3D reconstruction using the proposed method.
3D Representation GSOM Extension: 3D Reconstruction
37/79
Index Introduction
3D Representation using Growing Self-Organizing Maps
Improving keypoint detection from noisy 3D observations
• Review
• Improving keypoint detection
• Correspondences matching
• Results
GPGPU Parallel Implementations
Applications
Conclusions
38/79
Review
General System Overview
Motivation: dealing with noisydata obtained from 3D sensorsas the Microsoft Kinect or lasers
Result: Improving 3Dkeypoint detection and therefore registrationproblem
Filtering and down-sampling have become essential steps in3D data processing
We propose the use of the GNG algorithm for downsamplingand filtering 3D data
Beneficial attributes will be demonstrated through the 3Dregistration problem
Improving Keypoint detection Review
39/79
Review (II)Registration: Aligning various 3D point cloud data views into a complete model
Pairwise matching
Improving Keypoint detection Review
40/79
3D Keypoint detection Applying keypoint detection algorithms to filtered point
clouds
State-of-the-art 3D keypoint detectors
• Different techniques are used to test and measure theimprovement achieved using GNG method to filterand downsample input data
Improving Keypoint detection Keypoint detection
41/79
3D Keypoint detection (II)
3D Keypoint detectors
• SIFT3D: using depth as the intensity value in theoriginal SIFT algorithm
• Harris3D: use surface normals of 3D points
• Tomasi3D: performs eigenvalue decomposition overcovariance matrix
• Noble3D: evalutes the ratio between the determinantand the trace of the covariance matrix
Improving Keypoint detection Keypoint detection
42/79
3D Feature descriptors
Feature descriptors are calculated over detected keypointsto perform feature matching
• FPH and FPFH: based on an histogram of the differences of angles between the normals of the neighbour points
• SHOT and CSHOT: a spherical grid centered on the point divides the neighbourhood so that in each grid bin a weighted histogram of normals is obtained
FPFH CSHOT
Improving Keypoint detection Feature descriptors
43/79
Feature matching (II) Correspondences between keypoints are validated
through RANSAC algorithm, rejecting thoseinconsistent correspondences
Improving Keypoint detection Feature matching
44/79
Results: Feature matching
Correspondences matching computed on different input data
Top: raw point clouds
Middle: reduced representation using the GNG (20, 000 neurons)
Bottom: reduced representation using the GNG (10, 000 neurons)
RANSAC is used to reject wrong matches
Improving Keypoint detection Results
Raw 3D data
GNG 20,000 nodes
GNG 10,000 nodes
45/79
Results: Transformation errors
Mean, median, minimum and maximum RMS*2 errors of the estimated transformations using different keypoint detectors. (metres).
Lowest transformation errorLowest max errors
*2 Root Mean Square - Transformation error
Improving Keypoint detection Results
*1 Uniform Sampling
*1
46/79
Index Introduction
3D Representation using Growing Self-Organizing Maps
Improving keypoint detection from noisy 3D observations
GPGPU Parallel Implementations
• Graphics Processing Unit
• GPU-based implementation of the GNG algorithm
• GPU-based tensor extraction algorithm
• Conclusions
Applications
Conclusions
47/79
Graphics Processing Unit
GPUs have democratized High Performance Computing (HPC)• Massively parallel processors on a commodity PC
• Great ratio FLOP/€ compared with other solutions
However, this is not for free• New programming model
• Algorithms need to be re-thought and re-implemented
Growing Neural Gas algorithm is computationally expensive• Most computer vision applications are time-constrained
• A GPGPU implementation is proposed
GPGPU Implementations GPUs
48/79
Graphics Processing Unit (II) More transistors for data processing
GPU are comprised of streaming multi-processors
High GPU Memory bandwidth
GPGPU: General Purpose computing on Graphics Processors Units
Key hardware feature is that the cores are SIMT
• Single instruction multiple threads
G80CUDA NVIDIAs Architecture
GPGPU Implementations GPUs
49/79
GPU Implementation GNG Stages of the GNG algorithm
that are highly parallelizable
• Calculate distance to neurons for every pattern
• Search winning neurons
• Delete neurons and edges
• Search neuron with max error
Other improvements
• Avoid memory transfers between CPU and GPU
• Hierarchy of memories
Highly parallelizable stages
GPGPU Implementations GPU Implementation GNG
50/79
Parallel Min/Max Reduction A parallel Min/Max reduction
that computes the Min/Max of large arrays of values (Neurons)
Strategy used to find Min/Max winning neurons
Reduce linear computational cost n of the sequential version to the logarithmic cost log(n)
Provides better performance for a large number of neurons
Proposed version: 2MinParallelReduction
• Extended version to obtain 2minimum values in the same number of steps
Example of Parallel Reduction Algorithm
GPGPU Implementations GPU Implementation GNG
51/79
Experimental setup Main GNG Parameters:
• ~0-20,000 neurons and a maximum λ (entries per iteration) of 1,000-2,000
• Others parameters have been fixed based on previous works [García-Rodriguez et al., 2012]
o єw = 0.1 , єn = 0.001
o amax = 250, α = 0.5 , β = 0.0005
Hardware
• GPUs: CUDA capable devices used in experiments
• CPU: single thread and multiple thread implementations were tested
o Intel Core i3 540 3.07Ghz
GPGPU Implementations Experimental Setup
52/79
Experiments: 2MinParallelReduction
Runtime and speed-up using CPU and GPU implementations
GPGPU Implementations Exp: 2MinParallelReduction
53/79
Experiments: GNG learning runtime
GPU and CPU GNG runtime, and speed-up for different devices
GPGPU Implementations Exp: GNG Runtime
54/79
Experiments: Hybrid version
Example of CPU and Hybrid GNG runtime for different devices
CPU implementation was faster for small network sizes
• We developed an hybrid implementation
• GPU version automatically starts computing when it is detected that computing time is lower than the one obtained by the CPU
GPGPU Implementations Exp: Hybrid version
55/79
GPU-based Tensor extraction algorithm
Time-constrained 3D feature extraction
• Most feature descriptors cannot be computed online due to their high computational complexity
o 3D Tensor - [Mian et al., 2006b]
o Geometric Histogram - [Hetzel et al., 2001]
o Spin Images - [Andrew Johnson, 1997]
An accelerated GPU-based implementation of an existing 3D feature extraction algorithm is proposed
• Accelerate entire pipeline of RGB-D based computer vision systems
• Highly parallelizable• Geometrical properties• Invariant to linear
transformations
GPGPU Implementations GPU Feature extraction
56/79
GPU-based Tensor extraction algorithm (II)
The surface area of the mesh intersecting each bin of the grid is the value of the tensor element
As many threads as voxels are launched in parallel where each GPU thread represent a voxel (bin) of the grid
Each thread computes the area of intersection between the mesh and its corresponding voxel using Sutherland Hodgman’s polygon clipping algorithm. [Foley et al., 1990]
GPGPU Implementations GPU Feature extraction
57/79
Experiments: Performance
Runtime comparison and speed-up obtained for proposed methods using different graphics boards
GPGPU Implementations Exp: performance
58/79
Index Introduction
3D Representation using Growing Self-Organizing Maps
Improving keypoint detection from noisy 3D observations
GPGPU Parallel Implementations
Applications
• Robotics
• Computer Vision
• CAD/CAM
Conclusions
59/79
Applications
Different cases of study where the GNG-based method proposed in this PhD thesis was applied to different areas
• Robot Vision
o 6DoF Pose Registration
• Computer Vision
o 3D object recognition under cluttered conditions
• CAD/CAM
o Rapid Prototyping in Shoe Last Manufacturing
Applications Exp: performance
60/79
Robotics: 6DoF pose registration
The main goal of this application is to perform six degrees of freedom (6DoF) pose registration in semi-structured environments
• Man-made indoor and outdoor environments
We combined our accelerated GNG-based algorithm with the method proposed in [Viejo and Cazorla, 2013]
• Planar patches extraction
It provides a good starting point for Simultaneous Location and Mapping (SLAM)
GNG was applied directly to raw 3D data
Applications Robotics
61/79
Robotics: 6DoF pose registration (II)
Left: planar patches extracted from SR4000 camera
Right: filtered data using the GNG network: more planar patches areextracted
Applications Robotics
Without GNG GNG
62/79
Robot trajectory
Robotics: 6DoF pose registration (III)
Planar based 6DoF pose registration results
Left image shows map building results without using GNG while theresults shown on the Right are obtained after computing a GNG mesh
Applications Robotics
Without GNG GNG
63/79
3D Object Recognition
The main goal of this application is the recognition of objects under time constraints and cluttered conditions
The GPU-based of the semi-local surface feature (tensor)is successfully used to recognize objects in cluttered scenes
A library of models is constructed offline, storing all extracted 3D tensors in an efficient way using a hash table
• Multiple views
Applications 3D Object recognition
64/79
3D Object Recognition (II)
Object recognition is performed on sceneswith different level of occlusion
Objects are occluded by other objects stored and non-stored in the library
The averaged recognition rate was 84%, wrong matches 16% and false negatives0%
Applications 3D Object recognition
65/79
Scene 2
3D Object Recognition (III) GPU-based 3D feature implementation is successfully used in a 3D object
recognition application
Parallel matching is performed on the GPU: correlation function
Implemented prototype took around 800 ms with a GPU implementation to perform 3D object recognition of the entire scene
Applications 3D Object recognition
66/79
Rapid Prototyping in Shoe Last Manufacturing
With the advent of CAD/CAM and rapid acquisitiondevices it is possible to digitize old raised shoe lasts forreusing them in the shoe last design software
Applications CAD: Rapid Prototyping
Process to reconstruct existing shoe lasts and computing topology preservation error regard the original CAD design
67/79
Rapid Prototyping in Shoe Last Manufacturing (II)
The main goal of this research is to obtain a grid of points that isadapted to the topology of the footwear shoe last from a sequenceof sections with disorganized points acquired by sweeping anoptical laser digitizer
Typical sequence of sections of a shoe last. Noisy data obtained from the digitizer
Applications CAD: Rapid Prototyping
68/79
Rapid Prototyping in Shoe Last Manufacturing (III)
Voxel Grid versus GNG: Mean erroralong different sections of the shoe last
Applications CAD: Rapid Prototyping
69/79
Rapid Prototyping in Shoe Last Manufacturing (IV)
GNG vs VG topological preservation comparison
Applications CAD: Rapid Prototyping
3D Reconstruction GNG 3D Reconstruction VG
GNG nodes
VG nodes
Input space
70/79
Index Introduction
3D Representation using Growing Self-Organizing Maps
Improving keypoint detection from noisy 3D observations
GPGPU Parallel Implementations
Applications
Conclusions
• Contributions
• Future work
• Publications
71/79
Contributions Contributions made in the topic of research:• Proposal of a new method to create compact, reduced and
efficient 3D representations from noisy data
‐ Development of a GNG-based method capable to deal with different sensors
‐ Extension of the GNG algorithm to consider colour information
‐ Extension of the GNG algorithm for 3D surface reconstruction
‐ Sequences management
‐ Integration of the proposed method in 3D keypoint detection algorithms improving their performance
‐ GPU-based implementation to accelerate the learning process of the GNG and NG algorithms.
‐ An hybrid implementation of the GNG algorithm that takes advantage of the CPU and GPU processors
Conclusions Contributions
72/79
Contributions (II)• Integration of 3D data processing algorithms in complex
computer vision systems:
‐ Normal estimation has been ported to the GPU considerably decreasing its runtime
‐ Point cloud triangulation has been ported to the GPU accelerating its runtime
‐ A GPU time-constrained implementation of a 3D feature extraction algorithm
• Application of the proposed method in various real computer vision applications:
‐ Robotics: Localization and mapping: 6DoF pose registration
‐ Computer vision: 3D object recognition under cluttered conditionsCAD/CAM: rapid prototyping in shoe last manufacturing
Conclusions Contributions
73/79
Future work Other improvements on the GPU implementation of the
GNG algorithm:
• Using multi-GPU to manage several neural networks simultaneously
• Distributed computing
• Testing new architectures: Intel Xeon Phi [Fang et al., 2013a]
• Generating random patterns using GPU
More applications of the accelerated GNG algorithm will be studied in the future
• Clustering multi-dimensional data: Big Data
• Medical Image Reconstruction
Extension of the real-time implementation of the 3D tensor
• Visual features extracted from RGB information
• Improve implicit keypoint detector used by the 3D tensor
Conclusions Future work
74/79
Publications• 4 JCR Journal paperso “Real-time 3D semi-local surface patch extraction using GPGPU”
S. Orts-Escolano, V. Morell, J. Garcia-Rodriguez, M. Cazorla, R.B. Fisher; Journal of Real-Time Image Processing. December 2013; ISSN: 1861-8219; Impact Factor: 1.156 (JCR 2012)
o “GPGPU implementation of growing neural gas: Application to 3Dscene reconstruction”S. Orts, J. García Rodríguez, D. Viejo, M. Cazorla, V. Morell; J.Parallel Distrib. Comput. 72(10); pp: 1361-1372 (2012); ISSN: 0743-7315; ImpactFactor: 1.135 (JCR 2011)
o “3D-based reconstruction using growing neural gas landmark: application to rapid prototyping in shoe last manufacturing”A. Jimeno-Morenilla, J. García-Rodriguez, S. Orts-Escolano, M. Davia-Aracil; The International Journal of Advanced Manufacturing Technology: May 2013. Vol 69. pp: 657-668; ISSN: 0268-3768; Impact Factor: 1.205 (JCR 2012)
o “Autonomous Growing Neural Gas for applications with time constraint: Optimal parameter estimation” J. García Rodríguez, A. Angelopoulou, J. M. García Chamizo, A. Psarrou, S. Orts-Escolano, V. Morell-Giménez; Neural Networks 32: pp: 196-208 (2012), ISSN: 0893-6080; Impact Factor: 1.927 (JCR 2012)
Conclusions Publications
75/79
Publications (II)• International conferenceso “Point Light Source Estimation based on Scenes Recorded by a RGB-D
camera”B. Boom, S. Orts-Escolano, X. Ning, S. McDonagh, P. Sandilands, R.B. Fisher; British Machine Vision Conference, BMVC 2013, Bristol, UK. Rank B
o “Point Cloud Data Filtering and Downsampling using Growing Neural Gas”S. Orts-Escolano, V. Morell, J. Garcia-Rodriguez and M. Cazorla; International Joint Conference on Neural Networks, IJCNN 2013, Dallas, Texas. Rank A
o “Natural User Interfaces in Volume Visualisation Using Microsoft Kinect”A. Angelopoulou, J. García Rodríguez, A.Psarrou, M. Mentzelopoulos, B. Reddy, S. Orts-Escolano, J.A. Serra. International Conference on Image Analysis and Processing, ICIAP2013, Naples, Italy: 11-19. Rank B
o “Improving Drug Discovery using a neural networks based parallel scoring functions”H. Perez-Sanchez, G. D. Guerrero, J. M. Garcia, J. Pena, J. M. Cecilia, G. Cano, S. Orts-Escolano and J. Garcia-Rodriguez. International Joint Conference on Neural Networks, IJCNN 2013, Dallas, Texas. Rank A
Conclusions Publications
76/79
Publications (III)• International conferenceso “Improving 3D Keypoint Detection from Noisy Data Using Growing
Neural Gas”J. García Rodríguez, M. Cazorla, S. Orts-Escolano, V. Morell. International Work-Conference on Artificial Neural Networks, IWANN 2013, Puerto de la Cruz, Tenerife, Spain: 480-487. Rank B
o “3D Hand Pose Estimation with Neural Networks”J. A. Serra, J. García Rodríguez, S. Orts-Escolano, J. M. García Chamizo, A. Angelopoulou, A. Psarrou, M. Mentzelopoulos, J. Montoyo-Bojo, E. Domínguez. International Work-Conference on Artificial Neural Networks, IWANN 2013, Puerto de la Cruz, Tenerife, Spain: 504-512. Rank B
o “3D Gesture Recognition with Growing Neural Gas” J. A. Serra-Perez, J. Garcia-Rodriguez, S. Orts-Escolano, J. M. Garcia-Chamizo, A. Angelopoulou, A. Psarrou, M. Mentzeopoulos, J. Montoyo Bojo. International Joint Conference on Neural Networks. IJCNN 2013, Dallas, Texas. Rank A
o “Multi-GPU based camera network system keeps privacy using Growing Neural Gas”S. Orts-Escolano, J. García Rodríguez, V. Morell, J.Azorín López, J. M. García Chamizo. International Joint Conference on Neural Networks (IJCNN) 2012, Brisbane, Australia, June: 1-8. Rank A
Conclusions Publications
77/79
Publications (IV)• International conferenceso “A study of registration techniques for 6DoF SLAM”
V. Morell, M.Cazorla, D. Viejo, S. Orts-Escolano, J. García Rodríguez. International Conference of the Catalan Association for Artificial Intelligence, CCIA 2012, University of Alacant, Spain: 111-120. Rank B
o “Fast Autonomous Growing Neural Gas”J. García Rodríguez, A. Angelopoulou, J. M. García Chamizo, A. Psarrou, S. Orts, V. Morell. International Joint Conference on Neural Networks, IJCNN 2011, San Jose, California: 725-732. Rank A
o “Fast Image Representation with GPU-Based Growing Neural Gas”J.García Rodríguez, A. Angelopoulou, V. Morell, S. Orts, A. Psarrou, J. M. García Chamizo. International Work-Conference on Artificial Neural Networks, IWANN 2011, Torremolinos-Málaga, Spain: 58-65. Rank B
o “Video and Image Processing with Self-Organizing Neural Networks”J. García Rodríguez, E. Domínguez, A. Angelopoulou, A. Psarrou, F. J. Mora-Gimeno, S. Orts, J. M. García Chamizo. International Work-Conference on Artificial Neural Networks, IWANN 2011, Torremolinos-Málaga, Spain: 98-104. Rank B
Conclusions Publications
78/79
Publications (V)• National conferenceso “Procesamiento de múltiples flujos de datos con Growing Neural Gas
sobre Multi-GPU” S. Orts-Escolano, J. García-Rodríguez, V. Morell-Giménez. Jornadas de Paralelismo JP, Elche, España, 2012
• Book chapterso “A Review of Registration Methods on Mobile Robots”
V. Morell-Gimenez, S. Orts-Escolano, J. García Rodríguez, M. Cazorla, D. Viejo. Robotic Vision: Technologies for Machine Learning and Vision Applications. IGI GLOBAL
o “Computer Vision Applications of Self-Organizing Neural Networks”J. García-Rodríguez, J. M. García-Chamizo, S. Orts-Escolano, V. Morell-Gimenez, J.Serra-Perez, A. Angelolopoulou, M. Cazorla, D. Viejo. Robotic Vision: Technologies for Machine Learning and Vision Applications. IGI GLOBAL
• Poster presentationso “6DoF pose estimation using Growing Neural Gas Network”
S.Orts, J. Garcia-Rodriguez, D. Viejo, M. Cazorla, V. Morell, J. Serra. 5th International Conference on Cognitive Systems, Cogsys 2012, TU Vienna, Austria
o “GPU Accelerated Growing Neural Gas Network”S. Orts, J. Garcia, V.Morell. Programming and Tuning Massively Parallel Systems, PUMPS 2011,Barcelona, Spain. (Honorable Mention by NVIDIA)
Conclusions Publications
79/79
This presentation is licensed under a Creative Commons Attribution-
NonCommercial-ShareAlike 4.0 International License.