Heterogeneous Data Fusion with Multiple Kernel Growing Self Organizing Maps
Maheshakya Wijewardena[1], Thimal Kempitiya[1], Thilina Rathnayake[1], Kevin Ratnasekera[1]Thushan Ganegedara[1], Amal Perera[1], Damminda Alahakoon[2]
[1]Department of Computer Science and Engineering, University of Moratuwa[2] La Trobe University, Bundoora Victoria
Overview
● About data fusion - what, why, how?● Data fusion in unsupervised learning● Introduction to Growing Self Organizing Maps● Kernel methods in Self organizing Maps● Data fusion with Multiple Kernel Growing Self Organizing Maps● Road traffic visual analysis with heterogeneous sources ● Conclusion
2
What is Data Fusion and why?
● Data fusion is the process of integrating multiple data sources and knowledge concerning the same real world object in order to obtain more ○ consistent○ accurate ○ robust○ descriptive information
“Combination of multiple sources to get improved information; i.e. less
expensive, higher quality or more relevant information.”
3
Data Fusion in Unsupervised Learning
● Data fusion in supervised learning is has become trivial:○ Kernel based methods○ Model ensembles○ …
● Unsupervised learning - learning is to discover hidden structure in unlabeled data○ Clustering is an ill-posed problem where solutions violate at least one of the
common assumptions about scale invariance, richness, and cluster consistency
○ Different solutions may look equally plausible without prior knowledge about the underlying data distributions
4
Goals● Investigation of how data fusion can be applied for unsupervised learning.● Employing data fusion in a self adapting structure which is capable of mutating
according to input data.● Building a single representation for multiple sources of heterogeneous data.● Improving road traffic visual analysis with data fusion to identify different levels of
congestion thoroughly.
5
Growing Self Organizing Maps● SOM is a vector quantization method and a dimensionality reduction
method: produces a fixed size 2D grid● GSOM: a dynamically growing SOM, an adapting structure according to
the input data○ Initialize the 2D map with a set of nodes (usually 4) with random weights.○ Grow new nodes as the aggregated error in existing nodes exceeds a specified
tolerance level.
○ Stop new node growing and smoothen the map.
●
6
Kernel Self Organizing MapsKernel ?A kernel is a function such that for all examples x and z in an input space X ⊂ Rd :K(x, z) =< φ(x), φ(z) > where φ is a linear or nonlinear mapping from the input spaceX to the feature space F, and < , > is an inner product.
● Two methods to employ kernels in SOM:○ Type I Kernel
■ Depends on the size of the map and input data size○ Type II Kernel
■ Can support online learning, no dependency on the size of the map● These two types have been proved equivalent.
7
Data Fusion with Multiple Kernel GSOM● For a single kernel:
○ Distance calculation
○ Determining winner node
○ Updating error
8
Data Fusion with Multiple Kernel GSOM● Employing multiple kernels:
○ Node vectors have the aggregated dimensionality of all input data sources○ A function Kd : Gd × Gd → R such that Kd is symmetric and positive kernel.○ A convex combination of kernels => A global kernel
○ Update rule of kernel coefficients: a stochastic gradient descent procedure
○ where V(t) is a standard rate that decreases with time t, x is input vector, w is node weight, LR is predefined learning rate and,
9
Data Fusion of Road Traffic video Data with Multiple Kernel GSOM
● UCSD highway traffic video data set which was taken over two days from a stationary camera overlooking I-5 in Seattle, Washington totaling to 20 minutes of time.
● 3 congestion levels:○ Heavy : 44○ Medium : 45○ Low : 165
11
Data Fusion of Road Traffic video Data with Multiple Kernel GSOM
● Video feature extraction: motion○ Optical flow based features - Histogram of Optical Flow descriptor(HOF)○ Change direction based features - Frame difference
● HOF - 400 to 1600 dimensions - Linear Kernel● Frame difference - 40 dimensions - Gaussian (RBF) Kernel
Data fusion in MK-GSOM involves both HOF features with Linear Kernel and Frame difference features with Gaussian Kernel.
12
Data Fusion of Road Traffic video Data with Multiple Kernel GSOMAfter training MK-GSOM, K-Means clustering has been applied on the map in order to identify clusters.
HOF
FD
13
Data Fusion of Road Traffic video Data with Multiple Kernel GSOM
Clusters formed by HOF features with Linear Kernel
Clusters formed by FD features with Gaussian Kernel
Clusters formed by Fusion of HOF features with Linear Kernel and FD features with Gaussian Kernel
14
Conclusions
● Data fusion mechanisms significantly improves the quality of the results of unsupervised learning algorithms.
● Multiple kernel algorithms provide means of smooth integration of fusion techniques into existing learning algorithms.
● MK-GSOM successfully adopts data fusion into its’ adaptive structure.● The experimental results of road traffic video data elaborate the effectiveness of this
approach in the application of heterogeneous data fusion.● Ability to identify novel clusters which otherwise could not have been identified with
individual data sources.● Challenges: Tuning hyperparameters of GSOM and selecting appropriate kernel.● Limitations:
○ When a large number of data sources are available, training time can grow rapidly.18
References[1] F. Castanedo. “A Review of Data Fusion Techniques”. In: The Scientific World Journal2013 (2013). [2] T. Kohonen, “The self-organizing map,” Proceedings of the IEEE, vol. 78, no. 9, pp. 1464–1480, 1990.[3] D. Alahakoon, S. Halgamuge, and B. Srinivasan, “Dynamic self-organizing maps with controlled growth for knowledge discovery,” Neural Networks, IEEE Transactions on, vol. 11, no. 3, pp. 601–614, 2000.[4] D. MacDonald and C. Fyfe, “The kernel self-organising map,” in Knowledge-Based Intelligent Engineering Systems and Allied Technologies, 2000. Proceedings. Fourth International Conference on, vol. 1. IEEE, 2000, pp. 317–320.[5] M. Olteanu, N. Villa-Vialaneix, and C. Cierco-Ayrolles, “Multiple kernel self-organizing maps,” in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2013, p. 83.[6] A. B. Chan and N. Vasconcelos, “Modeling, clustering, and segmenting video with mixtures of dynamic textures,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 30, no. 5, pp. 909–926, 2008.
19