Introduction to Local Binary Patterns€¦ · local binary pattern histogram fourier features....

[email protected] 1/16/2013 Lightning Talk: Local Binary Patterns

Introduction to Local Binary Patterns

Marc Norvig Principal Electronics Engineer

Embedded Video @ FLIR Systems, Inc.


Three Questions

• What are Local Binary Patterns?

• Why should I care?

• Where can I use them?


What are Local Binary Patterns?

• Computationally efficient texture descriptor

– Texture: • “Variation of data at scales smaller than the scale of interest” *

– Descriptor: • Compact representation that captures at least one essential

characteristic of the sample under examination

* (Petrou and Garcia Seville 2006, p. 1)


Why should I care?

• Rare technique that is good, fast and cheap

– Good • Competitive with state-of-the-art for many image processing tasks

• Robust to monotonic changes in illumination

(http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html #id22)


Why should I care?

• Rare technique that is good, fast and cheap

– Good • Competitive with state-of-the-art for many image processing tasks

• Robust to monotonic changes in illumination

– Fast • O(n)

– Cheap • Integer math, no multiply or divide

• No parameter tuning or image pre-processing required

• Compact descriptor (good performance with < 64 element vector)


Where can I use LBP?

• Facial Analysis – Face Recognition – Face Detection – Expression Recognition – Gender Classification

• Texture Analysis – Classification – Segmentation – Background Subtraction – Visual Inspection

Your Problem Here

• Image Analysis – Interest Region Description – Image Forensics – Image Retrieval – Biometrics

• Motion Analysis – Gesture Recognition – Lip Reading – Object Detection – Gaze Tracking


50

31 88

67

46

34

22

55 59

Implementation

Identify all pixels with luminance less than center pixel luminance

• 22 < 50

• 31 < 50

• 34 < 50

• 46 < 50

Remaining pixels are greater than or equal to center pixel luminance

• 55 > 50

• 59 > 50

• 67 > 50

• 88 > 50

Next, Binarize results


0 1

1

0

0

0

1 1

Implementation

• Invariant to global illumination change

• Invariant to local contrast magnitude

• Captures gradient sign which describes the brighter/darker “shape” of the pattern

• Since contrast is orthogonal, it can be an effective additional feature in some cases


1 10 00 101

1 10 00 101 binary

139 decimal

00

0

0

1 1

1

1

Implementation

Unroll the circular pattern to get LBP label


Implementation details

Circular pattern doesn’t fit standard square sampling grid very well



On-grid points use sample data directly



Off-grid points use interpolation



Nearest Neighbor is often sufficient


• Calculate LBP for every pixel in exemplar

• Histogram is model of micro-textures in exemplar

Texture Classification: Training

LBP label

Freq

uen

cy

Model


• Create LBP histogram from sample

• Find distance between sample & model histograms

Texture Classification: Recognition

Sample

LBP label

Freq

uen

cy

LBP label

Freq

uen

cy

Model


• Three commonly used distance measures

– Histogram Intersection

– Log-likelihood

– Chi-squared


χ2 𝑆,𝑀 = (𝑆𝑏 − 𝑀𝑏)

2

𝑆𝑏 +𝑀𝑏

𝐵

𝑏=1

𝐷 𝑆,𝑀 = min (𝑆𝑏 𝑀𝑏)

𝐵

𝑏=1

𝐿 𝑆,𝑀 = − 𝑆𝑏 log𝑀𝑏

𝐵

𝑏=1


• Machine learning finds threshold for classification


Sample1

LBP label

Freq

uen

cy

LBP label

Freq

uen

cy

Model

𝐷 𝑆,𝑀 = 0.93

Sample2 𝐷 𝑆,𝑀 = 0.28

LBP label

Freq

uen

cy


Dimensionality Reduction • 256 bin histogram → 256 dimension feature

– Promised: Fewer than 64 dimensions


Dimensionality Reduction • Uniform Local Binary Patterns

– Patterns with at most two contiguous regions

• Two patterns with one contiguous region

• Seven basic patterns with two contiguous regions

• Each basic pattern has eight orientations


Dimensionality Reduction • 58 Uniform Local Binary Patterns plus one

OTHERS


Dimensionality Reduction • Uniform Local Binary Patterns

– Reduces feature vector from 256 to 59 elements

• Helps with curse of dimensionality

– Natural images are ≈ 90% Uniform LBP

– Statistically more robust

• Produces better recognition in many applications

• Non-uniform patterns may not be robust to noise


• Not robust to rotation - histogram comparison fails

Dealing with Rotation

(Ahonen, Matas, He, Pietikäinen 2009)


• Solution: Use Fourier Transform of histogram

Dealing with Rotation

(Ahonen, Matas, He, Pietikäinen 2009)


• Divide face into blocks, concatenate LBP histograms

Face Recognition: Face Descriptor

(Ahonen, Hadid, Pietikäinen 2006)


• Compare test Subject to target Descriptor with Chi-squared distance measure:

j = image block, i = histogram bin

• Block-by-block weighting can improve performance

Face Recognition: Identification


χ2 𝑆, 𝐷 = (𝑆𝑖,𝑗 − 𝐷𝑖.𝑗)

2

𝑆𝑖,𝑗 + 𝐷𝑖,𝑗 𝑗,𝑖

χ2𝑤 𝑆, 𝐷 = 𝑤𝑗(𝑆𝑖,𝑗 − 𝐷𝑖.𝑗)

2

𝑆𝑖,𝑗 + 𝐷𝑖,𝑗 𝑗,𝑖


• Block weights example, 7x7 grid of blocks




• Results for FERET database



Expression Variation Illumination Variation


• Hierarchical splitting

Unsupervised Segmentation

(Ojala, Pietikäinen 1999)


• Agglomerative merging




• Pixelwise classification




• Natural image example




Lipreading

(Zhao, Barnard, Pietikäinen 2009)


Lipreading

(Zhao, Barnard, Pietikäinen 2009)


Computer Vision using Local Binary Patterns Springer 2011

University of Oulu (Finland) Center for Machine Vision Research:

http://www.cse.oulu.fi/CMV

LBP Face Recognition available in OpenCV

Many implementations at MATLAB Central

http://www.mathworks.com/matlabcentral/

Resources








• Ahonen T, Hadad A, Pietikäinen M. Face description with local binary pattens: application to face recognition. IEEE transactions on Pattern Analysis and Machine Intelligence 28. 2006:2037-2041

• Ahonen T, Matas J, He C, Pietikäinen M. Rotation invariant image description with local binary pattern histogram fourier features. Image Analysis, SCIA Proceedings, Lecture Notes in Computer Science 5575. 2009:61-70

• Ojala T, Pietikäinen M. Unsupervised texture segmentation using feature distribution. Pattern Recognition 32. 1999:477-478

• Petrou M, Garcia-Sevilla P. Image Processing: Dealing with Texture. Wiley; 2006

• Zhao G, Bernard M, Pietikäinen M. Lipreading with local spatiotemporal descriptors. IEEE Transactions on Multimedia vol. 11 no. 7. 2009:1254-1265

References

Date post:	25-Jun-2020
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

Introduction to Local Binary Patterns€¦ · local binary pattern histogram fourier features....

Documents