Anil K. Jain and Kai Cao Michigan State University
Project # 12S-05W-12
Automatic Segmentation of Latent
Fingerprints
Fingerprint Types
Rolled fingerprint Plain fingerprint latent fingerprint
AFIS achieved a rank-1 identification rate of 99.4% (NIST FpVTE 2003)
Latent matcher achieved a rank-1 identification rate of 63.4% (NIST ELFT 2012)
2
Challenges in Latent Matching
Unclear ridges Partial fingerprint
Large distortion Complex background
Reliable
feature
extraction
Robust
feature
matching
3
Goals of Latent Segmentation (Cropping, Region of Interest, Foreground)
• Develop an automatic algorithm to separate friction ridge
pattern from background
• Define a ridge quality measure
• Enhance friction ridge structure in foreground
• Provide a confidence value for segmentation
4 (a) A latent from
NIST SD27
(b) Segmented &
enhanced image of (a) (c) Mate rolled print
Algorithm
Input latent
image
Fine estimation
Segmentation and
enhancement result
Confidence
level (CL)
estimation
CL ≤ TH Reject
Texture
extraction
0 ≤CL ≤1
CL > TH
Dictionary
learning
Dictionary
learning
(0, π/16]
(π/16, 2π/16]
(15π/16, π]
Patch size:
32×32
Patch size:
64×64
Quality map
Frequency field
Orientation field
Quality map
Frequency field
Orientation field
Coarse estimation
Cao, Liu and Jain, Segmentation and Enhancement of Latent Fingerprints: A Coarse to Fine Ridge Structure Dictionary, PAMI, 2014 5
Latent Fingerprint Decomposition
• local total variation (LTV)
• relative reduction rate of LTV
A. Buades, T. Le, J.-M. Morel, and L. Vese, Fast cartoon + texture image filters, IEEE TIP, 19(8):1978 –1986, 2010.
( ) *| | ( ),LTV f L f x
Background patch Fingerprint patch
Separate friction ridge texture from background
= +
Texture part Cartoon part Gray image
• Cartoon part and texture part
Cartoon part
Texture part
Latent Fingerprint Decomposition
Training Set
Rolled prints
from NIST 4
80K 32×32
patches
100K 64×64
patches
Dictionary
learning
1 coarse-level
dictionary
(1,024 elements)
16 fine-level
dictionaries
(64 elements)
Provides coarse-
level quality map
with orientation and
frequency fields
Provides fine-level quality
map with orientation and
frequency fields
Friction Ridge Dictionary Learning
8
(0, π/16]
(π/16, 2π/16]
(15π/16, π]
• Fingerprint selection: NFIQ index <= 2
• Patch selection: average quality >=3.75
(a) A subset of elements on the coarse-
level dictionary (patch size: 64×64). The
total number of dictionary element is 1,024
(b) A subset of elements in the 16 orientation
specific fine-level dictionaries (patch size:
32×32). The total number of elements in
each orientation specific dictionary is 64.
Dictionary Elements
9
Patch Reconstruction: Ridge Quality
• A given latent patch is reconstructed with T
dictionary elements
• Structure similarity (SSIM) measures the “ridge
quality” of a latent patch
(a) (b) (c) (d) (e)
SSIM=0.52 SSIM=0.59 SSIM=0.63 SSIM=0.65
SSIM=0.17 SSIM=0.28 SSIM=0.33 SSIM=0.37
SSIM=0.01 SSIM=0.01 SSIM=0.02 SSIM=0.02
Quality estimation Wang et al., Image Quality Assessment: From Error Visibility to Structural Similarity, IEEE TIP, 13(4): 600-612, 2004.
1 2
2 2 2 2
1 2
2( , )
x y x y
x y x y
C CSSIM x y
C C
T=1 T=2 T=3 T=4
10
Patch Reconstruction: Ridge Flow
• Orientation and frequency fields are estimated
from the reconstructed patch Orientation field estimation
T=1 T=2 T=3 T=4
Instead of extracting level-1 features in the input latent patches,
we extract them in the reconstructed patches 11
Future Work
• Improve ridge quality in dictionary elements
– Use enhanced training set
– Post-process dictionary elements
• Better definition of ridge quality; robust to
– “Dry” fingerprints (broken ridges)
– Linear structures in the background
• Improve confidence value measure
12
Coarse to Fine Strategy
• Why do we need coarse to fine strategy?
32 x 32 patch Most similar
dictionary element 64 x 64 Patch Most similar
dictionary element
Latent Coarse-level quality Fine-level quality
13
Coarse to Fine Strategy
• How does coarse to fine strategy work?
64 x 64
patch
Most similar
dictionary
element
32 x 32
patch
Most similar
dictionary element
in selected fine-
level dictionaries
(0, π/16]
(π/16, 2π/16]
(15π/16, π]
16 fine-level dictionaries
(4π/16, 5π/16]
• Quality map: average of coarse-level and fine-level quality maps
• Ridge flow : fine-level ridge flow
• Frequency field: average of coarse-level and fine-level frequency fields 14
Demo
Texture part
+
Cartoon part
=
Gray image
• Feature: local total variation
• Method: nonlinear filter decomposition
15
Texture part
Coarse quality map Coarse orientation and
frequency fields
……
d1 d2 d3 d4 d5 x
Specific fine-level dictionary Patch
32 ×32
Fine-level
dictionary selection
Fine-level estimation 18
Coarse orientation and
frequency fields
Texture part
Coarse quality map
Fine quality map Fine orientation and
frequency fields
Segmentation result Enhancement result
19
Results on NIST SD27 Latents
Good
latent
Bad
latent
Ugly
latent
(a) Gray image (b) Texture image (c) Coarse-level quality (d) Fine-level quality 20
Results on NIST SD27 Latents
Good
latent
Bad
latent
Ugly
latent
(a) Gray image (b) Texture image (c) Segmentation result (d) Segmentation and
enhancement result 21
Matching Performance Evaluation • Latent Database: 258 latents in NIST SD27 and 449 latents in WVU DB
• Background Database : ~32K total; rolled prints in NIST SD27 (258), WVU
DB (4,739) and NIST SD14 (27,000)
• Input to COTS : (i) latent image, (ii) segmented & ehnanced latent
(a) NIST SD27 (b) WVU DB 22
Examples
(a) A latent from NIST SD27 (b) Segmented & enhanced
23
(c) Mated rolled print
Mate found at rank 1 Mate found at rank 5
Examples
(a) A latent from WVU DB
(b) Segmented and enhanced
24
(c) Mated rolled print
Mate found at rank 1 Mate found at rank 31,000
• Confidence value:
– mean quality value in the segmented foreground
– Latent is rejected, if confidence in segmentation is low
Confidence Value Evaluation
(a) NIST SD27 (b) WVU DB 25
Contributions
• A fridge dictionary based segmentation
and enhancement algorithm for latents
• Ridge quality definition
• Coarse to fine strategy to balance
accuracy vs. robustness
Cao, Liu and Jain, Segmentation and Enhancement of Latent Fingerprints: A Coarse to Fine Ridge Structure Dictionary, IEEE Trans. PAMI, 2014 (to appear)
26