REAL TIME FACE DETECTION
Justin RillingPooja MhapsekarMoinuddin Sayed
Ogom J Obinor
2
IntroductionIntroduction
• The idea is to be able to detect faces appearing in an image.
• The faces can be of different sizes and orientations.
• The motivation behind implementing this on an FPGA is that image processing is inherently parallel in nature and lends itself well to an FPGA.
3
Viola and Jones’ DetectorViola and Jones’ Detector
Key Contributions
1. Integral Image – fast computation of features used by the detector.
2. Combines complex classifiers in “cascade” – focus on facial features.
4
Integral ImageIntegral Image
• Major contribution of the Viola and Jones detector.
• Summation of pixel values of the original image.
• Value at location (x,y) = sum of values of pixels above and to the left of (x,y).
• Whole image converted to integral image and a window buffer used to scan the entire image.
5
Haar ClassifiersHaar Classifiers
• Classifiers are composed of 2-3 rectangles.
• Weights and sizes associated with features – obtained through AdaBoost.
• Classifier sum = ∑ (area * weights)
• Integral image makes area calculation easy. Area = L4–L3–L2+L1.
• Several classifiers compose a stage.
6
StageStage
• Sum of all Haar feature classifiers compared with the stage threshold.
• Multiple stages, each stage has different number of classifiers.
• Threshold obtained from AdaBoost algorithm.
Face Detection Example
0.0Classifier Sum
< 0.0Classifier Threshold
0.0Right Value
0.0Left Value0.0
Stage Sum
+0.0Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
426Classifier Sum
< 542Classifier Threshold
0.8378106Right Value
0.0337941Left Value0.0
Stage Sum
+0.8226894Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
426Classifier Sum
< 542Classifier Threshold
0.8378106Right Value
0.0337941Left Value0.0337941
Stage Sum
+0.8226894Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
887Classifier Sum
< 2045Classifier Threshold
0.7488812Right Value
0.1514132Left Value0.0337941
Stage Sum
+0.8226894Stage Threshold
<1
0
0Stage Number
1Classifier Number
Face Detection Example
887Classifier Sum
< 2045Classifier Threshold
0.7488812Right Value
0.1514132Left Value0.1852073
Stage Sum
+0.8226894Stage Threshold
<1
0
0Stage Number
1Classifier Number
Face Detection Example
349Classifier Sum
< 568Classifier Threshold
0.6374819Right Value
0.0900493Left Value0.1852073
Stage Sum
+0.8226894Stage Threshold
<1
0
0Stage Number
2Classifier Number
Face Detection Example
349Classifier Sum
< 568Classifier Threshold
0.6374819Right Value
0.0900493Left Value0.2752566
Stage Sum
+0.8226894Stage Threshold
<1
0
0Stage Number
2Classifier Number
Face Detection Example
349Classifier Sum
< 568Classifier Threshold
0.6374819Right Value
0.0900493Left Value0.2752566
Stage Sum
+0.8226894Stage Threshold
<1
0
0Stage Number
2Classifier Number
Window Buffer Failed Stage 0
Face Detection Example
0.0Classifier Sum
< 0.0Classifier Threshold
0.0Right Value
0.0Left Value0.0
Stage Sum
+0.0Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
0.0Classifier Sum
< 0.0Classifier Threshold
0.0Right Value
0.0Left Value0.0
Stage Sum
+0.0Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
0.0Classifier Sum
< 0.0Classifier Threshold
0.0Right Value
0.0Left Value0.0
Stage Sum
+0.0Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
0.0Classifier Sum
< 0.0Classifier Threshold
0.0Right Value
0.0Left Value0.0
Stage Sum
+0.0Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
0.0Classifier Sum
< 0.0Classifier Threshold
0.0Right Value
0.0Left Value0.0
Stage Sum
+0.0Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
0.0Classifier Sum
< 0.0Classifier Threshold
0.0Right Value
0.0Left Value0.0
Stage Sum
+0.0Stage Threshold
<1
0
0Stage Number
0Classifier Number
Face Detection Example
0.0Classifier Sum
< 0.0Classifier Threshold
0.0Right Value
0.0Left Value0.0
Stage Sum
+0.0Stage Threshold
<1
0
0Stage Number
0Classifier Number
22
Top Level DesignTop Level Design
YES
Get ScaledImage
Get Integral Image
Window Buffer (21x21)
Face Detection PipelineEnd of
Image?
NO
Monitor
Draw RectangleIf face is det.
Get Image
Face Detection Pipeline
Original End Product
USB Webcam FPGA Display
USB DVI
Updated End Product
Digital Interface
1/3 Color Camera Mod C3188A-6018• Supports several standard image data
formats including YCrCb 4:2:2 • 640 x 480 resolution• 5 V signaling • 30 fps
FPGA Display DVI
3.3 VSignaling
Updated End Product
XC95144XLCPLD
Display
5 VSignaling
DVI1/3 Color Camera Mod C3188A-6018• Supports several standard image data
formats including YCrCb 4:2:2 • 640 x 480 resolution• 5 V signaling • 30 fps
FPGA
DVI Controller
Questions ???
References:
1. P. Viola and M. Jones, “Robust real-time object detection,” International Journal of Computer Vision, 57(2), 137-154, 2004.
2. Junguk Cho, Shahnam Mirzaei, Jason Oberg, Ryan Kastner, “FPGA-Based Face Detection System Using Haar Classifiers,” Proceeding of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 103-112, 2009.
OpenCV Program – Lena.jpg
OpenCV Program – ER.jpg
OpenCV Program – ER.jpgWindow Buffer (2, 152, 122)Window Buffer (2, 152, 122)
Face Detection Pipeline
OpenCV Program – ER.jpgWindow Buffer (3, 165, 20)Window Buffer (3, 165, 20)
Face Detection Pipeline
Questions ???