Practical Computer Vision:Theory & Applications
Carmen Alonso Montes
23rd-27th November 2015
Carmen Alonso Montes 2
Wrap up
Today, we are here
Carmen Alonso Montes 3
Learned concepts
Hough Transform Distance mapping
Active contoursWatershed
Carmen Alonso Montes 4
Contents
Wrap up
Object Recognition
Practical application in industry: code target recognition
Biometrics Recognition
Practical application in authentication applications using medical images
Summary
Carmen Alonso Montes 5
Object Recognition
Carmen Alonso Montes 6
Object Recognition
Object recognition is a process for identifying a specific object in a digital image or video.
Object recognition algorithms rely on matching, learning, or pattern recognition algorithms using appearance-based or feature-based techniques
Humans recognize a lot of objects in images with little effort even when
the image may vary in view points, sizes and scales, translated or rotated.
partially obstruction
http://www.engadget.com/2014/09/08/google-details-object-recognition-tech/
Carmen Alonso Montes 7
Challenges
The main challenges is:
Images are big
Viewing conditions are infinite
Computers are finite
Objects are surrounded by other objects
Carmen Alonso Montes 8
How does the machine “recognise”?
Once the objects are detected (segmented, edges, skeletons, etc) it is the moment to recognise
Highly dependent on the application
Real-time?
Offline processing
Do we have a database with reference images?
How much is the variability of the images? Do I need any similarity measure?
As engineers, we can go for sophisticated approaches (neural networks) or more simple ones template matching or image registration
Usually techniques from pattern recognition are useful in this area.
https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTdhVMLS7x7b5avBWy_aZNaF8KD5BN9PWvO6Dq2XHIDhP5HvumiAA
Carmen Alonso Montes 9
Carmen Alonso Montes 10
Template matching
Template matching is a technique in digital image processing for finding small parts of an image which match a template.
It can be used in manufacturing as a part of quality control, a way to navigate a mobile robot, or as a way to detect edges in images.
http://goo.gl/mj3Mei
http://goo.gl/7VfQRM
Carmen Alonso Montes 11
Optical Character Recognition (OCR)
Optical character recognition(optical character reader) (OCR) is a classical example of pattern recognition. Hand made characters shall be recognised by the computer and provided a digital version. First test and approaches were developed to recognise Chinese characters due to its singularity
Nowadays, it is a popular technique thanks to initiatives to put in digitall format old printed books
OCR is a field of research in pattern recognition, artificial intelligence and computer vision.
Resources: Stanford data set: http://ai.stanford.edu/~btaskar/ocr/
Some popular techniques
Matrix matching involves comparing an image to a stored glyph on a pixel-by-pixel basis; it is also known as "pattern matching", "pattern recognition", or "image correlation".
Feature extraction decomposes glyphs into "features" like lines, closed loops, line direction, and line intersections. These are compared with an abstract vector-like representation of a character, which might reduce to one or more glyph prototypes. (e.g. k-nearest neighbors algorithm)
Software such as Cuneiform and Tesseract use a two-pass approach to character recognition. The second pass is known as "adaptive recognition" and uses the letter shapes recognized with high confidence on the first pass to recognize better the remaining letters on the second pass.
Carmen Alonso Montes 12
Robotics: Obstacle Avoidance
Obstacle avoidance: non-intersection or non-collision
The willow garage project
ROS: Robot operating system
R3-COP - Resilient Reasoning Robotic Co-operating Systems
Carmen Alonso Montes 13
Collision Avoidance
Carmen Alonso Montes 14
Practical case in industry
Carmen Alonso Montes 15
Practical Industrial Case
Problem: Automatic raw part alignment for machining process by means of artificial Coded Targets (CT) within a Phogrammetry-based Application
Published: Miguel Fernandez-Fernandez, Carmen Alonso-Montes, et al.: Industrial Non-intrusive Coded-Target Identification and Decoding Application. IbPRIA 2013: 790-797
http://goo.gl/bwDtVg
Carmen Alonso Montes 16
What do we need to do as CV Engineers?
Analyze the environment
Where will the camera be placed?
Light conditions?
Pollution?
External factors that can affect my system?
Analyze the elements you have
The camera properties and capacities
Crucial: Calibration phase
The images
Do I have a set of training images ?
Is this training set realistic?
Must I have real time processing?
How do I integrate with the other systems?
Carmen Alonso Montes 17
Our environment: Schema
Wireless communication among camera & PC
Processing must be done in the camera or the image must have lower resolution
Carmen Alonso Montes 18
Coded Target (CT)
The reliability of CT for industrial applications comes from its structure
invariant to rotation, scale, and distortion
The central circle target is surrounded by the code band, which is divided into equally spaced angular intervals corresponding to bit positions (meaning 1 for a white interval and 0 for a black one).
The code band is read anticlockwise from a predefined starting point extracting the associated bit pattern, and consequently the associated number.
The bit pattern is 0010111101 which corresponds to the associated number 190.
The exact raw part geometry correspondence is guaranteed thanks to the uniquity of the CT decoded numbers.
Carmen Alonso Montes 19
Let's start
We already have an black and white image
High resolution image
Objective: read it and get the associated number
Challenges:
Light conditions
Pollution and dust
Occlusions and changes of perspective depending on where is placed
Carmen Alonso Montes 20
The algorithm
We have split up the system in 2 phases
Phase 1: aims to locate approximately the valid Cts
Phase 2: To read the selected CT from Phase 1
Carmen Alonso Montes 21
Phase 1: Where are my Cts?
The bilateral filtering is an edge preserving and noise reducing smoothing filter, which performs specifically well in poor illumination conditions and the pollution type of this industrial raw part images.
An adaptive threshold algorithm (mean value of all pixels in its 5x5 neighbourhood) is applied to binarise the image.
Then, erosion and dilation operations are performed in order to remove the salt-and-pepper noise.
Carmen Alonso Montes 22
Identification of the potential CT
Search for the CT and to compute the surrounding area (ROI). The CT identification is done based on the analysis of the detected contour structures.
valid or non-valid CT structures according to their conformance with an ideal CT pattern
Circularity factor : minimum enclosing technique to distinguish those contours that are actually circles
Three concentric circles
Ratio descriptors: distance ratio among circles and diameter ratio of the central target
Carmen Alonso Montes 23
Phase 2: My identified CTs, let's read them
Step 2. Image denoising
Noise will be removed by means of the bilateral filter.
Image is binarised by the adaptive threshold and then,
Erosion and dilation operations are performed to remove noise.
Step 3 performs a contour identification.
Minimum enclosing circle technique.
The CT center coordinates must be the same for all the circles.
In order to compute the exact center, the momentum of the image is used, in particular, the centroid or center of mass of the CT
The Step 4 will perform the analysis of the CT structure for its number decoding.
Carmen Alonso Montes 24
Examples
Carmen Alonso Montes 25
Biometrics
Carmen Alonso Montes 26
Biometrics
Biometrics refers to metrics related to human characteristics.
Biometrics authentication (or realistic authentication) is used in computer science as a form of identification and access contro, to identify individuals in groups that are under surveillance.
Biometric identifiers are the distinctive, measurable characteristics used to label and describe individuals
fingerprint, palm veins, face recognition, DNA, palm print, hand geometry, iris recognition, retina and odour/scent.
biometric identifiers
unique to individuals
more reliable in verifying identity
Challenge: personal privacy concerns about the ultimate use of this information.
Carmen Alonso Montes 27
Fingerprint Recognition
http://www.nitgen.com/eng/technology/img/02_img_01.jpg
Carmen Alonso Montes 28
Face Recognition
http://news.bbc.co.uk/nol/shared/spl/hi/guides/456900/456993/img/noflash/facial_scan_416_dc.gif
Carmen Alonso Montes 29
Example
http://biometrics.mainguet.org/types/face/face_visualskinprint.jpg
Carmen Alonso Montes 30
Iris Recognition
http://4.bp.blogspot.com/-Yb-isbxtQoQ/VPqDNJlcf5I/AAAAAAAAAMQ/OSq7mozt11o/s1600/iris%2Brecognition%2Bsystem%2Balgorithm.gif
Carmen Alonso Montes 31
Practical case: biomedical authentication based on retina pattern
Carmen Alonso Montes 32
Retinal Authentication
Carmen Alonso Montes 33
Practical case: Retinal AuthenticationMinority Report → Future or reality?
Carmen Alonso Montes 34
The problem
We want to use the retinal vessel tree to identify persons using the retina vessel pattern.
Carmen Alonso Montes 35
Thinking about the problem
Images
Segmentation issues
Image acquisition
Vessel
Diseases
Central reflex
How do I get the minutae?
How do I register and get the authentication?
Validation of the system.
How well do I perform?
Carmen Alonso Montes 36
My schema
Carmen Alonso Montes 37
Processing the retinal images
Retinal images can be obtained by several methods
We are only interested in those points of the vessels corresponding to junctions
Authentication using images → the same philosophy than fingerprints
We only need crossing points
Carmen Alonso Montes 38
Challenges (I)
Central reflex
Small vessels
Carmen Alonso Montes 39
Challenges (II)
Pathologies can make that the images under study does not comply with our minimum requirements
Optic disk usually is not considered in global retinal process due to its particular brigthness
Carmen Alonso Montes 40
Our specific needs
Carmen Alonso Montes 41
Vessel skeleton
Stage 1 is devoted to clean the vessels from noise, and to homogeneize vessel locations
The results will be used to compute the initial contours and the external energy needed by the active contours
Pixel Level Snakes are a specific active contours technique were all the pixels of the contours evolve
No nodes are needed in this case
Carmen Alonso Montes 42
Phase 1. Vessel pre-estimation
Diffusion acts as a kind of smoothing kernel
Goal: remove noise due to the acquisition device
The thresholded image is obtained through adaptive segmentation due to the high variability in the different areas of the retina
Carmen Alonso Montes 43
Phase 2. Computing the input for the Active Contours
Active contours need
Initial contour
External Potential
Initial contour must be close to vessel locations
External Potential must guide the contours towards the real edges
Carmen Alonso Montes 44
Computing the External Energy
Simple operations were used since only addition, substraction, multiplication and neighbourhood operations were available in the SCAMP device
Distance mapping to the real edges was computed with several dilations over the detected edges
Carmen Alonso Montes 45
The evolution of the active contours was splitted in 2 steps
Fast evolution (mainly guided by external energy only)
Hole filling to remove noisy regions
Slow evolution to fit the details
Carmen Alonso Montes 46
Retinal vessel skeleton
Carmen Alonso Montes 47
Computing the Skeleton
Carmen Alonso Montes 48
Minutae in Retina
Skeletonised image barely is optimal
Discontinuities usually appears
Two types discontinuities
Union of the same vessel
Junctions
Carmen Alonso Montes 49
Results
No possibility of fraud
No false positives
Carmen Alonso Montes 50
Conclusion
Segmentation techniques are in the core of a good range of computer vision applications, and in particular for object recognition
Different images from industrial and from medical domain benefit from segmentation for
Normalizing intensities
Segmenting important objects for their applications
Usually, computer vision engineers needs to thing in the big picture before start doing
Think in your needs and in what you have
Go for a simple approach, and then try to improve it moving towards more sophisticated algorithms to fulfill your needs
After this course, you have a minimum knowledge about classical techniques used in the literature
Try to play with the classical ones and move forward to more exciting approaches