Post on 23-Dec-2015
transcript
IMAGE ANALYSIS AND OBJECT DETECTION-RECOGNITION USING COMPUTER VISION ALGORITHMS
Ramakrishna LankaRamakrishna.lanka@mavs.uta.edu
1001126160
Contents
• Introduction to images and image processing• Overview of computer vision• Overview of image and video processing workflow• Image enhancement• Computer vision search algorithms and techniques• Tools to be used• References
PROPOSAL
The objective of this project is to implement computer vision algorithms to analyze an image and detect objects in the image
based on object reference images.
Acronyms• 3D – Three dimensional• CVIU – Computer vision and image understanding• FAST – Features from accelerated segment test • MSER – Maximally stable extremal regions• PCB – Printed circuit board• RANSAC – Random sample consensus• RGB – Red, green and blue• SURF – Speeded up robust features• VTP – Virtual terrain Project
What is an Image ?
• Pictorial reproduction of an object in physical or digital space.
• A digital Image is made up of Pixels.• More the pixels, much detailed is the Image as
shown in figure(1).• Resolution of an Image is measured by the
number of pixels it consists of. Ex: 1366x768Fig(1). Illustration of image pixels. [19]
Need for Image and video processing.
• Computer Vision: Object/Activity detection as shown in figure (15) and figure (16).
• Medical and biological image processing as shown in figure (10) and figure (11).
• After effects: Adding effects and image correction as shown in figure (2).
• Efficient use of media in Consumer multimedia products: Storage, display and transfer.
Fig(2). Illustration of image after effects.
OVERVIEW OF COMPUTER VISION
• Vision has given humans the natural ability to picture the world around in a three-dimensional (3D) space.
• Computer vision gives the man-made device the ability to perceive the world.
• The goal of computer vision is to extract meaningful information from the images and videos that the image sensors capture.
OVERVIEW OF THE IMAGE AND VIDEO PROCESSING WORKFLOW
Fig(2). Image and video processing workflow. [2]
OVERVIEW OF THE IMAGE AND VIDEO PROCESSING WORKFLOW
• The raw images from the source may contain unwanted image source characteristics or noise.
• Preprocessing to enhance the images will prepare it for further processing based on the requirement.
• Image enhancement can be achieved using various filtering techniques and transforms.
Negative of an image
Fig(3). Original image of a PCB [21] Fig(4). Histogram of the original PCB image
N(r, c) = 255 – I(r, c) where 0 <= r <= R and 0 <= c <= C (1.1) Where, N is the negative of image I, both with dimension R x C.
Negative of an image
Fig(7). Negative image of the PCB
Fig(8). Histogram of the negative PCB image
Negative of an image
Fig(10). Original image of a knee scan [22] Fig(11). Negative image of the knee scan figure (10)
Fig(9). Histogram of the knee scan Fig(12). Histogram of the negative knee scan
Thresholding Transformation
(1.2)
Where s is the value of the resultant pixel intensity and r is the actual pixel intensity.
Fig(13). Image of some coins [28] Fig(14). Threshold image of figure(13)
Thresholding Transformation
Fig(15). Image of obstacles [24] Fig(16). Multilevel threshold image of figure(15)
• Multilevel thresholding
Thresholding Transformation• Thresholding – color labelling
Fig(17). Image of my cluttered desk Fig(18). Threshold segmentation of figure(17)
Powers-Law Transformations / Gamma corrections = crγ (1.3)Where, s is the corrected pixel intensity, r is the actual pixel intensity, c and γ are gamma correction constants.
Fig(19). Image of the gamma palette [27] Fig(20). Gamma corrected image of figure(19) with γ=0.7 Fig(21). Gamma corrected image of figure(19) with γ=0.5
Histogram Equalizationh(rk) = nk (1.4)Where, h(rk) is the histogram function, rk is the intensity value and nk is the number of pixels with the intensity rk.[4]
COMPUTER VISION SEARCH ALGORITHMS AND TECHNIQUES
Edge feature detection
• The Sobel operator
• The Prewitt operators
• The Robert operator
• Laplacian/Laplacian of Gaussian operator
• Canny edge detection
Stop sign
Stop scene
Fig(24). Reference object image [25]
Fig(25). Object in scene image [25]
Edge feature detection• The Sobel operator
(1.5)
Where Gx is the vertical filter and Gy is the horizontal filter.[14][15]
Figure(26) and figure(27) show the edge detections of figure(24) and figure(25).
Sobel stop sign
Sobel stop scene Fig(26). Sobel edge detection of figure(24)
Fig(27). Sobel edge detection of figure(25)
Edge feature detection• The Prewitt operators
Where Gx is the vertical filter and Gy is the horizontal filter.[14][15]Figure(28) and figure(29) show the edge detections of figure(24) and figure(25).
(1.6)
Fig(28). Prewitt edge detection of figure(24)
Fig(29). Prewitt edge detection of figure(25)
Edge feature detection• The Robert operator
(1.7)
Where Gx is the vertical filter and Gy is the horizontal filter.[14][15]
Figure(30) and figure(31) show the edge detections of figure(24) and figure (25).
Edge feature detection• The Laplacian/Laplacian of Gaussian operator
- The Laplacian of an image highlights regions of rapid intensity change.
- The Laplacian is often applied to an image that has first been smoothed with something approximating a Gaussian smoothing filter in order to reduce its sensitivity to noise. [14][15]
Edge feature detection• Canny edge detection
- Low error rate. - Edge points should be well localized. - Single edge point response.
COMPUTER VISION SEARCH ALGORITHMS AND TECHNIQUES
Local point/region detection
• Corner point detection – FAST (Features from accelerated segment test )
• MSER(Maximally stable extremal regions) feature detection The Robert operator
• SURF(Speeded up robust features) feature detection.
Stop sign
Stop scene
Fig(24). Reference object image [25]
Fig(25). Object in scene image [25]
Local point/region feature detection• Corner point feature detection
- FAST corner detector uses a circle of 16 pixels (a Bresenham circle of radius 3) to classify whether a candidate point p is actually a corner. [31]
Local point/region feature detection• Maximally stable extremal regions (MSER)
MSER is a method for blob detection in images. The MSER algorithm extracts from an image a number of co-variant regions, called MSERs: an MSER is a stable connected component of some gray-level sets of the image. [29]
Local point/region feature detection• Speeded up robust features (SURF)
SURF is a fast and robust algorithm for local, similarity invariant representation and comparison.
SURF selects interest points of an image from the salient features
OBJECT RECOGNITION
• Recognition [9][10]– Recognition is a process that follows detection and/or segmentation.– Recognition approaches in this project,
• Object feature recognition [10]– Recognition with point feature matching– Recognition with edge feature matching
• Object Segmentation recognition [1]– Recognition with segmentation
OBJECT FEATURE RECOGNITION• Recognition with point feature matching [1][10]
– Match the extracted feature points of reference and scene images.– The RANSAC (Random sample consensus ) algorithm works by estimating the
best set of point matches between the images. This is shown in figure(43).
Fig(43). Feature matching between figure(25) and figure(26)
OBJECT FEATURE RECOGNITION• Recognition with point feature matching [1][10]
Fig(44). Object in figure(25) recognized in figure(26)
Project status
The aim of this project is to analyze and enhance images; and design a system to identify objects in an image. The identity of the objects will be achieved by using feature detection and recognition based on the features of the reference objects that the system would have learnt before recognition.
• Image analysis : Done• Image Enhancement : Done• Feature detection : Done• Feature Extraction : Done• Segmentation : Done• Performance comparison : Ongoing• Recognition with point feature matching : Done• Recognition with edge feature matching : Done (yet to document)• Recognition with segmentation : Done (yet to document)• Multiple object recognition : Done (yet to document)
Tools being used
• Matlab – Image processing tool box [13]• Matlab – Computer vision tool box [9]• Matlab – R2013a• OpenCV v2.4.11 [10]• Microsoft Visual Studio v2010• Windows 8.1 64bit• Central processor – Intel® Core™ i3-3120 2.50Ghz• Graphics processor – NVidia GT 720M 2GB• Memory – 8.00 GB
References
[1] R. Szeliski, “Computer Vision: Algorithms and Applications”, 2010 Springer, Draft: 3, September 2010.
[2] A. Thé, “Image Processing Made Easy”, MathWorks Videos and Webinars, Recorded: 15 Oct 2005. http://www.mathworks.com/videos/image-processing-made-easy-81718.html?form_seq=conf1050
[3] VTP software tutorials webpage (Source for hazy image): http://vterrain.org/Doc/Tut3/
[4] R. Maini and H. Aggarwal, “A Comprehensive Review of Image Enhancement Techniques”, Journal of Computing, Volume 2, Issue 3, pp 8-12 March 2010. : http://arxiv.org/ftp/arxiv/papers/1003/1003.4053.pdf
[5] T. Gupta, “Seeing through Chaos: Noise Filtering”, ahumaninmachinesworld.blogspot, January 2014 (Internet Blog on Noise Filtering): http://ahumaninmachinesworld.blogspot.com/2014/01/seeing-through-chaos-noise-filtering.html
[6] Computer Vision Algorithms, “COMPUTER VISION TUTORIAL”. Link: http://www.societyofrobots.com/programming_computer_vision_tutorial_pt3.shtml#middle_mass
[7] H. Spont´on and J. Cardelino, “A Review of Classic Edge Detectors”, Image Processing On Line, Preprint Issue, pp 1-8, June 2012. Link: http://www.ipol.im/pub/pre/35/preprint.pdf
References
[8] ‘Performance criteria’, Document in the Course website of EE5356 Digital Image processing (rate the performance of the filtered/enhanced image). Link: http://www.uta.edu/faculty/krrao/dip/Courses/EE5356/performance.pdf
[9] Matlab computer vision toolbox: http://www.mathworks.com/help/vision/
[10] Open source Computer vision software: OpenCV v3.0 – download: http://opencv.org/downloads.html
[11] Open source Computer vision software: OpenCV v3.0 – Reference Manual: http://docs.opencv.org/3.0-beta/modules/refman.html
[12] R.C. Gonzalez, R.E.Woods and S.L. Eddins, “Digital Image Processing Using MATLAB”, Pearson Education, 2004.
[13] Matlab image processing toolbox: http://www.mathworks.com/products/image/index-b.html
[14] A. K. Jain, “Fundamentals of Digital Image Processing”. Englewood Cliffs, NJ: Prentice Hall, 1989.
References
[15] R.C. Gonzalez and R.E. Woods, “Digital Image Processing”. 3nd edition, Prentice Hall, 2007.
[16] Line detection image: http://www.camera-sdk.com/p_253-how-to-accomplish-line-detection-in-c-onvif.html
[17] Open source Computer vision software: OpenCV v3.0 – User Guide: http://docs.opencv.org/3.0-beta/doc/user_guide/user_guide.html
[18] Image segmentation example link: http://www.bioclinica.com/sites/default/files/u1/atlas-based-brain-segmentation-lg.jpg
[19] Test images: http://www.imageprocessingplace.com/root_files_V3/image_databases.htm
[20] Computer vision test image: http://www.cs.cmu.edu/afs/cs/project/cil/www/v-images.html
[21] PCB image: http://images.fineartamerica.com/images-medium-large-5/printed-circuit-board-russell-shively.jpg
References
[22] Medical image – knee: http://www.leutron.com/english/product/vip_d.htm
[23] Geometric shapes image: http://www.maxton.com/gallery/con3-all2a.jpg
[24] Need for image processing, computer vision path planning image: http://www.juliangough.com/photos/assorted-stolen-images-i-like/1107153
[25] Stop reference image: http://safety.fhwa.dot.gov/intersection/resources/casestudies/fhwasa09010/images/f1.jpg
[26] Stop scene image: http://www.driversedguru.com/wp-content/gallery/jamie-gallery/Stop%20Sign.jpg
[27] Gamma correction image: http://www.maximumpc.com/files/u16580/HowMonTest1.jpg
[28] Coins image: http://upload.wikimedia.org/wikipedia/en/e/ec/Eagle_coins.jpg
References
[29] Ppt on MSER : http://www.micc.unifi.it/delbimbo/wp-content/uploads/2011/03/slide_corso/A34%20MSER.pdf
[30] H. Bay, A. Ess, T.Tuytelaars and L.V. Gool, "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346--359, June 2008.
[31] PowerPoint presentation on FAST: http://web.eecs.umich.edu/~silvio/teaching/EECS598_2010/slides/11_16_Hao.pdf