Date post: | 29-Mar-2015 |
Category: |
Documents |
Upload: | esther-goodlow |
View: | 219 times |
Download: | 2 times |
1ECE 495 – Integrated System Design I
- Ninad Pradhan, Matt Pepper
Introduction to Image Processing
ECE 495, Spring 2013
2ECE 495 – Integrated System Design I
Overview
• What is image processing?• What is an image?• Introduction to image processing techniques• Experimental setup• Basic MATLAB commands for image processing
3ECE 495 – Integrated System Design I
What is Image Processing?
Image processing is the collective name for techniques used to extract information from digital images or to manipulate them to
render variations of the input image.
Photo stitching Color boost
Vehicle detection and tracking
4ECE 495 – Integrated System Design I
What is Image Processing?
Popular technologies which make use of the camera as a sensor
The Wii Remote uses an IR camera to sense its location relative to the
Wii Sensor Bar.
The Kinect uses image processing techniques on depth images to detect
and track locations of multiple persons in the field of view.
5ECE 495 – Integrated System Design I
Pixels
Pixel
• A pixel (abbr. for picture element) is the smallest unit of an image.
• Therefore, a 640x480 image is a matrix of 640 columns and 480 rows, each element of this matrix is called an image pixel.
6ECE 495 – Integrated System Design I
MATLAB Image Coordinates
• MATLAB stores images as matrices.• In MATLAB, image pixels are referenced using (row, col)
values.• Origin of the coordinate system (1,1) is the top left corner of
the image
img
Thus, img(4,3) refers to the pixel at the 4th row and 3rd column.
(1,1)
7ECE 495 – Integrated System Design I
RGB and Grayscale
• In RGB format, each Pixel has 3 color components: Red, Green, and Blue.
• Other color representations, e.g. HSV, YUV, CMYK, are also used. Transformations from RGB to these color spaces and back are defined in MATLAB.
• If only intensity (bright/dark) variations are considered, the resultant image is called a grayscale image. Each pixel has only 1 component: intensity.
RGB Gray
8ECE 495 – Integrated System Design I
Basic Image Processing techniques
• Background Subtraction• Intensity Thresholding • Morphological Operations• Color-based Processing• Shape-based Processing
A combination of the above methods is usually used in basic image processing applications. There is no ‘perfect solution’, so keep discussing (and trying) what suits your project best.
9ECE 495 – Integrated System Design I
Background Subtraction
• Background subtraction is a technique used to isolate useful information in an image (foreground) from the rest of the image (background).
• A reference image is selected as the background.• Each successive image in a video stream is compared against
this image.• If the difference between the images is significant, the areas
which are different are considered to be the foreground for that image.
10ECE 495 – Integrated System Design I
Background Subtraction
Reference image or background
Incoming image contains a new object. This is our foreground, i.e. useful information for detecting the system state.
The new object is identified as the foreground after the background is subtracted.
11ECE 495 – Integrated System Design I
Intensity Thresholding
• Intensity thresholding is another foreground separation technique. It uses histograms.
• A histogram is an image statistic which usually operates on an intensity image, i.e. pixels having a single value between 0-255
• The range 0-255 is divided into bins, e.g. each intensity value may be given its own bin
• The Y axis shows the count of number of pixels in an image which lie within limits for a bin.
Bin
Cou
nt
12ECE 495 – Integrated System Design I
Intensity Thresholding
• Assumption: Foreground is sufficiently bright/dark compared to the background, i.e. the histogram has a clear separation.
• Assumption: The threshold value has been estimated using prior observations or can be computed automatically.– This method is very sensitive to lighting conditions
• Given these assumptions, set all pixels on the foreground side to 255 (bright), and all other pixels to 0 (dark).
Foreground
Threshold
13ECE 495 – Integrated System Design I
Morphological Operations
• Morphological operations are used to either remove noise from an image or recover some details lost in thresholding or background subtraction.
• Dilation adds to the foreground areas in an image, but can also add to the noise.
• Erosion removes stray foreground pixels from it, but also reduces genuine foreground areas.
• Usually, a combination of morphological operators is used to improve the output.
After dilation After erosionOriginal
14ECE 495 – Integrated System Design I
Color Processing
• Used in identifying well color for the Minesweeper problem.• In MATLAB, a color (RGB) image is stored as a 3D matrix with
the third dimension being color.• Each pixel has a Red, Green, and Blue value. The values range
(for each color) from 0 to 255, e.g. a purely red pixel at (25,30) will have the RGB value (255,0,0).
• To access the Red value for this pixel in the 3D image matrix, we reference img(25,30,1). Similarly, Green will be img(25,30,2) and Blue, img(25,30,3).
img
15ECE 495 – Integrated System Design I
Color Processing
• Color processing algorithms need to have a tolerance to color, e.g. a RED sticker won’t have RGB (255,0,0).
• This tolerance can be built into the algorithm by – Testing under varied ambient lighting
conditions, e.g. morning, evening, and setting color thresholds accordingly.
– Experimenting with more variation-tolerant color spaces, e.g. HSV.
– Experimenting with logical operators, e.g. a RED sticker = (Red enough) AND (NOT Green enough) AND (NOT Blue enough).
– Take average of pixel values in area
Is there a red sticker in this well?
RGB (173,58,86)
16ECE 495 – Integrated System Design I
Shape-based Processing
• Once the foreground objects have been detected, it is important to know their properties such as area, shape, location, etc.
• MATLAB has a function called regionprops which takes a binary image as input and identifies properties of contiguous (connected) pixels in foreground regions.
• The output of regionprops is an array of structures. Each element of the array contains information about one foreground region.
• MATLAB help file is very usefulRegion 1
Region 2Region 3
17ECE 495 – Integrated System Design I
Shape-based Processing
Regionprops can be used to filter out poor foreground detection results. For example:
1
2
3
4
56
All detected foreground regions
Eliminate those which are too small [5]
Eliminate those which are not circular [6]
Eliminate those which are not at permitted
locations [4]
Genuine foreground detections
1
2
3
[1 2 3 4 5 6]
[1 2 3 4 6]
[1 2 3 4]
[1 2 3]
18ECE 495 – Integrated System Design I
Experimental setup
When the camera is set up in a known configuration relative to the rest of the system, we can estimate physical dimensions using image data. For example:
Camera field of view
Fixed height 480
pixels
640 pixels
16”
12”
This implies• 40 pixels/inch
along image width
• 40 pixels/inch along image height
A circle 1.25” diameter will be bounded by an area of:
1.25” * 40 pix/inch * 1.25” * 40 pix/inch = 2500 pixUse information to improve game state detection , criteria for algorithms
Width Height
19ECE 495 – Integrated System Design I
Some MATLAB functions
Name Functionality
imread Read input image
imshow Show the image in a figure window
rgb2gray Convert a RGB image to grayscale
rgb2hsv Convert a RGB image to HSV
imdilate Dilation
imerode Erosion
im2bw Convert image to binary using thresholding
regionprops Get properties of connected components in an image
imhist Show image histogram