Date post: | 18-Aug-2015 |
Category: |
Mobile |
Upload: | rahat-yasir-anindo |
View: | 132 times |
Download: | 2 times |
Imagine Camp:Image Processing Camp
followed by 2 weeks HackathonConducted by: Rahat Yasir Anindo
Imagine Camp Overview
• Image• Image Processing• Applications of image processing• Image processing algorithms (10-12)• Matlab live demo• Image manipulation in Matlab• Windows Phone Demo• Brain storming for Hackathon• Idea selection (one out of three)• Live presentation on selected idea• Suggestion for converting your idea into an app
Time Schedule:10:30-1:00 & 2:00-3:00 = Image processing
discussion & demo3:00-3:30 = Brain storming for Hackathon idea3:30-4:00 = Idea selection4:00-5:00 = presentation on idea & suggestion for
development.
A Picture is Worth 100 Words
A Picture is Worth 10,000 Words
A Picture is Worth a Million Words
Lighting
Scene
Camera
Computer
Scene Interpretation
Components of a Computer Vision System
The goal of computer vision• To extract “meaning” from pixels
What we see What a computer sees
The goal of computer vision• To extract “meaning” from pixels
Source: “80 million tiny images” by Torralba et al.
Humans are remarkably good at this…
What kind of information can be extracted from an image?
• Metric 3D information• Semantic information
10
What is a digital image?
11
Processing of images which are Digital in nature by a Digital computer
Common image formats include:– 1 sample per point (B&W or Gray scale)– 3 samples per point (Red, Green, and Blue)– 4 samples per point (Red, Green, Blue, and “Alpha”,
Opacity)
12
Why we need image processing?It is motivated by three major application
Improvement of pictorial information for human perception
Image processing for autonomous machine application
Efficient storage and transmission
Matrix Representation of an Image
13
1,11,10,1
1,10,10,1
1,01,00,0
)1,1()1,1()0,1(
)1,1()1,1()0,1(
)1,0()1,0()0,0(
),(
:image NAn
NMMM
N
N
aaa
aaa
aaa
A
NMFMfMf
Nfff
Nfff
yxf
M
Visual image formation-Digital Version
14
3D object
Digital Camera
Digitizer 2D Digital ImageCCD / CMOS
sensor
Lens
Sampling and Quantization, 2D Case
15
Continuous Sampled
f(0,0) f(0,1)
f(N-1,M-1)
16
DIGITAL IMAGES: Digital images are 2D arrays (matrices) of numbers:
Hybrid Images
• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
Why do we get different, distance-dependent interpretations of hybrid images?
?
Slide: Hoiem
Why study computer vision?• Vision is useful• Vision is interesting• Vision is difficult
• Half of primate cerebral cortex is devoted to visual processing• Achieving human-level visual perception is probably “AI-complete”
Why is computer vision difficult?
Challenges: viewpoint variation
Michelangelo 1475-1564
Challenges: illumination
image credit: J. Koenderink
Challenges: scale
Challenges: deformation
Xu, Beihong 1943
Challenges: occlusion
Magritte, 1957
Challenges: background clutter
Challenges: Motion
Challenges: object intra-class variation
Challenges: local ambiguity
Challenges: local ambiguity
Challenges: local ambiguity
Challenges or opportunities?• Images are confusing, but they also reveal the structure of
the world through numerous cues• Our job is to interpret the cues!
Image source: J. Koenderink
Biometrics
Fingerprint scanners on many new laptops, other devices
Face recognition systems now beginning to appear more widelyhttp://www.sensiblevision.com/
Source: S. Seitz
Biometrics
How the Afghan Girl was Identified by Her Iris Patterns
Source: S. Seitz
Vision-based interaction: Xbox Kinect
. Digitizing an image ( convert an continuous image to a digital one)
. Enhancing an image ( Let an image better suit for a specific application)
. Restoring an image ( Recover a damaged image)
. Compressing an image ( Store it with less bytes )
. Segmenting an image ( Partition objects in an image from background )
. Recognizing an image ( Tell what the objects are in an image )
36
What can digital image processing do?
When an image damaged, we can recover it
37
Cracked parts
torn
Restoring an image
04/18/2023 38
Original image 257kb Compressed image 147kb
Redundant Info.
Compressing an image
Take car license plate recognition as an example.
04/18/2023 39
Recognizing an image
EXAMPLE OF DIP•One of the most common uses of DIP techniques: improve quality, remove noise etc
EXAMPLES:
(e) Poorly exposed x-ray image
(f) The result from contrast and edge enhancement
(g) Image blurred by motion
(h) The result of de-blurring
Object Recognition (CVPR 2006)
Examples: The Hubble Telescope•Launched in 1990 the Hubble telescope can take images of very distant objects•However, an incorrect mirror made many of Hubble’s images useless•Image processing techniques were used to fix this....
Finding the outline and shape of image objects, e.g. character recognition.
FACE DETECTION:
Face detection
FACE TRACKING
1)Biological Research: e.g. DNA typing and matching; automatic counting and classification of cell structures in bone and tissue.
2) Defence and Intelligence: e.g. Reconnaissance photo-interpretation of objects in satellite images; target acquisition and missile guidance.
3) Document Processing: e.g. Scanning, archiving and transmission (fax); automatic detection and recognition of printed text (postal sorting office, tax return processing, banking cheques).
4) Law Enforcement Forensics: e.g. Photo-ID kits, criminal photo-search, automatic fingerprint matching, DNA matching and fibre analysis
.
APPLICATIONS OF (DIP):
5) Photography: e.g. altering colours, zooming; adding and subtracting objects to a scene;
6) Remote Sensing: e.g.
Land cover analysis (water, roads, cities and cultivation),
vegetation features (water content and temperature) and crop yield analysis;
3-D terrain rendering from satellite or aircraft data (road and dam planning); fire and smoke detection.
7) Space exploration and Astronomy: satellite navigation and altitude control using star positions.
8) Video and Film Special Effects: Animation,and special effects (Star Wars).
APPLICATIONS OF (DIP):
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image Processing:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image Processing:Image Aquisition
IMAGE ACQUISITION:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image Processing:
Image Enhancement
Image Enhancement:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image restoration:
Digital Image restoration:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image processing, morphological processing:
Digital Image Morphological Processing:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital ImageProcessing,segmentation:
Digital Image Processing: Segmentation:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image Processing object recognition:
Digital Image ProcessingObject Recognition:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image Processing:Representation & Description:
Key Stages in Digital Image Processing: Representation & Description:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image Processing,Image Compression:
IMAGE COMPRESSION:
Image Restoration
Morphological Processing
Segmentation
Object Recognition
Representation & Description
Image Compression
Colour Image Processing
Problem Domain
Image Acquisition
Image Enhancement
Key Stages in Digital Image Processing,Colour Image Processing:
Colour Image Processing:
Image Sub-sampling-Gray
68
Image Re-sampling
69
70
Effect of Sampling and Quantization
250 x 210 samples 256 gray levels
125 x 105 samples
50 x 42 samples
25 x 21 samples
8 gray levels
4 gray levels
Binary image
16 gray levels
Windows Phone Camera Basics• We can build a camera application that can capture still photos.
• We can query device camera capabilities (where there are multiple cameras on the device (front and/or rear), where it has flash, etc.).
• We can get raw camera frames and process them ourselves (encode/decode).
• We can programmatically set flash and focus when taking a photo.
• We can change the resolution of the photo.
• We can continue using the hardware button for the camera shutter to capture a photo and for auto-focus when taking shots outside our application.
• A camera need not be present (for example, front facing camera was not a part of the original devices). Your application would need to handle that condition gracefully.
• To allow our application to have access to the camera capability, you need to specify ID_CAP_ISV_CAMERA in our application manifest file.
• To allow our application to have access to a front facing camera (if one exists), we need to specify ID_HW_FRONTCAMERA in our application manifest file.
Code to Capture images on Windows Phone
using Microsoft.Phone.Tasks;using Microsoft.Phone;private CameraCaptureTask ccTask;public MainPage() { InitializeComponent(); ccTask = new CameraCaptureTask();}private void btnCamera_Click(object sender, RoutedEventArgs e) { ccTask.Show(); }
Code to Capture images on Windows Phone
using Microsoft.Devices;using Microsoft.Xna.Framework.Media;PhotoCamera myCamera;MediaLibrary mediaLibrary;public MainPage() { InitializeComponent(); mediaLibrary = new MediaLibrary();
}private void buttonStartCamera_Click(object sender, RoutedEventArgs e) { myCamera = new Microsoft.Devices.PhotoCamera(CameraType.Primary); myCamera.CaptureCompleted += new EventHandler<CameraOperationCompletedEventArgs>(camera_CaptureCompleted); myCamera.CaptureImageAvailable += new EventHandler<Microsoft.Devices.ContentReadyEventArgs>(camera_CaptureImageAvailable); viewfinderBrush.SetSource(myCamera); }
Code to Preview images on Windows Phone
private void ccTaskCompleted(object sender, PhotoResult pr) { byte[] imgLocal; if (pr.ChosenPhoto != null) { imgLocal = new byte[(int)pr.ChosenPhoto.Length]; pr.ChosenPhoto.Read(imgLocal, 0, imgLocal.Length); pr.ChosenPhoto.Seek(0, System.IO.SeekOrigin.Begin); var bitmapImage = PictureDecoder.DecodeJpeg(pr.ChosenPhoto); this.imgCaptured.Source = bitmapImage; } }
Windows phone camera API’s
Windows phone camera API’s
77
Image manipulation Demo on Matlab (Matrix representation )
RGB to GREY Scale Conversion
RGB to GREY Scale Conversion
80
RGB 2 Grey Matlab & Windows Phone Demo
81
Effect of averaging:
(1) Signal power unchanged
(2) Noise power reduced by 1/M
Conditions:
(1) Signal and noise uncorrelated
(2) Observations are independent
(3) Noise in different observations has the same distribution (iid noise model, independent identically distributed random variables)
Effect of Image Averaging
www.cvcrbd.org
82
Moving Window Transform: Example
originaloriginal 3x3 average3x3 average
Another example Another example
www.cvcrbd.org
83
Moving Window Transform: Example
originaloriginal 3x3 average3x3 average
www.cvcrbd.org
84
Moving Window Transform: Example
originaloriginal 3x3 average3x3 average
www.cvcrbd.org
85
Image Averaging, Example
Averaging 2 images Original image
Averaging 8 images Averaging 16 images
Averaging 32 images Averaging 128 images
www.cvcrbd.org
86
Convolution Examples: Original Images
www.cvcrbd.org
87
Convolution Examples: 33 Blur
111111111
9
1
www.cvcrbd.org
88
Convolution Examples: 55 Blur
1111111111111111111111111
25
1
www.cvcrbd.org
89
Convolution Examples: 99 Blur
111111111111111111111111111111111111111111111111111111111111111111111111111111111
811
www.cvcrbd.org
90
Convolution Examples: 1717 Blur
1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
289
1
www.cvcrbd.org
91
Convolution Examples: 99 Blur using Gaussian mask
0.01124 0.01164 0.01193 0.01211 0.01217 0.01211 0.01193 0.01164 0.011240.01164 0.01205 0.01236 0.01254 0.01261 0.01254 0.01236 0.01205 0.011640.01193 0.01236 0.01267 0.01286 0.01292 0.01286 0.01267 0.01236 0.011930.01211 0.01254 0.01286 0.01305 0.01312 0.01305 0.01286 0.01254 0.012110.01217 0.01261 0.01292 0.01312 0.01319 0.01312 0.01292 0.01261 0.012170.01211 0.01254 0.01286 0.01305 0.01312 0.01305 0.01286 0.01254 0.012110.01193 0.01236 0.01267 0.01286 0.01292 0.01286 0.01267 0.01236 0.011930.01164 0.01205 0.01236 0.01254 0.01261 0.01254 0.01236 0.01205 0.011640.01124 0.01164 0.01193 0.01211 0.01217 0.01211 0.01193 0.01164 0.011242
22
222
1),(
yx
eyxh
www.cvcrbd.org
92
Average filter, Example 1
93
Average filter, Example 2
Beauty Mirror App of IOS
94
Average Filtering Demo (Matlab & Windows Phone)
95
MEDIAN FILTER
96
Effective for salt and pepper noise
- remove noise spikes
987654321
10025,20,20,20,20,20,15,10
100,25,20,20,15,20,20,20,10
Median = 20
ranking
Median Filtering-more later
www.cvcrbd.org
97
Impulse Noise Removal, Example 1
Original image Corrupted image
5 X 5 averaging 5 X 5 medianfiltering
www.cvcrbd.org
98
Impulse Noise Removal, Example 2
Original image 3 X 3 averaging 3 X 3 medianfiltering
www.cvcrbd.org
99
NOICE REDUCTION DEMO (Matlab + Windows Phone)
Effect of Reducing Gray Levels
100
256 128
64 32
16 8
4 2
101
Set OperationsTwo Sets Union Intersection
Complement Difference
102
Set Operations, Example
Complement(Negative)
Union of Constant Imageand Complement Image
The Original Image
103
Logic Operations on Binary Images
BA (A) AND (B)
AND
A B (A) OR (B)
OR
A B (A) XOR (B)
XOR
A B [NOT (A)] AND (B)
NOT-AND
A NOT (A)
NOT
104
Windows phone demo of set operation
105
rk
rk
Histogram
n
nrp kk MNnn
kk 10 Lk
Image Pixel Intensity Histogram
1 1 3 3 7
1 1 3 3 5
1 0 3 3 2
0 0 2 2 2
0 0 2 2 2
0 4321 5 76r
nk
0
2
4
6
r 0 1 2 3 4 5 6 7
nk 5 5 7 6 0 1 0 1
106
Image Histograms, Examples
107
Uneven Pixel Intensity DistributionsThe Histogram of a Grayscale Image
Let I be a 1-band (grayscale) image.
I(r,c) is an 8-bit integer between 0 and 255.
Histogram, hI, of I: – a 256-element array, hI – hI (g), for g = 1, 2, 3, …,
256, is an integer – hI (g) = number of pixels in
I that have value g-1.
Image Histogram
108
The Histogram of a Grayscale
Image
16-level (4-bit) image
black marks pixels w
ith intensity g
low
er R
HC
: nu
mbe
r of
pixe
ls w
ith
inte
nsit
y g
109
The Histogram of a Grayscale Image
Plot of histogram: number of pixels with intensity g
Black marks pixels with intensity g
110
The Histogram of a Grayscale Image
Plot of histogram: number of pixels with intensity g
Black marks pixels with intensity g
111
Image Histogram (Matlab & Windows Phone Demo)
112
. graylevel with
in pixels of
number the
g
I
ghI 1
Luminosity
The Histogram of a Grayscale Image
113
If I is a 3-band image (truecolor, 24-bit) then I(r,c,b) is an integer between 0 and 255. Either I has 3 histograms:
– hR(g+1) = # of pixels in I(:,:,1) with intensity value g– hG(g+1) = # of pixels in I(:,:,2) with intensity value g– hB(g+1) = # of pixels in I(:,:,3) with intensity value g
or 1 vector-valued histogram, h(g,1,b) where– h(g+1,1,1) = # of pixels in I with red intensity value
g– h(g+1,1,2) = # of pixels in I with green intensity
value g– h(g+1,1,3) = # of pixels in I with blue intensity value
g
The Histogram of a Color Image
114
There is one histo-gram per color bandR, G, & B. Luminosity histogram is from 1 band = (R+G+B)/3
There is one histo-gram per color bandR, G, & B. Luminosity histogram is from 1 band = (R+G+B)/3
RIhGIh
BIh
LIh
Luminosity
The Histogram of a Color Image
115
The Histogram of a Color Image
Histogram Processing• Histogram processing re-scales an image intensities so
that the enhanced image histogram follows some desired form.
• The modification can take on many forms:• histogram equalization or• histogram shaping
• e.g. exponential or hyperbolic histogram
116
117
Point Processes: Histogram Equalization
after
1255, gPcrJ I
before
Luminosity
118
Histogram EQ- LUT
The CDF (cummulative distribution) is the LUT for remapping.
The CDF (cummulative distribution) is the LUT for remapping.CDF
LUT
HEQ
119
Histogram EQ
HEQ HEQ
120
10
Image CDFImage CDF Target CDFTarget CDF
LUTLUT
Look Up Table for Histogram Matching
121
Remap an Image:
To Have Two of its Color pdfs Match the Third
original G & B R B & R G R & G B
122
Example: Histogram Matching
original target remapped
123
255,
256,
255
,,,
gcrI
gcrIgcrIcrJ
k
kkk if
if
,
Point Processes: Increase Brightness
0 127 255
0
127
255
g
transform mapping index. band the is and 3210 ,,kg
124
Point Processes: Decrease Brightness
0 127 255
0
127
255
transform mapping
255-g
index. band the is and 3210 ,,kg
crI
gcrI
gcrIcrJ
k
k
kk ,
0,
,,
0,
if
if
,
125
Image Brightness Matlab Demo
126
Image Sharpening Matlab Demo
127
Point Processes: Increase Contrast
.255,
,255,0
,0,
,255
,,
,0
,
crT
crT
crT
crTcrJ
k
k
k
kk
if
if
if
0.1127127,),( acrIacrT kk where, Let
0 127 255
0
127
255
transform mapping 321 ,,k
128
Point Processes: Decrease Contrast
.3,2,10.10
127127,),(
ka
crIacrT kk
and where
,
0 127 255
0
127
255
transform mapping
129
Point Processes: Contrast Stretch
.
,,
Then,
.,max , ,min
,,max , ,min Let
JII
IJJ
JJ
II
mmM
mcrImMcrJ
crJMcrJm
crIMcrIm
0 127 255
0
127
255
transform mappingmI MI
mJ
MJ
130
Information Loss from Contrast Adjustment
origorig
a = 0.5
a = 2.0
lo-clo-c
lo-cRestored
lo-cRestored
hi-chi-c
hi-cRestored
hi-cRestored
131
Information Loss from Contrast Adjustment
origorig
origorig
origorig
lo-clo-c hi-chi-c
lo-clo-c
hi-chi-c
restrest
restrest
lo-clo-c
hi-chi-c
diff diff
diffdiff
abbreviations: original low-contrast high-contrast restored difference
abbreviations: original low-contrast high-contrast restored difference
difference between original and restored low-contrast
difference between original and restored low-contrast
difference between original and restored high-contrast
difference between original and restored high-contrast
132
0 127 255
0
127
255
transform mapping
0.1
255,
255,/1
for
crIcrJ
Point Processes: Increased Gamma
133
Point Processes: Decreased Gamma
0 127 255
0
127
255
transform mappingm M
0.1
255,
255,/1
for
crIcrJ
134
Image Subtraction for Background Removal
g(x,y) = f(x,y) – h(x,y)
www.cvcrbd.org
135
Edge Detection Operator
136
Discrete Laplacian
www.cvcrbd.org
137
Highboost Filtering, Example 3
(a) Image of whole body bone scene
(b) Laplacian of (a)
+
(d) Sobel gredient of (a)
=(c) = (a)+(b)
Sharpened image
(g) = (a)+(f) Sharpened
image
(h) power-law transform of
(g)
www.cvcrbd.org
138
Commonly Used Derivative Masks
www.cvcrbd.org
139
Convolution Examples: Original Images
www.cvcrbd.org
140
Convolution Examples: Vertical Difference
010
020
010
www.cvcrbd.org
141
Convolution Examples: Horizontal Difference
000
121
000
www.cvcrbd.org
142
010141010
Convolution Examples: H + V Diff.
www.cvcrbd.org
143
Edge Detection (Sobel Operator) Matlab Demo
144
Edge Enhancement, Example 1
Original image MagnitudeGradientPrewitt Mask
Original image +Edges with
magnitude > 25
Edges magnitude > 25
www.cvcrbd.org
145
Hybrid Contrast Enhancement Method, Example
Original Image
Histogram Equalization
Highboot Filtering
HighbootFiltering
+Histogram
Equalization
www.cvcrbd.org
148
Medical Image Analysis Demo
149
Brain Storming for Hackathon
150
Idea Selection (One Idea our of Three)
151
Presentation on Hackathon App Ideas
152
Suggestion for converting the idea into an image processing app
Q / A ?
thank you