Scientific Computing with MATLAB3. Manipulating Data Sets
Dra. K.-Y. Daisy FanDepartment of Computer ScienceCornell UniversityCornell UniversityIthaca, NY, USA
UNAM IIM 2012
Working with data and files
Data that you have created or manipulated using MATLAB can be saved to a data file for future use. MATLAB can be saved to a data file for future use. Or you may use data provided by others. The MATLAB internal format mat is convenient The MATLAB internal format, mat, is convenient
if you and the future user uses MATLABTh ASCII i i l b f i The ASCII type is universal, but often requires more low-level work
Spreadsheet format is commonly used Image data comes in many form, with JPEG as one g y J
of the most commonly used.3
By the end of this lecture, you will be able to …
Save Workspace variable values to MATLAB’s .mat formatted file; import .mat data to .mat formatted file; import .mat data to Workspace
Save Workspace matrices to ASCII files; import Save Workspace matrices to ASCII files; import ASCII files as matricesI d f d h Import data from a spreadsheet
Import an image file, process the data, and write back an image file
Work with irregularly formatted ASCII filesg y
4
Saving Workspace data to file
save filename
Simple way to save data with basic formats Simple way to save data with basic formats Options include file type and saving select
i bl S lvariables. Some examples:% myFile.mat stores all variable values:sa e m Filesave myFile
% myFile.mat stores values of variables x,u:save myFile x, uy ,
% myFile.txt stores values of matrices ma and% mi in ASCII (plain text) format:
Fil t t i ii
5
save myFile.txt ma mi -ascii% matrices must have same no. of columns!
Importing .mat data into Workspace
load filenameOptions include loading only specified variables Examples:Options include loading only specified variables. Examples:% Load all variable values from myFile.mat:load myFileoad y e
% Load values from myFile.txt as a matrix:load myFiley% Any file with an extension that is not% .mat is treated as ASCII file.% Matrix name now in workspace is myFile% Matrix name now in workspace is myFile.
% Load values from myFile.dat as a matrix:load myDat dat
6
load myDat.dat% Matrix name now in workspace is myDat
Importing data from a spreadsheet (xls)
[ar, txt, raw] = xlsread(filename)
Options include specifying the range sheet Options include specifying the range, sheet Filename does not need to include extension .xls Which versions of Excel spreadsheets?
If your machine has Excel for windows—all versions supported by that version of Excel, e.g., xls, xlsx, …
If your machine does not have Excel for windows—basic functionality only (no range, no sheet, i.e., Excel 97-2003)
7
Importing data from a spreadsheet (xls)
[ar, txt, raw] = xlsread(filename)
ar – a numeric array storing only the numeric l i d t i t d N Nvalues; non-numeric data is stored as NaN
txt – a cell array storing only the text (non-numeric data); non-text is stored as the empty string, ’’
raw – a cell array storing all the data; each cell is numeric or text depending on the datap g
8
Single quotes enclose strings in Matlab
Anything enclosed in single quotes is a string (even f l k l k h l )if it looks like something else)
‘100’is a character array (string) of length 3 100 is a numeric value ‘pi’ is a character array of length 2 pi is the built-in constant 3.1416… ‘x’ is a character (vector of length 1) x may be a variable name in your program
9
Strings are vectorsVectors Strings
Assignmentv= [7 0 5];
Indexing
g Assignment
s= ‘hello’;
Indexing Indexingx= v(3); % x is 5v(1)= 1; % v is [1 0 5]
(2 3) % i [0 5]
Indexingc= s(2); % c is ‘e’s(1)= ‘J’; % s is ‘Jello’t (2 4) % t i ‘ ll’w= v(2:3); % w is [0 5]
: notationv= 2:5; % v is [2 3 4 5]
t= s(2:4); % t is ‘ell’
: notations= ‘a’:‘g’; % s is ‘abcdefg’
Appendingv= [7 0 5];v(4)= 2; % v is [7 0 5 2]
Appendings= ‘duck’;s(5)= ‘s’; % s is ‘ducks’( ) ; [ ]
Concatenationv= [v [4 6]];
% v is [7 0 5 2 4 6]
( ) ;
Concatenations= [s ‘ quack’];
% s is ‘ducks quack’
10
% v is [7 0 5 2 4 6] % s is ducks quack
Some useful string functionsstr= ‘Cs 1112’;str= Cs 1112 ;
length(str) % 7isletter(str) % [1 1 0 0 0 0 0]isspace(str) % [0 0 1 0 0 0 0]lower(str) % ‘cs 1112’lower(str) % ‘cs 1112’upper(str) % ‘CS 1112’
ischar(str)% Is str a char array? True (1)
t ( t (1 2) ‘ ’)strcmp(str(1:2),‘cs’)% Compare strings str(1:2) & ‘cs’. False (0)
strcmp(str(1:3),‘CS’)
11
strcmp(str(1:3), CS )% False (0)
Characters arrays
We have used strings already: n= input(‘Next number: ’) sprintf(‘Answer is %d’, ans)
A string is made up of individual characters, so a string is g p , ga 1-d array of characters
‘UNAM iim2012!’ is a character array of length 13; it has 7 letters, 4 digits, 1 space, and 1 symbol.
‘U’‘N’‘A’ ‘ ’‘M’ ‘i’ ‘m’‘2’‘0’‘1’‘2’‘!’‘i’
Can have 2-d array of characters as well‘u’‘n’‘a’‘m’
‘2’‘0’‘1’‘2’ 2×4 matrix
Matrix vs. Cell Array A cell array is a special array
Vectors and matrices store values of the same type in
whose individual components may contain different types of datayp
all componentsdata
-4 -1 ‘ ’‘ ’ ‘ ’3.1
2‘c’ ‘o’ ‘m’ ‘ ’ ‘s’
‘1’ ‘1’ ‘1’ ‘2’ ‘ ’
-4 -1 .
‘m’‘c’ ‘o’
-1
9
1 1‘ ’ ‘ ’ ‘L’ ‘A’ ‘B’
‘M’ ‘a’ ‘t’ ‘ ’ ‘ ’
.91
5 7
1.1
.4‘M’ -1 74 x 5matrix5 x 1
matrix
13
3 × 2 cell array
Cell Arrays of Strings
C= { ‘Alabama’,’New York’,’Utah’}
C ‘Alabama’ ‘New York’ ‘Utah’
C= { ‘Alabama’;’New York’;’Utah’}
C i h ‘Alabama’
‘New York’
C
M= [‘Alabama ’; ...
Contrast with 2-d array of characters
‘Utah’1-d cell array of
‘New York’; ...‘Utah ’]
‘A’ ‘l’ ‘a’ ‘b’ ‘a’ ‘m’ ‘a’ ‘ ‘My
strings ‘N’ ‘e’ ‘w’ ‘ ‘ ‘Y’
‘U’ ‘t’ ‘h’
‘o’ ‘r’ ‘k’
‘ ‘‘a’ ‘ ‘‘ ‘‘ ‘
M
Use braces { } for creating and addressing cell arrays
Matrix
Create
Cell Array
Create Create
m= [ 5, 4 ; …1 2
Create
C= { ones(2,2), 4 ; …‘ b ’ (3 1) 1, 2 ; …
0, 8 ]‘abc’ , ones(3,1) ; …9 , ‘a cell’ }
Addressing
m(2 1)= pi
Addressing
C{2 1}= ‘ABC’m(2,1) pi C{2,1} ABCC{3,2}= pidisp(C{3,2})
15
A picture as a matrix
1458-by-2084
150 149 152 153 152 155150 149 152 153 152 155151 150 153 154 153 156153 151 155 156 155 158154 153 156 157 156 1591 6 1 1 8 1 9 1 8 161
16
156 154 158 159 158 161157 156 159 160 159 162
Images can be encoded in different ways
Common formats include JPEG: Joint Photographic Experts GroupJ J g p p p GIF: Graphics Interchange Format
Data are compressed p We will work with jpeg files:
imread: read a .jpg file and convert it to a “normal imread: read a .jpg file and convert it to a normal numeric” array that we can work with
imwrite: write an array into a .jpg file (compressed data)
17
Grayness: a value in [0..255]
0 = black255 = white
These are integer valuesType: uint8Type: uint8
150 149 152 153 152 155150 149 152 153 152 155151 150 153 154 153 156153 151 155 156 155 158154 153 156 157 156 1591 6 1 1 8 1 9 1 8 161
18
156 154 158 159 158 161157 156 159 160 159 162
Reading a jpeg file and displaying the image
% Read jpg image and convert to % an array P
P = imread(‘bwduck.jpg');jpg
% Show the data in 3-d array P as% Show the data in 3-d array P as % an image
imshow(P)
19
A color picture is made up of RGB matrices 3-d array114 114 112 112 114 111 114 115 112 113114 113 111 109 113 111 113 115 112 113115 114 112 111 111 112 112 111 112 112116 117 116 114 112 115 113 112 115 114113 112 112 112 112 110 111 113 116 115115 115 115 115 113 111 111 113 116 114112 113 116 117 113 112 112 113 114 113115 116 118 118 113 112 112 113 114 114115 116 118 118 113 112 112 113 114 114116 116 117 117 114 114 112 112 114 115
153 153 150 150 154 151 152 153 150 151153 152 149 147 153 151 151 153 150 151154 153 151 150 151 152 150 149 150 150155 156 155 152 152 155 151 150 153 153151 150 150 150 150 148 149 151 152 151153 153 153 153 151 149 149 151 152 150150 151 152 153 151 150 150 151 152 151153 154 154 154 151 150 150 151 152 152154 154 153 153 149 149 150 150 152 153
212 212 212 212 216 213 215 216 213 213212 211 211 209 215 213 214 216 213 213213 212 210 209 212 214 213 212 213 212213 212 210 209 212 214 213 212 213 212214 215 214 214 213 216 214 213 215 212213 212 212 212 212 210 211 213 214 211215 215 216 216 213 211 211 213 212 210212 213 214 215 213 212 212 213 214 213215 216 216 216 213 212 212 213 214 214216 216 215 215 213 213 213 213 214 215
0 ≤ A(i,j,1) ≤ 255
0 ≤ A(i,j,2) ≤ 255E.g., color image data is stored in a 3-d array A:
0 ≤ A(i,j,3) ≤ 255
20
A color picture is made up of RGB matrices 3-d array
Color image 3-d Array
0 ≤ A(i,j,1) ≤ 255
0 ≤ A(i,j,3) ≤ 255
0 ≤ A(i,j,2) ≤ 255
O ti i t t ti
( ,j, )
Operations on images amount to operations on matrices!
21
Problem: produce a negative
22
Problem: produce a negative
“Negative” is what we say, but all color values are positive numbers!p
Think in terms of the extremes, 0 and 255. Then the “negative” just means the opposite side.g j pp
So 0 is the opposite of 255;1 … 254;1 … 254;5 … 250;30 225;30 … 225;x … 255-x
23
function newIm = toNegative(im)% newIm is the negative of image im% im, newIm are 3-d arrays; each component is uint8
[nr,nc,np]= size(im); % dimensions of imnewIm= zeros(nr,nc,np,’uint8’); % initialize newIm
for r= 1:nrfor r= 1:nrfor c= 1:nc
for p= 1:npnewIm(r,c,p)= ___________________;
endend
end
24
function newIm = toNegative(im)% newIm is the negative of image im% im, newIm are 3-d arrays; each component is uint8
[nr,nc,np]= size(im); % dimensions of imnewIm= zeros(nr,nc,np,’uint8’); % initialize newIm
for r= 1:nrfor r= 1:nrfor c= 1:nc
for p= 1:np2newIm(r,c,p)= 255 - im(r,c,p);
endend
end function newIm = toNegative(im)% newIm is the negative of image im% im, newIm are 3-d uint8 arrays
25
newIm = 255 – im; % vectorized code!
Finding Edges
26
What is an edge?
Near an edge, grayness values h b lchange abruptly
200 200 200 200 200 200200 200 200 200 200 100200 200 200 200 100 100200 200 200 200 100 100200 200 200 100 100 100200 200 100 100 100 100200 100 100 100 100 100
27
General plan for showing the edges in in image
Identify the “edge pixels” Highlight the edge pixels Highlight the edge pixels
make edge pixels white; make everything else black
200 200 200 200 200 200200 200 200 200 200 100200 200 200 200 100 100200 200 200 200 100 100200 200 200 100 100 100200 200 100 100 100 100200 100 100 100 100 100
28
General plan for showing the edges in in image
Identify the “edge pixels” Highlight the edge pixels Highlight the edge pixels
make edge pixels white; make everything else black
200 200 200 200 200 200200 200 200 200 200 100200 200 200 200 100 100BLACK
200 200 200 200 100 100200 200 200 100 100 100200 200 100 100 100 100BLACK200 100 100 100 100 100BLACK
29
Finding Edges
30
The Rate-of-Change-Array
Suppose A is an image array with integer values between 0 and 255.Let B(i,j) be the maximum difference between jand its eight neighbors.So B(i,j) is the maximum value inSo B(i,j) is the maximum value in
A( (1 i 1) i ( i+1)A(max(1,i-1):min(m,i+1),...
max(1,j-1):min(n,j+1)) - A(i,j)
31
Neighborhood of A(i,j)
Rate-of-change example
90 6581Rate-of-change at
iddl i l i 3059
58
60
56
62
57
middle pixel is 30
5856 57
Be careful! In “uint8 arithmetic”57 – 60 is 0
32
The Pixel (i,j) Neighborhood
iMin = i-riMax = i+riMax = i+rjMin = j-rjMax = j+rC = A(iMin:iMax,jMin:jMax)( ,j j )
r = 1 r = 2
Am
33
r = 1 r = 2n
The Pixel (i,j) Neighborhood
iMin = max(1,i-r)iMax = min(m i+r)iMax = min(m,i+r)jMin = max(1,j-r)jMax = min(n,j+r)C = A(iMin:iMax,jMin:jMax)( ,j j )
r = 1 r = 2
Am
34
r = 1 r = 2n
Edge finding: Rate-of-change matrix
A: grayness values B: rate-of-change values
Aij
i
j
Bij
Ν|| AAB
j
Let N be the set of subscripts (p,q) that is the neighborhood of (i,j)
Ν),(||max qpijpqij AAB
Edge finding: Rate-of-change matrix
A: grayness values B: rate-of-change values E: edge values
Aij
i
j
Bij Eij
Ν|| AAB
j
Let N be the set of subscripts (p,q) that is the neighborhood of (i,j)
Ν),(||max qpijpqij AAB
High rate of change Abrupt change in grayness An edgeg g p g g y g
ij
ij BblackBwhite
Eifif
ijBblack if
function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.jpg g g g jpg% At each pixel, if rate-of-change > tau% then the pixel is considered to be on an edge.A i d(j I ) fi i h (A)A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);[m,n] = size(A);
Built-in function to convert to grayscale [ , ] ( );
B = zeros(m,n,’uint8’);for i = 1:m
convert to grayscale. Returns 2-d array.
for j = 1:n
B(i j) = ?????B(i,j) = ?????
end
38
end
Recipe for rate-of-change B(i,j)
% The 3-by-3 subarray that includes A(i,j) % and its 8 neighbors (for an interior pixel)Neighbors = A(i-1:i+1,j-1:j+1);
% Subtract A(i,j) from each entryDiff= abs( double(Neighbors) – double(A(i,j)) );
% Compute largest value in each columncolMax = max(Diff);co a a ( );% Compute the max of the column max’sB(i,j) = max(colMax);
39
function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.% At each pixel, if rate-of-change > tau% At each pixel, if rate of change > tau% then the pixel is considered to be on an edge.A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);[m,n] = size(A);B = zeros(m,n,’uint8’);for i = 1:m
for j = 1:nNeighbors = A(max(1,i-1):min(i+1,m), ...
max(1,j-1):min(j+1,n));
endend
41
end
function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.% At each pixel, if rate-of-change > tau% At each pixel, if rate of change > tau% then the pixel is considered to be on an edge.A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);[m,n] = size(A);B = zeros(m,n,’uint8’);for i = 1:m
for j = 1:nNeighbors = A(max(1,i-1):min(i+1,m), ...
max(1,j-1):min(j+1,n));B(i j) ( ( b (d bl (N i hb )B(i,j)=max(max(abs(double(Neighbors)– ...
double(A(i,j)))));
endend
42
end
“Edge pixels” are now identified; display them with maximum brightness (255)
A
1 1 1 1 1 11 1 1 1 1 1
A
if B(i j) > tau
threshold1 1 1 1 1 11 1 1 1 90 901 1 1 90 90 901 1 90 90 90 90
if B(i,j) > tauB(i,j) = 255;
end 1 1 90 90 90 901 1 90 90 90 90
B(i j)0 0 0 0 0 00 0 0 89 89 89
B(i,j)0 0 0 0 0 00 0 0 255 255 255
0 0 89 89 0 00 89 89 0 0 00 89 0 0 0 0
0 0 255 255 0 00 255 255 0 0 00 255 0 0 0 0
43
0 89 0 0 0 00 89 0 0 0 0
0 255 0 0 0 00 255 0 0 0 0
function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.% At each pixel if rate of change > tau% At each pixel, if rate-of-change > tau% then the pixel is considered to be on an edge.A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);A = rgb2gray(A);[m,n] = size(A);B = zeros(m,n,’uint8’);for i = 1:mfor i = 1:m
for j = 1:nNeighbors = A(max(1,i-1):min(i+1,m), ...
max(1 j 1):min(j+1 n));max(1,j-1):min(j+1,n));B(i,j)=max(max(abs(double(Neighbors)– ...
double(A(i,j)))));if B(i j) > tif B(i,j) > tau
B(i,j) = 255;end
44
endend
function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.% At each pixel if rate of change > tau% At each pixel, if rate-of-change > tau% then the pixel is considered to be on an edge.A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);A = rgb2gray(A);[m,n] = size(A);B = zeros(m,n,’uint8’);for i = 1:mfor i = 1:m
for j = 1:nNeighbors = A(max(1,i-1):min(i+1,m), ...
max(1 j 1):min(j+1 n));max(1,j-1):min(j+1,n));B(i,j)=max(max(abs(double(Neighbors)– ...
double(A(i,j)))));if B(i j) > tif B(i,j) > tau
B(i,j) = 255;end
45
endendimwrite(B,jpgOut,’jpg’)
tau = 30
46
Edge finding: Effect of edge threshold,
tau = 20