(2d) Matrices
CS101 2012.1
Chakrabarti
Declaration and access
int imat[rows][cols];double dmat[rows][cols]; rows*cols cells allocated of the given type Access just like 1d vectors, example:
• imat[rx][cx] = rvalue;• lvalue = 5 * imat[rx][cx] – 3;
Make sure 0 rx < rows and 0 cx < cols Violations may lead to silent garbage
Chakrabarti
Storage: row major Internally, no distinction between [3][5] and [15] cellNumber(rx, cx) = rx * cols + cx Base address of imat[rx][cx] is base address of imat
plus 4 * cellNumber(rx, cx)
0 1 2 3 4
0 Cell 0, Byte 0
1, 4 2, 8 3, 12 4, 16
1 5, 20 6, 24 7, 28 8, 32 9, 36
2 10, 40 11, 44 12, 48 13, 52 14, 56…59
Demo
Chakrabarti
Basic operations Initialize square matrix to identityfor (int rx = 0; rx < rows; ++rx) { for (int cx = 0; cx < cols; ++cx) { dmat[rx][cx] = (rx == cx)? 1 : 0; } }
Better code uses two loops (why?)• In first rx,cx loop set all elements to zerofor (int rx=0; rx<rows; ++rx) {for (int cx=0; cx<cols; ++cx) { dmat[rx][cx]=0; } }
• In second loop set diagonal elements to onefor (int dx=0; dx<rows; ++dx) { dmat[dx][dx]=1; }
Chakrabarti
Basic operations Initialize to upper triangular matrix with 1s
for (int rx = 0; rx < rows; ++rx) {
for (int cx = 0; cx < cols; ++cx) {
dmat[rx][cx] = (rx <= cx)? 1 : 0;
}}
cx
r
x rx <= cx
Chakrabarti
Basic operations Transpose a square matrix Don’t double transpose back to square one!
for (int rx = 0; rx < rows; ++rx) { for (int cx = rx+1; cx < cols; ++cx) {
float tmp = fmat[rx][cx]; fmat[rx][cx] = fmat[cx][rx]; fmat[cx][rx] = tmp; }}
No need to transpose diagonal or below
Swap [rx][cx] with [cx][rx]
Chakrabarti
Transpose demo
1 2 3
4 5 6
7 8 9
1 4 3
2 5 6
7 8 9
1 4 3
2 5 6
7 8 9
1 4 7
2 5 6
3 8 9
1 4 7
2 5 6
3 8 9
1 4 7
2 5 8
3 6 9
Chakrabarti
Matrix vector multiplication code
float A[rows][cols];float x[cols], y[rows];// fill up A and xfor (int rx=0; rx < rows; ++rx) { y[rx] = 0; for (int cx=0; cx < cols; ++cx) {
y[rx] += A[rx][cx] * x[cx]; }}
Chakrabarti
Matrix vector multiplication demo
for (int rx=0; rx < rows; ++rx) { y[rx] = 0; for (int cx=0; cx < cols; ++cx) { y[rx] += A[rx][cx] * x[cx]; }}
a00 a01 a02
a10 a11 a12
x0
x1
x2
y0
y1=
a00*x0+a01*x1+a02*x2x0
x1
x2
a10*x0+a11*x1+a12*x2
Chakrabarti
Matrix as transformation
Consider all x’s whose arrow tips lie on the unit circle (i.e., norm=1)
After transformation by multiplying by A, what is the locus of the arrow tips of resulting y’s?
preimagex
yimage
Demo
Chakrabarti
Mapping a circle to…
Preimages onunit circle
Images y aftertransformation,on an ellipse
Two directionsto rule them all
Chakrabarti
Stretched but not turned The two directions that do not
turn when scaled by A are eigen vectors u0, u1
Length of semi-major and semi-minor axes are eigen values 0 > 1
u0 and u1 are perpendicular to each other, have unit norm
Can be used as new coordinate axes
Consider a vector x with coordinates (1, c) along new axes u0 and u1
Chakrabarti
Power iterations Find largest eigenvalue and corresponding
eigenvector u of matrix A Solves Au = u Initialize x arbitrarily Repeat
• y = Ax
• x = y / ||y||2 Why does this work? Demo using Scilab
Call the sequence ofvalues of x and y asx(0), y(0), x(1), y(1), etc.
Chakrabarti
Big crunchUnitnorm
Chakrabarti
When to stop When x(k-1) and x(k) are sufficiently close to
each other ||x(k-1)-x(k)||2 is smaller than some small For you to explore: how to find u1, 1?
Let’s write Boost code for power iterations Building blocks we need
• Matrix, vector• Vector difference, norm• Matrix vector multiplication
All provided by Boost UBLAS library
Chakrabarti
Boost code
for (;;) { vector<double> y = prod(A, x); vector<double> newX = y/norm_2(y);
if (norm_2(newX - x) < eps) { break; } x = newX;}
Already implementedIn Boost UBLAS library
Chakrabarti
Matrix-matrix multiplicationfloat amat[lsize][msize], bmat[msize][nsize], cmat[lsize][nsize];
for (int crx=0; crx<lsize; ++crx) { for (int ccx=0; ccx<nsize; ++ccx) { cmat[crx][ccx] = 0; for (int abx=0; abx<msize; ++abx) { cmat[crx][ccx] += amat[crx][abx] * bmat[abx][ccx];
} }}
Like matrix-vector multiplicationBut x has many columnsTriple nested loopTime required is lsize*msize*nsize
Chakrabarti
Least squares fit
Chakrabarti
Least square objective
But we don’t know how to invert matrices yet
Chakrabarti
Iterative solution to find a zero of g(w) Use an iterative scheme instead Fix an arbitrary step size>0 Start with some w(0)
Repeat for t = 1, 2, …• Set w(t) = w(t-1) - g(w(t-1))
Until w(t) and w(t-1) are very close
In this specific problem, guaranteed to converge
Demo
Chakrabarti
Determinant Adding a multiple of one row to another
leaves the determinant unchanged Exchanging a pair of rows or columns
causes the determinant to be multiplied by minus one
Determinant of a upper triangular matrix (all zeros below diagonal) is the product of diagonal elements
Chakrabarti
Determinant: approach
Chakrabarti
Determinant code sketch// matrix<double> A filled w/ valuesfor (int dx=0; dx<A.size1(); ++dx) { // make A(dx,0) through A(dx,dx-1) zeros for (int sx=0; sx<dx; ++sx) { double fac = A(dx,sx)/A(sx,sx); for (int cx=0; cx<A.size2(); ++cx) { A(dx,cx) -= fac * A(sx,cx); } //cx } //sx} //dx// finally multiply together diagonals
Chakrabarti
Near-zero diagonal elements If a diagonal element (d, d) becomes too
small, we look for a lower row z where dth column is not small
Then we swap rows d and z And remember to adjust the sign later Called a “pivot” step What if there are no such rows below d? More advanced pivoting needed Sometimes the determinant may be zero
• All pivoting attempts will fail
Chakrabarti
Toward better pivoting
Goal is to zero out first column below diagonal
for (int rx = 1; rx < A.size1(); ++rx) { double fac = A(rx,0) / A(0,0); for (int cx = 0; cx < A.size2(); ++cx) { A(rx, cx) -= fac * A(0, cx); }}
0
0
0
0
0
0 0
0 0
0 0
0
0 0
0 0 0
0 0 0
Chakrabarti
Nested loopsfor (int dx=0; dx < A.size1(); ++dx) {
// zero A(dx+1,dx) through A(n-1,dx) for (int rx=dx+1; rx < A.size1(); ++rx) {
double fac = A(rx,dx) / A(dx,dx); for (int cx=0; cx < A.size2(); ++cx) { A(rx, cx) -= fac * A(dx, cx); }//cx }//rx}//dx
If |A(dx,dx)| < do a pivot step
Chakrabarti
Modularity Thus far we have written most of our code in
the main function Invoking library functions like sqrt or log as
needed Increasingly important to modularize our own
code• Implement once, reuse from many programs• Debug part by part, maintain in one place• Read code and think more clearly• Separate specification from implementation
Chakrabarti
Detour: swap function Swap values in two variables of any type “Polymorphic” function like +, *, /
template <class T>
void swap(T& a, T& b) { T tmp = a; a = b; b = tmp;}
Works for any type T
Don’t forget the &
Chakrabarti
swapRowstemplate <class T>
void swapRows(matrix<T>& mat, int r1, int r2){ for (int cx=0; cx<mat.size2(); ++cx) {
swap(mat(r1,cx), mat(r2,cx)); }}
Chakrabarti
pivotIfNeededbool pivotIfNeeded(matrix<double>& A, int dx, double& sign){ if (abs(A(dx,dx)) > eps) return true; for (int rx = dx+1; rx < A.size1(); ++rx) { if (abs(A(rx,dx)) > eps) {
swapRows(A, dx, rx); sign = -sign; return true; } } return false; // failed to find a pivot}
In/OutIn/Out
In
Return true iff pivoting not needed,
or successful
Chakrabarti
Solving a system of equations Given Ax = y where A is nn; x, y are n1 Goal is to solve for x Operations we have done so far on A do not
change the constraints on x A turned into an upper triangular matrix Code to manipulate rows of y in sync with A Can solve backward: find xn1, then xn2, …
Chakrabarti
Pivoting y in sync with A Earlier, pivot function had inputs A, dx, sign New inputs will be A, y, dx No need to keep track of signbool pivotIfNeeded(matrix<double>& A, vector<double>& y, int dx) { if (abs(A(dx,dx)) > eps) return true; for (int rx = dx+1; rx < A.size1(); ++rx) { if (abs(A(rx,dx)) > eps) { swapRows(A, dx, rx); swap(y(dx), y(rx)); return true; } } return false; }
Chakrabarti
Nested loops including yfor (int dx=0; dx < A.size1(); ++dx) { pivotIfNeeded(A, y, dx); // zero A(dx+1,dx) through A(n-1,dx) for (int rx=dx+1; rx < A.size1(); ++rx) {
double fac = A(rx,dx) / A(dx,dx); for (int cx=0; cx < A.size2(); ++cx) { A(rx, cx) -= fac * A(dx, cx); }//cx y(rx) -= fac * y(dx); }//rx}//dx
Chakrabarti
Last step// A is already upper triangular// y available, x allocatedfor (int ix=x.size()-1; ix>=0; --ix) { x(ix) = y(ix); for (int jx=ix+1; jx<x.size(); ++jx) {
x(ix) -= A(ix, jx) * x(jx); } x(ix) /= A(ix, ix);}
Chakrabarti
Inverse Once we can solve Ax = y, inverting a matrix
is no big deal (Conceptually) invoke solver n times§
First with y =(1, 0, …, 0)T
Then with y=(0, 1, …, 0)T
And so on up to y=(0, 0, …, 1)T
I.e., columns of the identity matrix Collect x’s to form A-1
§Obviously, don’t triangularize A n times over! Pass in all different y’s in one shot
Chakrabarti
for (int dx=0; dx < A.size1(); ++dx) { pivotIfNeeded(A, R, dx); // zero A(dx+1,dx) through A(n-1,dx) for (int rx=dx+1; rx < A.size1(); ++rx) {
double fac = A(rx,dx) / A(dx,dx); for (int cx=0; cx < A.size2(); ++cx) { A(rx, cx) -= fac * A(dx, cx); R(rx, cx) -= fac * R(dx, cx); }//cx}//rx}//dx
Nested loops, version 3
R initialized to the identity matrix;its columns are our initial y’s
Chakrabarti
Image processing applications If you look at any “bitmap”
image up close, you see it is a 2d array of pixels
Each pixel has a red, green blue intensity
Mix RGB primary colors An intensity is an integer
between 0 and 255 (one unsigned byte)
Called “24 bit color”
Chakrabarti
Object identification Babies identify hundreds of objects by age 1 They get shapes before they get colors Need to perceive boundaries of objects Significant change of color among neighboring
pixels If you are very
different from theaverage of yourneighbors, you may be an edge pixel
Chakrabarti
Edge detection Find (weighted) average of neighbor pixels Compare with my value If difference large, (likely to be) edge
i,j
Chakrabarti
Contrast enhancement An image may have been captured under
less than ideal circumstances• Scene lighting, camera calibration• Tissue density and water content
Use statisticaltechniques to“correct” theimage
Critical in medicalimaging and diagnosis
Chakrabarti
Code details EasyBMP.h publishes function signatures for
reading and writing .bmp image files EasyBMP.cpp implements the functions Effectively turns them into three in-memory
pixel matrices with R, G, B intensities Compile EasyBMP.cpp to EasyBMP.o ahead
of time, once Compile Edgy.cpp and link with EasyBMP.o
to produce Edgy.exe Compile Enhance.cpp … Enhance.exe