Post on 22-Nov-2014
transcript
Machine Vision and Image Processing Algorithm –Machine Vision and Image Processing Algorithm Fall 2009
Mario
1
Introducion to OpenCVOpenCV structure and classesOpenCV functionsImage acquisition using webcam
2
OpenCV stands for Open Source Computer Vision LibraryyBeing developed at Intel since 1999Written in C/C++; Contains over 500 functions.Available on Windows, Linux and MacOSX.So far is extensively used in many companies and research centersHome page: www intel com/technology/computing/opencv/www.intel.com/technology/computing/opencv/
3
Computer Vision Market is large and continues to grow
There is no standard API (like OpenGL and DirectX in graphics, or OpenSSL in cryptography), most of CV software is of 3 kinds:◦ Research code (slow, unstable, independent/incompatible data types for every
library/toolbox) b a y/too bo )◦ Very expensive commercial toolkits (like Halcon, MATLAB+Simulink, …)◦ Specialized solutions bundled with hardware (Video surveillance, Manufacturing
control systems, Medical equipment …)
Standard library would simplify development of new applications and solutions much easier.
Special optimization for Intel Architectures:Special optimization for Intel Architectures:◦ Creates new usage models by achieving real-time performance for quite “heavy”
algorithms (like face detection)◦ Makes Intel platforms attractive for CV developers
4
The library is actively used by a large number of companies (such as Intel, IBM, Microsoft, SONY, Siemens, Google,…) and research centers (Stanford MIT CMU Cambridge INRIA etc )(Stanford, MIT, CMU, Cambridge, INRIA etc.)>14000 members of the forum OpenCV@yahoogroups com with average dailyOpenCV@yahoogroups.com, with average daily traffic ~10-20 messages.Community contributes to the project: bug reports, y p j g ppatches, new features (video acquisition, 3d tracking, textures, Python interface)
5
Extract from license:[Copyright Clause][Copyright Clause][Copyright Clause][Copyright Clause]Redistribution and use in source and binary forms, with or without modification,Redistribution and use in source and binary forms, with or without modification,are permitted provided that the following conditions are met:are permitted provided that the following conditions are met:
* Redistribution's of source code must retain the above copyright notice,* Redistribution's of source code must retain the above copyright notice,ed st but o s o sou ce code ust eta t e abo e copy g t ot ce,ed st but o s o sou ce code ust eta t e abo e copy g t ot ce,this list of conditions and the following disclaimer.this list of conditions and the following disclaimer.
* Redistribution's in binary form must reproduce the above copyright notice,* Redistribution's in binary form must reproduce the above copyright notice,this list of conditions and the following disclaimer in the documentationthis list of conditions and the following disclaimer in the documentationggand/or other materials provided with the distribution.and/or other materials provided with the distribution.
* The name of Intel Corporation may not be used to endorse or promote products * The name of Intel Corporation may not be used to endorse or promote products derived from this software without specific prior written permission.derived from this software without specific prior written permission.p p pp p p[Disclaimer][Disclaimer]
I b i f S !I b i f S !In brief: Sure!In brief: Sure!
6
7
CVImage processing
and vision algorithms HighGUIGUI, Image and Video I/Og
CXCOREbasic structures and algoritms,g ,XML support, drawing functions We will mostly
focus on these twoin this presentation
8
IplImage* cvLoadImage(image_path, colorness_flag);loads image from file, converts to color or grayscle, if need, and returns it (or g , g y , , (returns NULL).image format is determined by the file contents.
S I (i th i )cvSaveImage(image_path, image);saves image to file, image format is determined from extension.
BMP JPEG ( i libj ) PNG ( i lib ) TIFF ( i libtiff) PPM/PGM f tBMP, JPEG (via libjpeg), PNG (via libpng), TIFF (via libtiff), PPM/PGM formats are supported.
IplImage* img = cvLoadImage(“picture.jpeg”,-1);
if( img ) cvSaveImage( “picture.png”, img););
9
Windows GUI
Scanline()…
Timage(Bitmap)
Windows GUI
I lI T TBit ()
Image Processing(Conversion, Transform etc )
OpenCV-based Image Processing(ConversionIplImageToTBitmap() Transform, etc.)(Conversion, transform, etc.)
IplImage *cvLoadImage()
Image1->Picture->LoadFromFile()
Image File(bmp, jpg,
)cvSaveImage()
Other image Sourcepng) Source(e.g. Camera)
10
typedef struct CvMat {int type; // data type: CV_8UC1, CV 8UC3, …
widthData pointer
_ ,// … CV_16UC1, …, CV_64FC1int step;union { // aliased pointers to the dataunsigned char* ptr;float* fl; double* db;} data;int rows; // ~width
Pixel/elementheight
int cols; // ~height…} CvMat;
// create large color image as a matrixCvMat* image = cvCreateMat( 1024, 768, CV_8UC3 );// fill the image with red color (blue=0, green=0, red = 255)
Example:
255)cvSet( image, cvScalar(0,0,255));
11
cvGEMM(A,B,a,C,b,D,flags); // generalized matrix product: D=a*Aop(A) *Bop(B) + b*Cop(C)cvSolve(A,b,x,method); // solve Ax=b (linear system or least squares problem)cvSVD, cvSVDBkSb // singular value decomposition and back substitution
// some other functions to look atcvDet, cvTrace, cvInvert, cvDotProduct, cvCrossProduct, cvEigenVVcvDFT(A,B,flags); cvDCT(A,B,flags) // forward/inverse discrete Fourier andCosine transformsCosine transforms
// fast vector transcendental functions (5-20x faster than exp(),log() etc.!)cvExp(A,B); cvLog(A,B); cvPow(A,B,a); /* B=pow(A,a); */cvCartToPolar(X,Y,mag,phase); cvPolarToCart(mag,phase,X,Y);
12
The sample task: collect the positions of all non-zero pixels in the image.Where to store the coordinates?Where to store the coordinates?Possible solutions:◦ Allocate array of maximum size (2*image_width*image_height – a huge value)◦ Use two-pass algorithm◦ Or use a list or similar data structure
// create a sequence of non-zero pixel positionsCvSeq* get non zeros( const IplImage* img, CvMemStorage* storage )q g _ _ ( p g g, g g ){CvSeq* seq = cvCreateSeq( CV_32SC2, sizeof(CvSeq),
sizeof(CvPoint), storage );
for( int i = 0; i < img->height; i++ ) for( int j = 0; j < img->width; j++ )
if( CV_IMAGE_ELEM(img,uchar,i,j) ){
CvPoint pt={j,i}; cvSeqPush( seq, &pt ); }return seq;
}13
Memory storage is a linked list f bl kof memory blocks.
It acts like stack:◦ new data is allocated always at y
the end◦ data is not deallocated until
storage is deleted or cleared.gKey functions:◦ cvCreateMemStorage(block_size),◦ cvReleaseMemStorage(storage);◦ cvReleaseMemStorage(storage);◦ cvClearMemStorage(storage);◦ cvMemStorageAlloc(storage,size);
14
Need a config file?Want to save results of your program to pass it to another one, or look at it another day?Use cvSave, cvLoad and other XML I/O functions from cxcore.
// somewhere deep in your code… you get 5x5// matrix that you’d want to save{ CvMat A = cvMat( 5, 5, CV_32F, the_matrix_data);cvSave( “my matrix xml” &A );
<?xml version="1.0"?><opencv_storage><my_matrix type_id="opencv-matrix"><rows>5</rows>cvSave( “my_matrix.xml”, &A );
}// to load it then in some other program use …CvMat* A1 = (CvMat*)cvLoad( “my_matrix.xml” );
rows 5 /rows<cols>5</cols><dt>f</dt><data>1. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0.0 1 0 0 0 0 0 1 0 0 0 00. 1. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0.0. 1.</data></my_matrix></opencv_storage>
15
16
Create an IplImage object:◦ IplImage * srcImage;IplImage srcImage;Load Image◦ srcImage = cvLoadImage(FileName, -1););
Show Image◦ bitmap = new Graphics::TBitmap();
◦ IplImageToTBitmap(srcImap g p(ge, bitmap);
◦ Image1->Picture->Bitmap = bitmap;
Or◦ cvNamedWindow(“image”);◦ cvShowImage(“image”, srcImage);
17
src
Color conversion can f RGB Color space
src
be performed using cvCvtColor() function
RGB Color space
function.Void cvCvtColor(const dst
CV_RGB_HSV,
CvArr* src, const CvArr* dst, int code);◦ Src: source image HSV Color space
dst
g◦ Dst: destination image◦ Code: color conversion operation
E.g. CV_RGB_XYZ, CV_RGB_HSV, CV_RGB_YCrCb,
HSV Color space
18
19
We can set ROI in an i b i imageimage by using cvSetImageROI function.void
image
widthvoidcvSetImageROI(IplImage* image, CvRect rect);
Image: the source imageROI
x, y width
height◦ Image: the source image◦ Rect : the roi rectangleROI is defined by one point O s de ed by o e po t(x,y), width and the height.
CvRect r = cvRect(int x, int y, int width, intheight);
20
We can use cvCopy to copy selected elements from input
wsrc
ROIse ected e e e ts o putarray to output array.If the passed input array is IplImage, then its ROI fields is used
srchsrcROI
used.Both array must have the same type.void cvCopy( const CvArr*
C A * d t t C A * wdstsrc, CvArr* dst, const CvArr* mask=NULL );◦ src The source array. ◦ dst The destination array.
k O i k 8 bi dst
wdst
hdst◦ mask Operation mask, 8-bit
single channel array; specifies elements of destination array to be changed.
dsthdst
If there is ROI wdst = wroi and hdst = hroiIf there is ROI, wdst wroi and hdst hroiElse wdst = wsrc and hdst = hsrc
21
ROI
22
The simplest way to
jimg
access image pixel is by using a CvScalart t d
i
structure and cvGet2D() functionExample:
cvGet2DS[0]Example:
CvScalar s;s=cvGet2D(img,i,j);
s
[ ]
S[1]
S[2]
S[3]
23
Example of source code for reading an image:
image = cvLoadImage(filenames.at(k).c_str(), 1);
for(int i=0;i<image->height;i++){for(int j=0;j<image->width;j++){CvScalar n;CvScalar n;n = cvGet2D(image, i, j);intensity[j+roiX*i] = n.val[0];
}}
The value of each pixel is stored in array intensity[width*height]intensity[width*height]
24
We can use cvSampleLine() to imgcvSampleLine() to sample one line in the image.int cvSampleLine(
pt1
img
IplImage* img, CvPoint pt1, CvPointpt2, void* buffer );◦ img Image. pt2img Image.◦ pt1 Starting line point.◦ pt2 Ending line point.◦ buffer Buffer to store the
line points; must have enough
pt2
cvSampleLineline points; must have enough size to store MAX(|pt2.x -pt1.x| + 1,|pt2.y - pt1.y|+1) points. buffer
…
25
cvSampleLine(image, cvPoint(0, 120), cvPoint(319, 120), lineProfile->data.ptr);
Start point End point
for(int i=0;i<lineProfile->cols;i++)
{
int height = lineProfile->data.ptr[i];
int posX = i;
cvCircle(procImage, cvPoint(posX, height), 1, whiteLine);
}
Sh I ("P d I " I )cvShowImage("Processed Image", procImage);
26
We can make a binary image by applying threshold to an image.I O CV th h ldIn OpenCV, we can use cvThreshold()Types of threshold:◦ Binary (CV_THRESH_BINARY)
if src(x,y)>threshold d ( ) l 0
threshold
dst(x,y)=max, else 0;◦ Inverted Binary
(CV_THRESH_BINARY_INV)if src(x,y)>threshold dst(x,y)=0, else max;
◦ Truncated (CV THRESH TRUNC)_ _if src(x,y)>threshold dst(x,y)= threshold, else 0;
◦ Threshold to zero (CV_THRESH_TOZERO)
if src(x,y)>threshold dst(x,y)= src(x,y), else 0;( ,y), ;
◦ Inverted Threshold to zero (CV_THRESH_TOZERO_INV)
if src(x,y)>threshold dst(x,y)=0, else src(x,y);
27
IplImage *thresImage = cvCreateImage(cvSize(image->width, image->height), image->depth, 1);
thresImage->origin = 1;cvCvtColor(image, thresImage, CV_BGR2GRAY);cvThreshold(thresImage, thresImage, 125, 255, CV_THRESH_BINARY_INV);
Sh I ("Th h ld" th I )cvShowImage("Threshold", thresImage);cvReleaseImage(&thresImage);
28
There are several image filter that is available in OpenCV such as
CV_BLURp
gaussian filter, median filter, bilateral filter, and blur.We can use this function:void cvSmooth( const CV_GAUSSIAN, CvArr* src, CvArr* dst, intsmoothtype=CV_GAUSSIAN, int param1=3, intparam2=0);param2=0);◦ Src: The source image. ◦ Dst: The destination image. ◦ Smoothtype: CV_BLUR_NO_SCALE, CV_BLUR, CV_GAUSSIAN, CV_MEDIAN, CV_BILATERAL
◦ Param1 and param2 are size of the kernel
29
Draw a box◦ cvRectangle(img, cvPoint(100,100),
(100,100)g g, , ,
cvPoint(200,200), cvScalar(255,0,0), 1);
Draw a circle◦ cvCircle(img, cvPoint(100,100), 20,
cvScalar(0,255,0), 1); (200,200)Draw a line segment◦ cvLine(img, cvPoint(100,100),
cvPoint(200,200), cvScalar(0,255,0), 1);
Draw a set of polylines
( , )
(100,100)20
p y◦ cvPolyLine(img,curveArr,nCurvePts,nCurve
s,isCurveClosed,cvScalar(0,255,255),lineWidth);
Draw a set of filled polygons
(100,100)
◦ cvFillPoly(img,curveArr,nCurvePts,nCurves,cvScalar(0,255,255)); (200,200)
30
Rectangle CircleRectangle Circle
31Line
Opencv has a capability of acquiring images from a camera such as webcam.This is implemented in cvcam class, in the newer
i f th lib it i i l d d i hi h iversion of the library, it is included in highgui.Therefore, we can implement and try our developed algorithm in real time by writing ourdeveloped algorithm in real time by writing our code in the callback function of the image acquisition.q
32
A function that will be automatically called by a generated event.
camera
For example in a camera, a callback function is called Callback()function is called every time a new image is captured by
Callback()
image is captured by the camera Algorithm
33
Include files:◦ #include <cvcam.h>#include <cvcam.h>◦ #include <cv.h>◦ #include <highgui.h>Check the number of camera in the system// R t th b f il bl i th◦ // Returns the number of available cameras in the system.
◦ Int ncams = cvcamGetCamerasCount();◦ // Exit the program when the video camera is not a ailableavailable.
◦ if( ncams < 1 )◦ {◦ Application->MessageBox("Can not find a video
!" " " MB OK)camera!", "error", MB_OK);◦ ShowMessage("Can not find a video camera!");◦ exit(1);◦ }
34
Set the property of the camera◦ cvcamSetProperty(cameraSelected◦ cvcamSetProperty(cameraSelected, CVCAM_RNDWIDTH , &width);
◦ cvcamSetProperty(cameraSelected, CVCAM_RNDHEIGHT , &height);
◦ cvcamSetProperty(cameraSelected, CVCAM_PROP_ENABLE, &t);
◦ cvcamSetProperty(cameraSelected, CVCAM PROP RENDER &t);CVCAM_PROP_RENDER, &t);
◦ cvcamSetProperty(cameraSelected, CVCAM_PROP_CALLBACK, myCallBack);
Start the camera◦ cvcamInit();◦ cvcamStart();
35
36
37
Install the OpenCV Library into your computer.Create a new application with Image, OpenDialog and two Button components.Create an IplImage object;Create an IplImage object;◦ IplImage * image;
When the Button is pressed, initialize the IplImage to load the image from file.◦ Image = cvLoadImage(…); //read from the referencereference
When the second button is pressed, call the function to save the image into another file◦ if( img ) cvSaveImage( “picture.png”, img );
38
Using the same project file, create another button.When this button is pressed, convert the image from Color to Grayscale using this function:◦ cvCvtColor(bgr image, grayscale image,cvCvtColor(bgr_image, grayscale_image, CV_BGR2GRAY);
Add another button to perform filtering of an image. Use this function:◦ cvSmooth(image, image, CV_GAUSSIAN, 7, 7, 1., 1.); // inplace Gaussian7, 1., 1.); // inplace Gaussian smoothing// with 7x7 kernel and σ=1.
39
Create Histogram of the Image by accessing each pixel of the image.◦ Red◦ Green◦ Green◦ Blue◦ Grayscale
40
Create a program to extract ROI from an imageUse cvSetImageROI(IplImage *image, CvRectrect);
Image x,y width
ROIheight
41
Create two IplImage objects.Create an ROI from the first imageCopy the image into the second image
ROIcopy
ROI
Image1Image2
42
Create an Application withCreate an Application with◦ One Timage◦ One Tchart◦ TButtons and TEdit◦ TButtons and TEditDefine these variable:◦ unsigned char * buffer;◦ CvPoint start end;
start end
◦ CvPoint start, end;
Use this function:◦ cvSampleLine( IplImage* img CvPoint pt1 CvPointimg, CvPoint pt1, CvPointpt2, void* buffer );
Assign the value to the Tchart
43
Draw a rectangle in an image◦ First click sets the first point (use CvPoint)First click sets the first point (use CvPoint)◦ Second click sets the second point (use CvPoint)◦ Then draw the rectangle from first point to second point (use
cvRectangle())Draw a circleDraw a circle◦ First click sets the center of the image (use CvPoint)◦ Second click sets the radius of the circle (use CvPoint and calculate the
radius)◦ Then draw the circle using Ci l ( )◦ Then draw the circle using cvCircle( … )Draw a line◦ First click sets the first point (use CvPoint)◦ Second click sets the second point (use CvPoint)◦ Then draw the rectangle from first point to second point (use cvLine( … ))
Next, use the mouse to draw on the image, draw rectangle, line and circleand circle
44
Use the sample code that can be obtained from this url:http://140.124.201.17/opencv/Th fil W b O CV iThe filename: WebcamOpenCV.zipOpen the project using your C++ Builder.Implement these function:Implement these function:◦ color conversion◦ Thresholdingg◦ Drawing histogram of the image
45
Getting Started with OpenCV by VadimPisarevsky (vadim.pisarevsky@intel.com)http://www710.univ-lyon1.fr/~bouakaz/OpenCV-0 9 5/docs/ref/OpenCVRef ImageProcessing htm0.9.5/docs/ref/OpenCVRef_ImageProcessing.htmIntroduction to programming with OpenCV, GadyAgam, http://www.cs.iit.edu/~agam/cs512/lect-g , p gnotes/opencv-intro/opencv-intro.html
46
t6669036@ntut.edu.tw
47