Date post: | 19-Feb-2017 |
Category: |
Documents |
Upload: | sagarika-muthukumarana |
View: | 42 times |
Download: | 5 times |
1
Visual Based Product Identification System
by
W.F.R.Madushanka (E/11/252)
M.S.P.Muthukumaranage(E/11/267)
Supervised by Mr.Mahanama Darmawardhana
Interim Report
Submitted in partial fulfillment of the requirements
for course unit PR 406: Industrial Assignment
May 31, 2016
Department of Production Engineering
Faculty of Engineering
University of Peradeniya
1
Declaration
Declaration We hereby declare that this project report is our original work and is written by us in its
entirety. We have duly acknowledged all the sources of information including all images and tables
which have been used in this report
. …..........................................
M.S.P.Muthukumaranage
…...........................................
W.F.R.Madushanka
2
Acknowledgements
This report carried out the contain of our project Vision based product identification system for the PR
406 cause industrial assignment for the completion of degree of BSc Engineering, University of
Peradeniya.
First of all, our sincere thank goes to the supervisor of the project Mr.MahanamaDarmawardhana
Faculty of Engineering, University of Peradeniya for providing us this opportunity to gain a good
knowledge and experience in project.
We also grateful to everyone who supported us for the successful completion of our project other staff
members of engineering workshop and production department.
M.S.P Muthukumaranage
W.F.R Madusanka
Faculty of Engineering,
University of Peradeniya,
Peradeniya.
3
Executive Summary
The eye is one of the most important organs of the human body. So our skills greatly depend on our
capability to see, detect and categorize objects, and to estimate distances. Most of the industrial jobs
centralized on our ability of visual perception. Today machine vision has improved in production
technologies more and more often extend well beyond the limits of human visual capacities. This is
where machine vision technology comes in.
Our project is on vision based product identification system. In this project machine able to identify
objects based on colour and shape. Our aim is to build a system that can detect, recognize objects
according to the colour shape and barcodes. The whole process should be done in real-time, the thing
that necessities employing fast and efficient algorithms. In addition, the actual implementation of our
system should take practicality and ease of use into account. This is because a system as ours is
intended to be used in daily life and hence needs to be both simple and efficient to use.
There are so many methods to solve this problem. In our project single board computer use OpenCV
and Python as software. OpenCV is open source computer vision library. OpenCV and Python is one
of the best combinations. Instead of personal computer we use raspberry pi minicomputer.
In our project we successfully completed and implemented a vision system that can move objects
through conveyor and detect shape colour in real time.
4
Table of Content
1. Acknowledgement 2
2. Executive Summary 3
Chapter 1 Introduction
1.1 Overview 6
1.2 Problem Description 7
1.3 Software and hardware 7
Chapter 2 Solution Methodology 12
2.1 Install software to raspberry-pi mini computer 13
2.2 Colour detection method 14
2.3 Shape detection method 14
2.4 Process flow chart 15
2.5 Project layout 16
2.6 Limitations 16
Chapter 3 Results
3.1 Colour detection results 17
3.2 Shape detection results 18
Chapter 4 Deliverables 20
Chapter 5 Conclusion 21
References 22
Appendix
5
List of figures
1. Fig 1. Raspberry pi minicomputer 7
2. Fig 2. Light room. 9
3. Fig 3. Ring light. 9
4. Fig 4. Spot light. 9
5. Fig 5. Conveyor. 10
6. Fig 6. RGB colour wheel 13
7. Fig 7. Relationship between hue and saturation 13
8. Fig 8. Relationship among hue saturation value 14
9. Fig 9. Project layout 16
10. Fig 10. Red colour identification 17
11. Fig 11. Bluecolour identification 17
12. Fig 12. Square identification 18
13. Fig 13. Triangular identification 18
14. Fig 14. Circle identification 19
6
CHAPTER 1 – Introduction
1.1 Over view
With the rapid development of the production and manufacturing world demands on production
quality and production rate should be increased. For that instead of human eye and brain, have to use
another vision system. According to the above necessity various vision based product identification
system came to the manufacturing world.
Vision based product identification system is considered with the use of image processing technique.
Image processing is a method of analyzing and manipulating images with a computer or can be define
as capture real image and extract useful information from it. In the image processing input and output
both are images. Basically there are three steps include in image processing.
1. Importing the image via image acquisition
2. Analyzing and manipulating the image
3. Extract useful information and show output image with that information.
There are two types of image processing methods. Analogue image processing and digital image
processing. In the past most of image processing was done by using analogue image processing but
with rapidly development of the computer technology digital image processing overcome analogue
image processing.
There are many applications of vision based product identification system. For automotive visual
inspection system is the one of the major application. It is help to improve product quality and
production rate. In the automotive visual inspection system, we can have identified various kinds of
product, defective product, etc. Therefore, this project is the most helpful to production world.
Vision based product identification system can identify various characteristics. In our project Problem
is detecting objectives according to the colour and shape and send unique data for each object. The
project can be divided into mainly two sections. The first section is image capturing and second
section is image processing. There are large number of methods and large number of sources to solve
the above problem. For both shape and colour detection we can use blob detection method and we can
use large number of software for that. Mathematical algorithms are used to analyze the image. Mat lab,
ENVI can use analyze the image. There are so many open source tools. Such as VLFeat,
BoofCV,ILWIS& GRASS. For our project we use OpenCV. OpenCV is an open source computer vision
library. The library is written in C and C++and runs under Linux, Windows and provides interfaces for Python,
Ruby, Matlab and other languages. OpenCV library contains abundant advanced math functions, image
processing functions, and computer vision functions that span many areas in vision.
7
1.2 Problem Description
Our objective is to build a system that can detect, recognize objects according to the colour shape and
barcodes. The whole process should be done in real-time, the thing that necessities employing fast and
efficient algorithms. In addition, the actual implementation of our system should take practicality and
ease of use into account. This is because a system as ours is intended to be used in daily life and hence
needs to be both simple and efficient to use.
1.3 Software and Hardware
1.3.1 Raspberry pi minicomputer
Instead of personal computer we use raspberry pi 2 circuit. Raspberry pi is a series of credit card sized
single board computer. Raspbian-Jessie use as the operating system for compilation and execution of
image processing programs.
Raspberry pi minicomputer is the critical point of our project. It is slower than modern laptop but still
complete Linux computer and can be provided all expected abilities at a low power consumption level.
Raspberry pi is open hardware.
Fig 1. Raspberry pi minicomputer
8
Raspberry pi minicomputer has following specifications
Table 01 –Raspberry pi specifications
Product name Raspberry pi 2 model B
Product description This is the second generation of raspberry pi. It has an upgraded
Broadcom BCM2836 processor, which is powerful ARM cortex-A7
based quad-core processor that runs at 900MHz.The board also features
an increase in memory capacity to 1Gb.
Specifications
chip Broadcom BCM2836 SoC
Core architecture Quad-core ARM Cortex-A7
CPU 900 MHz
GPU Dual core VideoCore Multimedia co-processor provides open GL ES 2.0,
hardware-accelerated openVG AND 1080P30 H.264 high-profile decode
capable of 1Gpixel/s, 1.5Gtexel/s or 24 GFLOPs with texture filtering
and DMA infrastructure.
Memory 1GB LPDDR2
Operating system Boots from Micro SD card, running a version of the Linux operating
system
Dimensions 85 x 56 x 17mm
Power Micro USB socket 5V, 2A
Connectors
Ethernet 10/100 BaseT Ethernet socke
Video capture HDMI ( 1.3 & 1.4)
Audio output 3.5mm jack, HDMI
USB 4 x USB 2.0 Connector
GPIO connector 40-pin 2.54 mm (100 mil) expansion header: 2x20 strip
Providing 27 GPIO pins as well as +3.3 V, +5 V and GND supply lines
Camera connector 15-pin MIPI Camera Serial Interface (CSI-2)
Display connector Display Serial Interface (DSI) 15 way flat flex cable connector
with two data lanes and a clock lane
Memory card slot Micro SDIO
9
1.3.2 Camera System and Illumination Condition
Light became a crucial importance in machine vision. The goal of lighting in machine vision is to
obtain a robust application by:
1. Enhancing the features to be inspected.
2. Assuring high repeatability in image quality.
Illumination is the way an object is lit up and lighting is the actual lamp that generates the
illumination. Light can be ambient, such as normal indoor light or sunlight, or special light that has
been chosen with the particular vision application's needs in mind.
Lighting Types
There is a large variety of different lighting types that are available for machine vision. The types
listed here represent some of the most commonly used techniques. The most accepted type of light for
machine vision is the LED (Light-Emitting Diode), because it’s even light, long life, and low power
consumption.
1. Ring Light
2. Spot Light
3. Backlight
4. Darkfield
5. On-Axis Light
Fig 2. Light room. Fig 3. Ring light. Fig 4. Spot light.
10
We used ring light and spot light method to our project. Spot light has all the light emanating from one
direction that is different from the optical axis. For flat objects, only diffuse reflections should reach
the camera.
A ring light is mounted around the optical axis of the lens, either on the camera or somewhere in
between the camera and the object. The angle of incidence depends on the ring diameter, where the
lighting is mounted, and at what angle the LEDs are aimed.
Different objects reflect different colors. A blue object appears blue because it reflects the color blue.
Therefore, if blue light is used to illuminate a blue object, it will appear bright in a gray scale image. If
a red light is used to illuminate a blue object it will appear dark. It is thus possible to use color to an
advantage, even in gray scale imaging.
Optical Filters
An optical filter is a layer in front of the sensor or lens that absorbs certain wavelengths (colors) or
polarizations. For example, sunglasses have an optical filter to protect your eyes from hazardous UV
radiation. Similarly, we can use a filter in front of the camera to keep the light we want to see and
suppress the rest.
Two main optical filter types are used for machine vision:
1. Band-pass filter: Only transmits light of a certain color, i.e. within a certain wavelength interval. For
example, a red filter only lets red through.
2. Polarization filter: Only transmits light with a certain polarization. Light changes its polarization
when it is reflected, which allows us to filter out unwanted reflections.
Fig 5. Conveyor.
11
1.3.3 OpenCV and Python
OpenCV is an open source computer vision library. The library is written in C and C++and runs under
Linux, Windows and provides interfaces for Python, Ruby, Matlab and other languages. OpenCV
library contains abundant advanced math functions, image processing functions, and computer vision
functions that span many areas in vision
OpenCV library can be divided into several modules
1. The openCV_core module –That contains the core functionalities of the
library,inparticular,basic data structures and arithmetic functions
2. The openCV_imgproc module –That contains the main image processing functions
3. The openCV_highgui module –That contains the image and video reading and writing
functions along with some user interface functions
4. The openCV_feature2d module –That contains the feature point detectors and descriptors and
the feature point matching frame work
5. The openCV_calib3d module –That contains the camera calibratin,two view geometry
estimation and stereo function.
OpenCV provides lot of algorithms related to machine vision system. Python is a general purposes
programming language. It is very simple and readable, compared to other interface languages such as
C/C++ python is lower but very easy to code in python. Numpy is a numerical operations library. All
the openCV array structures are converted to and from numpy arrays. Therefore any operation can be
done in numpy. When considering all above factors openCV-python is an appropriate combinated tool
for fast prototyping of computer vision problems.
12
CHAPTER 2 -Solution Methodology
Considering whole project, we divide this project into six sections as follow
1. Conveyor design and fabrication
2. Install operating systemopenCV and other necessary software packages to raspberry pi
minicomputer
3. Colour detection method
4. Shape detection method
5. Data communication method between mini computer and Database
2.1 Install software to raspberry pi mini computer
Step Description
0 Updating and upgrading installed packages, followed by updating the Raspberry Pi
firmware
1 Install the required developer tools and packages.
2 Install the necessary image I/O packages. These packages allow you to load various
image file formats such as JPEG, PNG, TIFF, etc
3 Install the GTK development library. This library is used to build Graphical User
Interfaces (GUIs) and is required for the highgui library of openCV which allows you
to view images on screen
4 Install the necessary video I/O packages. These packages are used to load video files
using OpenCV
5 Install libraries that are used to optimize various operations within OpenCV
6 Install pip
7 install the Python 2.7 development tools
8 Download OpenCV and unpack it
Setup the build
Compile OpenCV
install OpenCV
13
2.2 Colour Detection Method
There are large number of methods for colour detection in openCV. In our project we use RGB to
HSV colour conversation method for colour detect.
Red, green, blue colour components can be represented as RGB colour format. Colour range is 0 to
255.The RGB colour model is an additive colour model. The main purpose of RGB colour model
sensing, representation and display of images in electronic systems.
HSV is cylindrical colour coordinate. HSV means hue, saturation, value.
Hue is represented the colour wheel.
Fig 6. RGB colour wheel
Saturated is represented the greyness of the colour.so that a Saturation value near 0 means it is dull or
grey looking whereas as a Saturation value of 0.8 might be a very strong color
Fig 7- Relationship between hue and saturation
14
Value is the brightness of the pixel, so 0.1 is black and 0.9 is white.
Fig 8- Relationship among hue saturation value
2.2.1 Steps of Colour Detection
1. Take each frame of the video
2. Convert from BGR to HSV color-space
3. Threshold the HSV image for a range of color
4. Show the mask image.
2.3 Shape Detection Method
There are so many methods to shape detect by using openCV. In our project we use contour method
for that purpose.
Contours can be explained as a curve joining all the continuous points along the boundary. Contour
method is the most useful method for shape analysis. For the better accuracy we use binary images for
that.
2.3.1 Steps for Shape Detection
1. Capture the image
2. Get an image after 30 frames
3. Delete the camera
4. Threshold the image
5. Find contours
6. Approximate contours
7. Show correct shape
15
2.4Process flow chart
16
2.5 Project Layout
Whole project layout can be described as following figure.
Fig 9 –Project layout
2.6 Limitations
There are limitations in our project.
1. Can’t be identify defective items.
2. Limited computer processing speed.
3. There are specified software versions that can compatible for raspberry pi
4. Software limitations.
5. Conveyor speed is limited.
6. Time duration for the project is not enough.
17
CHAPTER 3 -RESULTS
3.1 Colour Detection Results
Fig 10. Red colour identification
Mask 2 identify red colour
Fig 11 Bluecolour identification
Mask identify blue colour
Red Frame
Mask 3 Mask 2
Mask
Blue
Frame
Mask
Mask 2 Mask 3
18
3.2 Shape Detection Results
Fig 12 Square identification
Fig 13Triangular identification
Window
(Square)
Identify
square
Number of
contours=4
Square
Window
(Triangle) Identify
triangular
Number of
contours=3
Triangle
19
Fig 14 Circle identification
Number of
contours=16
Circle
Window
(Circle)
Identify
circle
20
CHAPTER 4 -Deliverables
Build a new system that can detect, recognize objects according to the colour shape. Sole purpose of
outputting their equivalent in string. The whole process should be done in real-time, the thing that
necessities employing fast and efficient algorithms. In addition, the actual implementation of our
system should take practicality and ease of use into account. This is because a system as ours is
intended to be used in daily life and hence needs to be both simple and efficient to use.
21
CHAPTER 5 -Conclusion
In this project, we build a new system that can detect, recognize objects according to the colour shape.
We used opencv and python as image processing software and raspberry pi circuit. We designed a
conveyor for moving objects and we designed new light system to reduce noise. This system can used
for further development in vision based projects.
22
References
HSV colour conversion
Available online http://shervinemami.info/colorConversion.html
Accessed(February 29,2016)
Intro robotics
Available online http://www.intorobotics.com/how-to-detect-and-track-object-with-opencv/
Accessed(February 29,2016)
C. Richard Johnson, Jr., Ella Hendriks, Igor J. Berezhnoy, Eugene Brevdo, Shannon M. Hughes, Ingrid
Daubechies, Jia Li, Eric Postma, and James Z. Wang
Image Processing for Artist Identification, IEEE signal processing magazine july 2008, 1053-
5888/08/$25.00©2008IEEE
Theo Gevers*, Arnold W.M. Smeulders
Color-based object recognition, ISIS, Faculty of WINS, University of Amsterdam, Kruislaan 403,
1098 SJ, Amsterdam, The Netherlands Received 22 December 1997; received for publication 4
February 1998
Y.Ramadevi, T.Sridevi, B.Poornima, B.Kalyani
SEGMENTATION AND OBJECT RECOGNITION USING EDGE DETECTION TECHNIQUES
International Journal of Computer Science & Information Technology (IJCSIT), Vol 2, No 6,
December 2010
23
Appendix
Colour Detection Code
import cv2
import numpyas np
cap= cv2.VideoCapture(0)
while True :
_,frame =cap.read()
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
lower_blue = np.array([0 , 100, 100])
upper_blue = np.array([ 130,255, 255])
lower_red =np.array([100,150,0])
upper_red =np.array([255,255,150])
lower_green =np.array([100,0,100])
upper_green=np.array([255,150,255])
mask = cv2.inRange(hsv,lower_blue, upper_blue)
mask2= cv2.inRange(hsv,lower_red,upper_red)
mask3= cv2.inRange(hsv,lower_green,upper_green)
output = cv2.bitwise_and(frame,frame,mask = mask)
cv2.imshow('frame',frame)
cv2.imshow('mask',mask)
cv2.imshow('mask2',mask2)
cv2.imshow('mask3',mask3)
k = cv2.waitKey(5) &0xFF
if k== 27:
break
cv2.destroyAllWindows()
cap.release()
24
Shape Detection Code
import numpyas np
import cv2
#import cv2.cv as cv
cap = cv2.VideoCapture(0)
#Capture the image
defget_image():
retval, img = cap.read()
return img
#Get an image after 30 frames
for iin xrange(30):
img = get_image()
print("Takingggg....")
#Save the image
file = "D:/tttt/test.png"
cv2.imwrite(file, img)
#Delete the camera
del(cap)
final_img = cv2.imread("D:/tttt/test.png")
cv2.imshow("final", final_img)
gray = cv2.imread("D:/tttt/test.png",0)
#identifying the object
ret,thresh=cv2.threshold(gray,127,255,1)
img,contours,h=cv2.findContours(thresh,1,2)
for cntin contours:
apporx=cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
print len(apporx)
if len(apporx)==3:
print"Triangle"
cv2.drawContours(img,[cnt],0,255,-1)
cv2.imshow('Triangle',img)
eliflen(apporx)==4:
print"Square"
cv2.drawContours(img,[cnt],0,(0,255,0),-1)
cv2.imshow('Square',img)
eliflen(apporx)>15:
print"Circle"
cv2.drawContours(img,[cnt],0,(0,0,255),-1)
cv2.imshow('Circle',img)
cv2.waitKey(0)
cv2.destroyAllWindows()