IN DEGREE PROJECT TECHNOLOGY,FIRST CYCLE, 15 CREDITS
, STOCKHOLM SWEDEN 2016
Color Sorting RobotSorting algorithm by color identification
TOMAS FREDRIKSSON
SARA STRÖM
KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF INDUSTRIAL ENGINEERING AND MANAGEMENT
Color Sorting Robot
Sorting Algorithm By Color Identification
TOMAS FREDRIKSSONSARA STRÖM
Bacherlor’s Thesis in Mechatronics
Supervisor: Nihad Subasic Examiner: Martin Edin Grimheden Approved: 2016-06-07
TRITA MMKB 2016-16 MDAB077
AbstractEfficiency and automatization can be improved in several ways. The focusin this report has been working with color identification and creating a smartrobot. A simple robotic arm is used to apply the color sorting to a physicalsystem. This model evaluates how well a robotic arm can sort different objectsusing a predefined color identification algorithm. A demonstrator was builtto perform tests for sorting speed and color identification. The robotic armcan sort a predefined shaped and sized object in 15,36 seconds. The coloridentification is sensitive to external factors and does not necessarily returnthe right RGB-value depending on lightning and brightness. The R-value oftenhas the largest error. To further improve the color sorting robot, another coloridentification method could be tested, other motor types should be incorporatedand a more precise sensor should be implemented.
iii
SammanfattningFärgsorterande Robot
Förbättringar inom effektivisering och automatisering kan göras på många oli-ka sätt, och i den här rapporten har en metod med färgidentifiering arbetatsfram för att skapa en smart robot. En enkel robotarm används för att applice-ra den fysiska tillämpningen av systemet samtidigt som själva färgsorteringenutgörs av en minidator. Denna modell utvärderar hur en robotarm med hjälpav en färgidentifieringsalgoritm kan sortera olika objekt. Resultatet visar attrobotarmen kan sortera det bestämda objektet på 15,36 sekunder. Färgiden-tifieringen är dock känslig mot externa faktorer, såsom ljus och exempelvisblanka ytor. Programmet ger nödvändigtvis inte ’rätt’ RGB-värde, beroendepå dessa externa faktorer. Det är ofta R-värdet som ger det största felet. Föratt förbättra färgsorteringsroboten, skulle en annan färgsortertingsmetod kun-na testas, motortypen kan bytas ut, samt en mer precis sensor skulle kunnaimplementeras.
v
Preface
We would like to thank our supervisor Nihad Subasic for support and feedbackand Staffan Qvarnström for all the help with the components and manufacturing.Finally, many thanks to all the student assistants in the lab who have been therethroughout the entire semester.
Tomas Fredriksson & Sara StrömKungliga Tekniska Högskolan, May, 2016
vii
Contents
Abstract iii
Sammanfattning v
Preface vii
Contents ix
Nomenclature xi
1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.4 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.5 Related projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Theory 32.1 Arduino . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Raspberry Pi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 Camera Pi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.4 Adafruit Motor Shield . . . . . . . . . . . . . . . . . . . . . . . . . . 52.5 Robotic Arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.6 OpenCV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.7 SimpleCV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 Demonstrator 73.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2.1 Robotic arm . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.2.2 Camera Pi . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2.3 Other assemblies . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3.1 Robotic arm connected to Arduino . . . . . . . . . . . . . . . 10
ix
3.3.2 Robotic claw . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3.3 Camera Pi . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3.4 Color identification . . . . . . . . . . . . . . . . . . . . . . . . 113.3.5 Connect Raspberry Pi and Arduino . . . . . . . . . . . . . . 11
3.4 Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.4.1 Robotic arm . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.4.2 Robotic claw . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.4.3 Connect Raspberry Pi and Arduino . . . . . . . . . . . . . . 13
4 Results 154.1 Robot Arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.2 Color Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5 Discussion and conclusions 195.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.1.1 Different motors . . . . . . . . . . . . . . . . . . . . . . . . . 195.1.2 Raspberry Pi . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.1.3 Camera Pi . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.1.4 Color Identification . . . . . . . . . . . . . . . . . . . . . . . . 205.1.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6 Recommendations and future work 236.1 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Bibliography 25
Appendices
A Breadboard layout for H-bridge 1
B Specifications of Raspberry Pi models 1
C All colors tested with Camera Pi 1
D Arduino Code for the Robot Arm 1
x
Nomenclature
AbbreviationsAbbreviation DescriptionAPI Application Programming InterfaceCPU Central Processing UnitCSI Camera Serial InterfaceDC Direct CurrentGPIO General-Purpose Input/OutputGPU Graphics Processing UnitHSB Hue, Saturation, BrightnessIDE Integrated Development EnvironmentI2C Inter-Integrated CircuitPWM Pulse Width ModulationRGB Red, Green, BlueRPi Raspberry PiSEK Swedish KronaSPI Serial Peripheral Interface BusSRAM Static Random-Access MemoryUART Universal Asynchronous Receiver/Transmitter
xi
Chapter 1
Introduction
This chapter introduces the subject and defines the purpose, scope and method.
1.1 BackgroundSituations where humans work repeatedly and contains the same process every cyclehave an ability to give uneven results. In the future a wide range of that kind ofapplications can be improved by using a robot. A robot’s work gives the sameresult each time as opposed to the human. When the robot is programmed it willdo the exact same operations every time. Further advantages of using a robotis the decreased time consumption. The quality and capability to repeat precisemovements makes it easy to increase the speed of the process. To invest in a robotis a one-time charge but in the long term the investment will be worth it due to theconsistent quality and labor costs. Robots can work both day and night, comparedto humans who need breaks and sleep. One kind of task which a robot would beable to perform is sorting objects into categories for instance by color, shape orsize. This is, however, not always a simple combination. Therefore, in this thesis,a design of a solution is presented and tested.
1.2 PurposeThe purpose of this project is to analyze how a robot with a camera module canrespond to objects of different color, then grip and sort them depending on thecolor identification. To achieve this, we have focused this thesis on the followingquestions:
• How fast can a system identify a specific object, lift it using a robotic arm andsort it using color identification?
• How similar in color can two objects be for the robot to see the difference andsort it accordingly?
1
CHAPTER 1. INTRODUCTION
1.3 ScopeIn order to complete this project within the time limit and with a budget of 1 000SEK (Swedish Krona) some boundaries had to be set. The object picked up bythe robot has specified dimensions, which can be seen in Tab 1.1. The camera issupposed to be able to see the difference between two very different colors (e.g. redand blue). The robotic arm has no speed requirement or limits.
Table 1.1. Limitations of graspable objects
Parameters ValuesWeight <100 gSize 30x30x30 mm (Length x Depth x Height)Shape Cubic
1.4 MethodThe Arduino and Raspberry Pi (RPi) are combined to fulfill the purpose of thisproject. The logic for interacting with robot hardware is on the Arduino and thecalculations are done by the RPi, which sends control signals to the Arduino. TheArduino is a micro-controller, which means the clock speed and memory limits itsability to process images. A mini-computer is required for the processing. For thisproject a RPi is selected because it is easy to implement a camera module. The RPiis utilised to manage the image processing and the Arduino operates the motors. Acamera programmed to identify colors is placed on the robot arm and connected tothe RPi. Depending on what color the object has the RPi will send different signalsto the Arduino for movement of the robot arm.
1.5 Related projectsThis project is inspired by combining two other projects. One existing project wherean Arduino is connected to a robot arm. The robot arm is driven by five DC-motorsand is controlled with information from potentiometers. A motor shield is used tocontrol four of the motors. This project does not include control of the fifth motor,the claw, and the arm is programmed to follow a specific path. [Instructables, 2016]Another project introduces image processing with the Raspberry Pi and Python isused. This project presents how to capture an image, store it and show the imageon the screen. This project includes a regular webcamera and does not includeidentification of shape, size or color. [raspberrypi, 2016]
2
Chapter 2
Theory
This chapter presents the research of all components and software used for designingthe robot arm.
2.1 ArduinoArduino board is a micro-controller, which are able to read different inputs andturn them into outputs. Relevant specifications are listed in Tab 2.1. Instructionscan be sent to the board using the Arduino programming language. [Arduino, 2016]The Arduino Uno is used in this project, see Fig 2.1.
Table 2.1. Specifications of Arduino UNO
Description ValuesOperating Voltage 5 VDigital I/O Pins 14 (of which 6 provide PWM output)Analog Input Pins 6SRAM 1 KBClock Speed 16 MHz
Figure 2.1. Arduino Uno
3
CHAPTER 2. THEORY
2.2 Raspberry PiThe RPi is a credit-card sized computer, which is a powerful tool for its size, seeFig 2.2. In this project, a Raspberry Pi A+ is used, relevant specifications are listedin Tab 2.2 [RaspberryPi, 2016]. RPi uses a computer operating system (OS) calledRaspbian, which is a Unix-like, open source OS. Our RPi in this project is usingthe latest release named "Raspbian Jessie". [RaspbianProject, 2016]
Table 2.2. Specifications of Raspberry Pi Model A+
Description ValuesCPU 700 MHz single-core ARM1176JZF-SGPU Broadcom Videocore IV @ 250 MHzMemory 256 MB (shared with GPU)
Figure 2.2. Raspberry Pi A+
2.3 Camera PiThe camera module to RPi is a five megapixel fixed-focus camera that supports high-defintion video as well as pictures, which translate into a resolution of 1920x1080,see Fig 2.3. The Camera is used in this project because of it’s relationship with theRPi; the module can be run through the OS on the RPi, which makes it simple toinstall and use. [RaspberryPiFoundation, 2016]
Figure 2.3. Camera Module for Raspberry Pi
4
2.4. ADAFRUIT MOTOR SHIELD
2.4 Adafruit Motor Shield
A motor shield is a driver module for motors that allows controlling of the workingspeed and direction of the motor [ArduinoMotorShield, 2016]. The motor shieldused in this project is called Adafruit Motor Shield v2.3 and can control up to fourDC-motors, see Fig. 2.4. [Adafruit, 2016]
Figure 2.4. Adafruit Motor Shield v2.3
2.5 Robotic Arm
This robotic arm, see Fig 2.5, is a manually controlled robot with five DC-motors, agripping claw and four joints. Rotation is enabled in several directions. [OWIRobots,2016]
Figure 2.5. Robotic Arm
5
CHAPTER 2. THEORY
2.6 OpenCVOpenCV stands for Open Source Computer Vision and is a software library forreal-time computer vision. It is originally written in C/C++, but now featuresboth Python and Java interfaces, in addition to C/C++. It also supports variousplatforms including Windows, Linux, Mac OS, iOS and Android. OpenCV’s appli-cation areas are many, some examples are object identification and color filtering.[Itseez, 2016] In this project the library is imported to Python 2.7.
2.7 SimpleCVSimpleCV is computer vision made simple. It is an open source framework, whichis built on and accesses the OpenCV libraries. It is written in Python and runs onWindows, Mac OS and Ubuntu Linux. The applications are the same as OpenCV’sbut is simplified into easy to use Application Programming Interface, API [Sight-Machine, 2016].
6
Chapter 3
Demonstrator
This chapter describes the design and the development process of the robot arm andit’s sorting functions. It addresses the hardware, software and electronics used inthis project.
3.1 Problem Formulation
To properly sort the objects, the demonstrator is used to solve several problems.
• The robot arm is equipped with five DC-motors, sensors needs to be addedto control movement.
• The Adafruit Motor Shield v2.3 can only control four DC-motors.
• The fifth DC-motor controls the claw, a H-bridge needs to be added to controlmovement.
• The Arduino UNO is too slow for image processing.
• The Raspberry Pi needs to communicate with the Arduino.
• The pins of the Raspberry Pi holds 3.3 volts while the Arduino’s holds 5 volts.
• The Camera Pi captures high-definition pictures, these pictures needs to beprocessed in high-speed.
3.2 Hardware
The demonstrator consists of a robotic arm with four different axis of rotation, aclaw, the camera module, the Raspberry Pi, Arduino and housing for the micro-controller and mini-computer. The robot can be seen as a whole in Fig. 3.1
7
CHAPTER 3. DEMONSTRATOR
Figure 3.1. The construction seen from the side
3.2.1 Robotic armThe robotic arm is equipped with four DC-motors which control their specific rota-tional axis. The arm is also equipped with a gripping claw driven by a DC-motor.This robotic arm was bought in small parts and assembled with instructions fromblueprints provided with the parts. The blueprints can be found in the list of ref-erences. [Elfa, 2016] The consequence of the robot arm being driven by DC-motorsis that it is not possible to know where in the room the arm is positioned. Thisproblem was solved using four potentiometers, which function as angular sensors.Depending on how the shaft is rotated that value can be translated to what max-imum and minimum angle that one specific axis is allowed to rotate. Limited bythese sensors, the arm will not pass the end points and break. The potentiome-ters were glued onto the robotic arm in such way that the potentiometer’s shaftis steady at all times during movement. Instead the base of the potentiometer isrotated when the motors rotate the axis. The potentiometer solution can be seenin Fig. 3.2
Figure 3.2. Close-up on potentiometer glued on the robot
8
3.2. HARDWARE
3.2.2 Camera Pi
The camera module is connected to the RPi using the Camera Serial Interface(CSI)-port and is mounted on the robotic arm with a self-made 3D-printed case,see Fig. 3.3. The camera is stationary and have the same view during the rotation,with the consequence that the object has to be placed in the same position everycycle.
Figure 3.3. Close-up on how Camera Pi is attached
3.2.3 Other assemblies
The Arduino, Adafruit Motor Shield and RPi are mounted on a 3D-printed casingwith screws and nuts. The final assembly can be seen in Fig. 3.4
Figure 3.4. Housing for Arduino, Raspberry Pi, toggle switch and H-bridge
9
CHAPTER 3. DEMONSTRATOR
3.3 Software
The RPi analyzes the color of the object and depending on which color the camerasees, it sends either a 0 or a 1 to the Arduino’s serial port. The Arduino uses thesevalues to determine which DC-motors should be powered, in order to run in differentdirections. The software algorithm is explained in Fig. 3.5.
Figure 3.5. Flowchart for the demonstrator software algorithm
3.3.1 Robotic arm connected to Arduino
The software for controlling the robotic arm is written in the Arduino IDE, whichcan be seen in Appendix D. The program contains different states depending onwhere in the process the arm is. After every complete cycle the arm return to astandby position. The states are programmed so that they are controlled based onthe values from the potentiometers. The movement of the arm is a predefined pathwhich is hard coded. These values are determined by manual control of the motors,meaning when the arm is in a position that is desirable, that value is programmedon the Arduino. This is done for all motors, in all different states. Because of this,the object needs to be placed in a specific position to be picked up properly.
10
3.3. SOFTWARE
3.3.2 Robotic claw
The claw is controlled with PWM-signals on digital pin 3 on the Arduino. The clawis programmed to open and close in a specific timing during the arm’s movement.This means that there are no actual sensors that tell us if the claw is actually holdingsomething. The PWM-signal is set below it’s middle value when we want to openthe claw, and vice versa if the claw should close.
3.3.3 Camera Pi
The software for the camera is written in Python by using the PiCamera [PiCamera,2016] library. The camera takes a still picture every second which is analyzed bythe color identification part. The resolution is lowered to 640x480 pixels for the RPito work faster.
3.3.4 Color identification
The color identification process uses an open-source library named OpenCV [Itseez,2016], which is accessed with SimpleCV [SightMachine, 2016]. Both are installedand compiled on the RPi, with Python 2.7. The software uses the picture taken bythe Camera Pi to analyze the center pixel of the image. The RGB (Red, Green,Blue) color channel values of that pixel is analyzed and depending on the red value,either a "0" or a "1" is sent from the Serial Port of the RPi. This means that thesorted object needs to be in the center of the image. The RGB value analyzed fromthe picture is dependant on how much light the camera is receiving. A consequenceof this can be that the robot needs to calibrated when moved to a different location,where the external lightning is different.
3.3.5 Connect Raspberry Pi and Arduino
Arduino and RPi connects with the General-Purpose Input/Output (GPIO) pins onthe Raspberry Pi and the Serial pins on the Arduino. Two different methods wereconsidered to connect the two devices, either using the Serial Peripheral Interface(SPI), or the Inter-Integrated Circuit (I2C) interface. The decision to use SPI wasmade because this method is easy to setup and operates faster. [Harrison, 2016]The communication is only one way because of the difference in voltage betweenthe two devices. The RPi’s serial pin holds 3.3 volts while the Arduino’s at 5 volts.Only the RPi will send data to the Arduino, because if the Arduino would senddata to the RPi, it could be damaged. The serial port called "ttyAMA0" is used tosend data from the color identification program, meaning all other communicationon that specific port has to be disabled. By default, the RPi uses this port tocommunicate general information that could be useful in other applications. It isnot in this case, which means it needs to be disabled for the method to work. This isdone in the "/root/" and "/boot/" directories on the RPi. All associations with this
11
CHAPTER 3. DEMONSTRATOR
port has been disabled or removed in these directories. [Liang, 2016] This allowsthe RPi only to communicate on this port from the color identification program.
3.4 Electronics
This section describes how all components, including DC-motors, potentiometers,H-bridge, the micro-controller and mini-computer are connected.
3.4.1 Robotic arm
The potentiometers are soldered onto the analog input ports on the Arduino Unoboard, the four DC-motors are connected to the Adafruit Motor Shield motor inputs,see Fig. 3.6. The motor shield also needs an external power source in order to runthe motors, a 5 volt power source is chosen.
Figure 3.6. Arduino block diagram and wiring
3.4.2 Robotic claw
The PWM signals are sent on the digital pin 3 on the Arduino, this port is con-nected to a H-Bridge that we built ourselves from a schematic distributed duringthe introduction of this project, see , see Appendix A [Grimheden, 2016]. The out-puts from the H-Bridge are soldered onto a toggle switch in order to turn the powerprovided for the claw off. This in order to re-upload the program on the Arduino,the PWM-signal is reset which makes the DC-motor run uncontrollably and couldpossibly destroy the claw. The schematic can be found in Fig 3.7
12
3.4. ELECTRONICS
Figure 3.7. Toggle switch block diagram and wiring
3.4.3 Connect Raspberry Pi and ArduinoSince the RPi’s serial pin only holds 3.3 volts and the Arduino’s at 5 volts, a voltagedivider must be used, which is basically two resistors. [Liang, 2016] The schematiccan be found in Fig 3.8.
Figure 3.8. Arduino and Raspberry Pi serial connection schematic
13
Chapter 4
Results
This chapter presents the results gathered from testing the robotic arm.
4.1 Robot ArmAn experiment were made where one movement cycle of the arm was observed.One cycle includes movement from standby, picking up the object, transporting theobject, dropping the object and back to standby. The measurements obtained fromthe experiments can be seen in Tab.4.1
Table 4.1. Time required for each cycle
Cycle no. Time (s)1 15,302 15,193 15,364 15,385 15,316 15,657 15,568 15,13Avg. 15,36
4.2 Color IdentificationSeveral tests were made with the Camera Pi, 16 different colors which was randomlyselected was photographed by the camera and ran through the color identificationprogram to analyze their RGB values. All tested colors can be found in AppendixC. The RGB-values from the output from the camera and color identifcation, aswell as input provided from the testsheet, can be found in Fig. 4.1
15
CHAPTER 4. RESULTS
Figure 4.1. Difference in RGB values captured by Camera Pi.
All results from the tests is gathered in Tab. 4.2
16
4.2. COLOR IDENTIFICATION
Table 4.2. All results from RGB tests
RGB (In) RGB (Out)255 255 255 180 148 1330 0 0 55 59 58
255 0 0 195 88 1060 255 0 215 214 00 0 255 166 96 71
255 255 0 222 226 1150 255 255 211 207 195
255 0 255 239 139 193125 125 125 179 146 137200 50 50 204 104 128)200 120 30 203 152 14940 100 70 155 145 58200 35 125 216 118 165150 30 30 180 86 111255 175 255 231 178 18450 50 100 148 89 89
17
Chapter 5
Discussion and conclusions
This chapter discusses and summarizes the results presented in the previous chapter.The summary is based on an analysis of results and aims to answer the researchquestions.
5.1 Discussion
The robot can pick up an object and sort it according to color, which makes itpossible for us to evaluate our research questions. However, we encountered manyproblems along the way and our solution can be improved in many ways. For morerecommendations and future work see chapter 6.
5.1.1 Different motors
The main factor that could improve the project is to change the type of motorused by the robot arm. Using DC-motors makes it difficult to control the robot.The potentiometers makes it possible to control it in a limited fashion, but not goodenough compared to other solutions. Servo-motors on the other hand can control thepositions of the motors. With a feedback system that a servo-motor enables it getseasier to pick up objects independent of its exact location. Using potentiometerslimits us to only picking up objects on a predefined position. Servo-motors wouldalso prevent the motors from running uncontrollably and possibly break, somethingwe had to spend a lot of time on during this project. However, this solves howmuch of the predefined path that needs to be hard coded, is not something wehave evaluated. Another problem we encountered with the motors was the high-pitch sound that the motors generated while running on PWM-signals. This wassolved in the Arduino-code by changing what frequency the PWM-signal operatedon. Other disturbing sounds that we could not fix was the creaking sounds from theplastic gearboxes that was included with the DC-motors. This is something thatcould be fixed if another motor type were chosen. Then another gearbox can beused, with higher strength and stability.
19
CHAPTER 5. DISCUSSION AND CONCLUSIONS
5.1.2 Raspberry Pi
We decided to use the Raspberry Pi Model A+ for this project, a choice motivated bythe low price and the fact that the hardware could do all image processing needed forthe project. A consequence of this choice was the additional time spent on installingthe required libraries and software on the RPi. We had to buy a USB hub and aEthernet-to-USB-adapter, which resulted in a more expensive device than if we forexample would have bought the latest RPi model. One advantage with using theolder model was the power usage, weight and size. This choice of mini-computercould have been evaluated more, to prevent unnecessary time consumption. Forreference, see all Raspberry Pi models specifications in Appendix B.
5.1.3 Camera Pi
The Camera Pi has great potential, which is not fully used is not used for thisproject. The resolution is lowered, but the quality is still satisfactory. The abilityto see different shapes is possible with the camera but this is not included in thisproject. This is something one could continue to work with in another project. Theconnection between camera module and mini-computer is very simple and was notsomething we had to spend extra work on. It is also very simple to mount themodule because of the screw holes on the camera.
5.1.4 Color Identification
The color identification is very sensitive to external lightning, because it is purelybased on the R-value in the RGB-value provided from the image. The camera sendsa picture to the RPi which is not an ideal solution due to the fact that the R-valuevaries depending on e.g. the shutter speed. This means that the camera might notsee the objects "real" R-value, and sort it incorrectly. One other solution could beusing Hue, Saturation, Brightness (HSB) color spacing method instead of basing thesorting on RGB-values. This requires more research and is mentioned in chapter6. Because our sorting algorithm only depends on the R-value the color recognizedas red could be magenta or yellow, depending on the G-values and B-values of theimage, which is another argument for changing color identification method.
5.1.5 Results
The final results of the project were in line with what we expected, the demonstratorcould answer our questions in a satisfactory fashion. With the hardware providedwe could not optimize the sorting path further, but the time of one sorting cycleis not fast enough for this project to be relevant for the market. Therefore, wewould recommend to continue working on the speed of the motors, and toughen upthe robot so that it can perform more precise sorting, with faster cycle speed. Allcolors were randomly chosen in the color identification experiment. One conclusionis that the camera is very sensitive in its red channels. For example, when the
20
5.2. CONCLUSIONS
input was 0 the system responded with an R-value of 215. To calibrate this camerais not a difficult task but it’s something we haven’t spent time on. Dependingon lightning, weather, and surroundings we had to re-calibrate the input channelseach time to attain decent results. Even though this is a tiring task it is stillpossible with our chosen method. We believe that if the robot would be installed ina specific environment, this problem could be solved using different color channelmodifications and change in the cameras brightness and contrast. This is somethingthat is possible with the Camera Pi and RPi. For us to be satisfied with the resultsfor our last research question we would like to do more tests and work with anothercolor identification method, this is something we are discussing further in the nextchapter.
5.2 ConclusionsMany time-consuming tasks throughout the project could have been avoided if wewould have invested in proper hardware from the beginning, which would also givebetter results and a more precise sorting. This would, however, increase the costfor the project which is a restriction we cannot overlook. With that in mind we arehappy with the results and satisfied with our demonstrator and the way it couldanswer our research questions.
21
Chapter 6
Recommendations and future work
This chapter provides recommendations for more detailed solutions and future work.
6.1 RecommendationsFirst and foremost our recommendations for attaining better results is to use adifferent RPi model. The time spent on building workarounds for the old modelwe used could have been spent on improving the several parts of the project. Asmentioned in the discussion, changing the motor type would be beneficial for moreprecise movement of the motors. Servo-motors is something we would recommend,because they have a controllable built-in parameter and often includes a gearbox.Another recommendation is to look over the color identification method using theRGB-values. The lightning caused problems for the camera to identify the givenRGB-value. A better method that could be implemented on the project is HSB-color space. HSB categorizes the color in three different parameters in comparisonwith RGB that only categorizes the color channels. The advantage of using thismodel is that one of the parameters is brightness, which could solve some of theproblems in this project. [Tech, 2016]
6.2 Future workOne thing that could be improved even more is the predefined paths that the roboticarm moves in. This can be optimized using linear interpolation, something that isnot done in its current state. The robot arm is adjusted manually, so it may bepossible that it is not moving in the fastest path. To sort objects based on bothcolor and shape is an easy next step for this project. The camera is capable toidentify several colors and shapes at the same time. That makes it possible toidentify object who contains more than one color but still have different shapes.Using the SimpleCV application with OpenCV library we can modify the coloridentification program so that the image is processed into black and white withthe Binarize-function. The next step is to use a method based on Hugh Transform
23
CHAPTER 6. RECOMMENDATIONS AND FUTURE WORK
to define the contour of the object and then check if it is a specific shape. Thesefunctions already exits in the SimpleCV library. [OpenElectronics, 2016] Throughour research we have encountered several examples of robotic arms that is usedin the industry today. The latest robots often has six possible axial movements[Andersson, 2016], enabling faster and more precise sorting. This is something onecould look at in modifying our project idea. Another idea is to make the robotmobile, the range of applications then broadens. The camera and identificationpart used in this project could be modified so that it also could detect where theobject is placed and then pick it up accordingly. A more integrated design couldalso be developed, making the robot more desirable on the market.
24
Bibliography
[Adafruit, 2016] Adafruit (2016). Overview | adafruit motor shieldv2 for arduino. Available from: https://learn.adafruit.com/adafruit-motor-shield-v2-for-arduino/overview [cited 2016-04-15].
[Andersson, 2016] Andersson, J.-E. (2016). Slutrapport robotteknik isagverk.pdf. Available from: http://www.ltu.se/cms_fs/1.82945!/file/SlutrapportRobotteknikisagverk.pdf [cited 2016-05-08].
[Arduino, 2016] Arduino (2016). What is arduino? Available from: https://www.arduino.cc/en/Guide/Introduction [cited 2016-04-12].
[ArduinoMotorShield, 2016] ArduinoMotorShield (2016). Arduino - ar-duino motor shield. Available from: https://www.arduino.cc/en/Main/ArduinoMotorShieldR3 [cited 2016-04-12].
[Elfa, 2016] Elfa (2016). owi 535 manual. Available from: https://www.elfa.se/Web/Downloads/_d/ts/c-9895_1_eng_dts.pdf?mime=application.pdf [cited2016-05-02].
[Grimheden, 2016] Grimheden, M. E. (2016). Fim 2016 - experiment 3. Avail-able from: https://www.kth.se/social/files/56a89c13f2765417dfd6fe1c/Experiment3.pdf [cited 2016-04-28].
[Harrison, 2016] Harrison, M. (2016). Tradeoffs when considering spi or i2c.Available from: http://electronics.stackexchange.com/questions/29037/tradeoffs-when-considering-spi-or-i2c [cited 2016-05-02].
[Instructables, 2016] Instructables (2016). Robotarm controlled witharduino. Available from: http://www.instructables.com/id/Intro-and-what-youll-need/step3/Load-the-Arduino-code/ [cited 2016-05-02].
[Itseez, 2016] Itseez (2016). Opencv. Available from: http://opencv.org/ [cited2016-04-15].
[Liang, 2016] Liang, O. (2016). Raspberry pi and arduino connected overserial gpio - oscar liang. Available from: https://oscarliang.com/raspberry-pi-and-arduino-connected-serial-gpio/ [cited 2016-04-29].
25
BIBLIOGRAPHY
[OpenElectronics, 2016] OpenElectronics (2016). Computer vision with raspberrypi and the camera pi module. Available from: http://www.open-electronics.org/computer-vision-with-raspberry-pi-and-the-camera-pi-module/[cited 2016-04-27].
[OWIRobots, 2016] OWIRobots (2016). Owi-535 robotic arm edge.Available from: http://www.owirobots.com/store/catalog/robotic-arm-and-accessories/owi-535-robotic-arm-edge-kit-110.html[cited 2016-04-12].
[PiCamera, 2016] PiCamera (2016). Python picamera - raspberry pi documenta-tion. Available from: https://www.raspberrypi.org/documentation/usage/camera/python/README.md [cited 2016-05-08].
[raspberrypi, 2016] raspberrypi (2016). Basic image processing. Avail-able from: https://www.cl.cam.ac.uk/projects/raspberrypi/tutorials/robot/image_processing/ [cited 2016-05-02].
[RaspberryPi, 2016] RaspberryPi (2016). Raspberry pi. Available from: https://www.raspberrypi.org/help/what-is-a-raspberry-pi/ [cited 2016-04-12].
[RaspberryPiFoundation, 2016] RaspberryPiFoundation (2016). Camera module- raspberry pi. Available from: https://www.raspberrypi.org/products/camera-module/ [cited 2016-04-12].
[RaspbianProject, 2016] RaspbianProject (2016). Frontpage - raspbian. Availablefrom: https://www.raspbian.org/FrontPage [cited 2016-04-28].
[SightMachine, 2016] SightMachine (2016). Simplecv. Available from: http://simplecv.org/ [cited 2016-04-20].
[Tech, 2016] Tech, N. M. (2016). Introduction to color theory. Avail-able from: http://infohost.nmt.edu/tcc/help/pubs/colortheory/web/hsv.html [cited 2016-05-04].
26
Appendix A
Breadboard layout for H-bridge
1
Appendix B
Specifications of Raspberry Pi models
1
APPENDIX B. SPECIFICATIONS OF RASPBERRY PI MODELS
2
Appendix C
All colors tested with Camera Pi
(255,255,255) (0,0,0) (255,0,0) (0,255,0)
(0,0,255) (255,255,0) (0,255,255) (255,0,255)
(125,125,125) (200,50,50) (200,120,30) (40,100,70)
(200,35,125) (150,30,30) (255,175,255) (50,50,100)
1
Appendix D
Arduino Code for the Robot Arm
1
APPENDIX D. ARDUINO CODE FOR THE ROBOT ARM
2
3
APPENDIX D. ARDUINO CODE FOR THE ROBOT ARM
4
5
www.kth.se
