World Applied Sciences Journal 18 (10): 1435-1442, 2012ISSN 1818-4952© IDOSI Publications, 2012DOI: 10.5829/idosi.wasj.2012.18.10.1717

Corresponding Author: Ali Adelkhani, Science and Research Branch, Islamic Azad University, Tehran, IranMob: +989124492068.

1435

Optimization of Lighting Conditions and Camera Height for Citrus Image Processing

Ali Adelkhani, Babak Beheshti, Saeid Minaei and Payam Javadikia1 1 2 3

Department of Agricultural Machinery engineering, 1

Science and Research Branch, Islamic Azad University, Tehran, IranDepartment of Agricultural Machinery Engineering, Tarbiat Modares University, Tehran, Iran2

Department of Agricultural Machinery Engineering, 3

Razi University of Kermanshah, Kermanshah, Iran

Abstract: Machine vision and image processing are methods which have various applications in agriculture,including volume determination, grading and diagnosing surface damages of products. If lighting or cameraheight is not suitable, processing by machine vision will not have acceptable performance. Thus, a project wasundertaken to optimize lighting type and intensity along with camera height in an image acquisition chamberwhich was tested using oranges. To do so, a suitable algorithm was developed and a chamber equipped withadjustable LED, Tungsten and fluorescent lighting was constructed. Using this setup, major and minordiameters as well as, the number of pixels all over the surface were determined from the image and comparedwith actual measurements. The two-row LED lighting combined with white light of 50.33 lux and camera heightof 10cm was found to be the best condition providing the closest result to manual measurements.

Key words: Camera height % Citrus % Image processing % Light intensity % Light type % Machine vision

INTRODUCTION technique is very important. The tests of determining

The annual production of citrus in Iran is 3.5 million taste and sugar content can be done on different kind oftons, making Iran the sixth rank producer in the world. citrus such as orange.This shows the importance of grading, quality and The most significant benefits of machine vision arequantity of citrus production. Production of a fruit which the speed of descriptive data from product, the reductionis palatable and attractive to the consumer is the main of working volume of user, economically and comfort,priority of each producer. this requires selection and no destructiveness and use of control system. Theresorting of substandard fruits. Thus, development of are some demerits, too, such as sensitivity of lightingsuitable grading systems is important for the industry. system which makes it difficult to obtain accurate imageVarious methods may be used for nondestructive tests in any background. In addition, operation in low-lightsuch as RAMAN spectroscopy, NIR and NMR, sound condition and darkness is difficult.diffusion and ultrasonic techniques. Machine vision is Arefi et al. [2] developed a new segmentationone of the first methods to assess agricultural products algorithm for guidance of a robot arm to pick the ripeand its spread has been concomitant with hardware tomato using a machine vision system. They used adevelopment. In general, the wide spread use of this vision system to acquire images from tomato plant.technique is in grading systems for agricultural products, The recognition algorithm had to be adaptive to thecolor diagnosis, surface deficiency and texture which lighting conditions of greenhouse. According to thecan be best obtained in accurate technical conditions results, the total accuracy of proposed algorithm wassuch as camera height, light intensity and type of light. 96.36%. Leemans et al. [15] surveyed grading fruits inConsequently optimizing the condition to use this real time, based on external quality by machine vision.

various conditions of volume, weight, sphericity, PH,

World Appl. Sci. J., 18 (10): 1435-1442, 2012

1436

They obtained 73% system grading efficiency for and other materials such as zinc, copper, nickel, sodiumjonagold apple. Blasco et al. [5] built an automaticmachine vision system to grade pomegranate based onR/G ratio and LDA. The results of two methods weresatisfying.

Siristhitkul et al. [21] applied a system to gradeorange by using color, the results of which showed aworking efficiency of 98%. Omid et al. [17] designed asystem to measure mass and volume of citrus. The resultsshowed that the size of citrus did not had any effect onaccuracy of operation and mass and volume wereindependent parameters. Blasco et al. [4] presented amachine vision system based on diagnosis andclassification of outer deficiencies of citrus, assessing2000 sample of orange and tangerine and got theefficiency of 86%. Alexios et al. [1] used a visual machinesystem for citrus spectroscopy in a continuous form.The system was able to diagnose size, color and citrusdeficiency percentage with the 5% error. Blasco et al. [3]examined a visual machine system for scratchers on thecitrus skin by a piece-wise algorithm on varieties oforange and tangerine and could diagnose 95% of fruitdeficiency. Xu liming and Zhao Yanchao. [16] built anautomatic system for strawberry grading based on imageprocessing. The results showed that the detection error offruit size was less than 5%, grading accuracy in terms ofcolor and that in terms of shape were 88.8% and over 90%respectively. Iqbal et al. [11] designed an online systemfor fruit grading based on the image processing method.Jimenez et al. [12] examined a number of systems to locatethe fruit on tree and determined the greatest performanceof the systems. The sensor based systems by imageprocessing methoth, were able to detect the phase of fruitripeness from the taken photos of trees. Vaysee et al. [23]used image processing as a tool to assess fruit quality instoring boxes. They reported image processing as asuitable method to predict the quality of fruits, but it mustbe done after harvesting and before filling the boxes offruit. Kavdir et al. [13] graded the golden delicious appleand Imperor apple based on apparent quality by using neural network and image processing. Omid et al.[18] devised an intelligent system for pistachio grading.The system was equipped with neural network along witha sound analyzer. Topuz et al. [22] compared somephysical and nutritional properties of four types of fruit.Length, diameter, volume, mass, the average geometricaldiameter, spherical area, fruit density, volume density,porosity, compactness sufficient, static friction sufficient,apparent color, dried matter, dried matter soluble in water,vitamin C, PH, titratible acid, sugar and sucrose content

and calcium were amongst these parameters. Theyconsidered the measured items necessary to build animage diagnosing device. Sahraroo et al. [19] examinedthe physical properties of tangerine, gathering threedifferent types of tangerine from different regions andmixed them. By calculating mass, volume, dimension andarea, they obtained a logarithmic relationship betweenvolume and diameters, between mass and volume and nonlinear relationship between area and volume. The presentstudy aims to find the best combination of camera heightand lighting conditions for machine vision. In doing so, achamber was designed for image acquisition. in which 600LED lamps, 12 fluorescent lamps, 12 tungsten lamps, adimmer for adjusting intensity and a mechanism foradjusting camera height installed. The basis was laid outto vary the camera height simulation with change of lighttype and intensity. In most previous research for carriedout image processing no accuracy was considered tochoose the photography method. The objective of thisresearch, was to devise an adjustable set up to providesuitable conditions for image acquisition. This set up wastested in various conditions to arrive at the combinationof lighting and camera position for obtaining citrus imagefor machine vision purposes.

MATERIALS AND METHODS

Three LED, fluorescent and tungsten light sourceswere used with three different intensities to get the bestconditions for image processing algorithm (Figure 1).The type of light, the light source intensity, the lightheight and the camera height were the adjustableparameters in this research. After photography, the majorand minor diameters of each sample and surface areaswere measured using caliper and planimeter. An algorithmwas written in MATLAB for calculating the major andminor diameters of the object, the number of surface area,the average RGB of the fruit, surface roughness, the rateof reliefs on orange skin, the center of mass, the ratio ofthe major to minor diameters, that of major diameter tosurface area and that of minor diameter to surface area aswell as separating the image from the background(Figure 2). A camera (BOSCH DINON2X D/N 1/3"CCD540TVL-HV PAL) was used to take image with aresolution of 288*352 in the designed chamber. The takenphotos were separated in terms of RGB values (Red, Greenand Blue) using MATLAB based algorithm.

As seen in Figure 1, the chamber was so designed toprovide for adjusting the type, height, color and intensityof light. Three lamps were located at three heights.

World Appl. Sci. J., 18 (10): 1435-1442, 2012

1437

Fig. 1: Test chamber for image acquisitions. Fig. 2: Algorithm of the operating process.

Fig. 3: Lighting chamber in different conditions

The set of LED lamps have five rows, each consisting of the second color of light blue and the last number; 3;40 LED lamps mounted in different sides. The rows were represents the third level of light intensity (Table 1).orange, blue, red, white and green in color from top to Similarly, the treatment Tungsten 20-1-1 refers to tungstenbottom, respectively, while tungsten bulbs were lamp with camera height of 20cm, white light and the firstconsidered in red, yellow and pink. level of intensity (Table 2). There were 99 test treatments

It is worth mentioning that the intensity of all used in all.lamps could be changed to conduct the test, the camera As shown in Figure 3 while the camera height of 10was adjusted at the height of 10cm. then, the first row of cm, is fixed. all five rows of the first set of LED wereall three sets of LED lamps (orange light) were turned on turned off, the fourth row of the two other setsand the images were taken Figure 3(a). While the camera (white light ) were turned on. Then, images were taken andheight was fixed, the first row was turned off and the the light intensity was recorded. Next, camera height wassecond row of all three sets of LED lamps (blue light )were increased from 10 to 15 and 20 cm and the aboveturned on and images were taken Figure 3(b). These steps procedure was repeated and images were taken at threewere repeatect for the third (red light, Figure 3c), fourth light intensities of 50.33, 95.92 and 153.34. Tables 1, 2 and(white light, Figure 3e) and fifth rows (green light, Figure 3 show one of the repetitions in the test.3d) and the images were saved. With the change of lights, In Figure 3(g), all the lamps were turned off andthe intensity of lights was measured using LUX meter tungsten lamps were tested. First, the camera was fixed at(TES1339R) apparatus simultaneously (Figure 3h). The the height of 10 cm. Then, the white light of all three rowsexperiment included three parameters of camera height, of tungsten lamps were turned on and images werethe lamp row and light intensity. Treatments combinations acquised. Other cases are shown in the Table 2.are given in figure 3 and Tables 1, 2 and 3. For example in In Figure 3(f), all lamps were turned off andtreatment “LED 10-2-3”, the first number; 10; represents fluorescent lamps were switched. Images were then takenthe camera height in cm, the second number; 2; represents at three camera heights of 10, 15 and 20 cm (Table 3).

Te E=

World Appl. Sci. J., 18 (10): 1435-1442, 2012

1438

Table 1: Different situations of LED lamp

LED lamp Camera Height(cm) Intensities Orange light 10, 15, 20 50.33,95.92,153.34 Blue light 10, 15, 20 50.33,95.92,153.34Red light 10, 15, 20 50.33,95.92,153.34White light 10, 15, 20 50.33,95.92,153.34Green light 10, 15, 20 50.33,95.92,153.34White light of two first row ON 10, 15, 20 50.33,95.92,153.34

Table 2: Different situations of tungsten lampTungsten lamp Camera Height(cm) Intensities

White light 10, 15, 20 50.33,95.92,153.34Pink, Yellow, Red light 10, 15, 20 50.33,95.92,153.34Yellow light 10, 15, 20 50.33,95.92,153.34

Table 3: Different situations of fluorescent lamp

Fluorescent lamp Camera Height (cm) IntensitiesWhite light 10, 15, 20 50.33,95.92,153.34

Obtaining the Best Background Color: In order to arriveat the best background color, images were taken whitwhite, orange, cream, brown and black backgrounds andgray scale histograms were drown for both thebackgrounds and fruits in order to obtain the bestbackground color. Having determined the most suitablebackground color, photography of main samples withsuitable background in the image chamber was taken.Separation of fruit from the background took place on thebasis that RGB image taken by the camera wastransformed in to R, G and B values and then red, greenand blue histograms were developed. Data such as RGBof the fruit and the background, major and minordiameters, surface area, center of mass, circumference andwhite pixels of the fruit texture were measured. The pixelsremaining in the background after application of filter areknown as noise. Ratios of Major diameter to area, minordiameter to area and minor to major diameter ratio werecalculated and recorded for all 99 cases. On the otherhand, the actual values of major diameter, minor diameterand actual surface area of the samples were measuredmanually and actual ratios of major diameter to surfacearea, minor diameter to surface area and minor diameter tomajor diameter were determined.

Total error between the algorithm output andactual values were calculated using equations 1 and 2. Adiagram was drawn for all cases and all intensities fromwhich the outlying values were removed. The error withthe least frequency was chosen as the best case for imageacquisition.

E = (LA-la) + (WA-wa) + (WL-wl) + N (1)2 2 2 2 2

(2)

LA : The ratio of major diameter to surface area(determined by the algorithm)

WA : the ratio of minor diameter to surface area(determined by the algorithm)

WL : The ratio of minor diameter to major diameter(determined by the algorithm)

La : The ratio of major diameter to surface area(measured by with caliper)

Wa : The ratio of minor diameter to surface area(measured by with caliper)

Wl : The ratio of minor diameter to major diameter(measured by with caliper)

N : The number of noises in the background texture

The tests were conducted at three light intensities for99 different cases. To ensure the accuracy of thementioned methods, first, 99 different cases were testedexperimentally. After trouble shooting, the experiment wascarried out in the same 99 cases. Initial test indicated thatwhen photo is taken from the top or bottom of orange, thehole in the bottom or the pedicel may create difficulties forthe algorithm. Therefore, images were taken from the sidesof each fruits.

Four treatments of photography are shown in Figures6 to 10 each of letter (a, b, c and d) in each picturereferring a treatment. (a: LED10 with light intensity of50.33, b: tungsten10 with light intensity 50.33, c:Fluorescent 20 with light intensity 95.92, d: Tungsten 20with light intensity 153.34).

RESULTS AND DISCUSSION

After the selection of background, the best color forcalculations of diameter, texture, RGB, etc. must bespecified. As seen in Figure 4, the two visible peaks ineach image are associated with fruit color andbackground. In Figure 4c, these two peaks are closestdistance to each other while in Figure 4a, they are furthestapart. Therefore, Figure 4a is more suitable than the twoother figures for determining the threshold ofdistinguishing the fruit from the background.

The threshold value is the one through which thealgorithm can recognize the fruit range from thebackground. The more exact this value, the better theseparation and the lower the noise. Figure 5a,b and cshow histograms of the fruit against black, brown andcream backgrounds respectively. As seen in Figure 5,

World Appl. Sci. J., 18 (10): 1435-1442, 2012

1439

Fig. 4: Determination of best threshold to separate the fruit from background

Fig. 5: Determination of the best color for background

Fig. 6: Images of real samples

Fig. 7: Red image if initial photos

Fig. 8: Separation of fruit from background by algorithm

World Appl. Sci. J., 18 (10): 1435-1442, 2012

1440

Fig. 9: Detection of fruit roughness and background

Fig. 10: Removal of background roughness

Fig. 11: Graphs of the error

Table 4: Total error

Condition (LA-la) (WA-wa) (WL-wl) N Total error<62 2 2 2

Red LED10-3(Intensity=50.33) 0.007819 0.00011 0.000103 9 3.001338463Orange LED15-1(Intensity=50.33) 0.007194 5.64E-05 4.72E-05 9 3.001216083LED10-first row off(Intensity=50.33) 0.009629 0.000118 0.00011 4 2.002462779LED15-first row off(Intensity=50.33) 0.007484 6.21E-05 5.28E-05 16 4.000949768LED20-second row off(Intensity=50.33) 0.005813 2.43E-05 1.69E-05 9 3.000975YellowTngestan20-3(Intensity=95.92) 0.007622 2.21E-05 1.38E-05 9 3.001275975florsent10(Intensity=153.34) 0.006429 0.000112 0.000106 25 5.000664673green LED20-5(Intensity=153.34) 0.106587 2.23E-05 5.48E-07 4 2.02647718

when we proceed from black to lighter backgrounds, the In fact, there are pixels in this background which thefrequency lines of color distributions of the fruit and the algorithm mistakes as fruit resulting in erroneousbackground become closer to each other and the prediction.threshold of separating the fruit from the background The Figures 6 to 10 indicate the importance ofbecomes more difficult with more likely errors. In adjusting the height, intensity and color of the light. Withhistogram of fruit color and the cream background, it is the change in height and type of light, there appear greatnot possible to separate the fruit form the background. differences in the result of photography and processing

World Appl. Sci. J., 18 (10): 1435-1442, 2012

1441

the final image by the algorithm. In sample c, the quality reported, 5 treatments belong to the first lightof red image (Figure 7c) is not satisfactory and separating intensity. As 6 out of 8 treatments lie in the range ofthe fruit form the background has not been accomplished LED lamps, it is concluded that LED lamp is morecorrectly (Figure 8c). Regarding the threshold level for suitable than the two other lamps to reflect light. Based onseparating background pixels, there are still white pixels in Tables 1, 2, 3 and 4 Diagram(d), treatments 22 and 90 givethe background. Since the exact value of the threshold by the best conditions and there is the least error inthe background colors was obtained by the operator treatment 22, making it the best condition for imagebased on trial and error, the algorithm must be able to acquisition. distinguish the fruit from the background (Figure 8),However, in Figures 8(b), 8(c) and 8(d) unsuitable ACKNOWLEDGMENTconditions of photography caused some problems. Thus,the algorithm cannot accomplish the action of separation. The authors would like to thank the University ofIn figure 8a, the fruit has not been separated from the Razi (Department of Agricultural Machinery,) for technicalbackground; therefore, it cannot distinguish major and supporting of this work. minor diameters, pixels of fruit surface and RGB(Figure 8c and Figure 8d) and there appear errors. For REFERENCESexample, since the algorithm recognizes the backgroundpixels as fruit pixels, the diameter and the area are over 1. Aleixos, N., J. Blasco, F. Navarron and E. Molto.2002.estimated. Diameter and area are the main parameters for Multispectral inspection of citrus in real-time usingestimating mass and volume which must be estimated machine vision and digital signal processors.accurately. Otherwise, significant errors can arise in Computers and Electronics in Agriculture. 33 : 21-137.calculations. As shown in figure 9, different conditions of 2. Arefi, A., A. Modarres Motlagh, K. Mollazade andphotography can influence the diagnosis of fruit texture R. Farrokhi Teimourlou, 2011. Recognition and(surface roughness). More favorable the conditions, lead localization of ripen tomato based on machine vision.to better diagnosis of fruit texture, which is critical to the Australian Journal of Crop Science. 5(10): 1144-1149.development of systems for fruit type determination. 3. Blasco, J., N. Aleixos and E. Molto, 2007. Computer

In Figures 10(d) and 10(c) in which it was aimed to vision detection of peel defects in citrus by means ofclear white pixels of the background, it is observed that a region oriented segmentation algorithm. Journal ofthe goal was not accomplished due to the absence of food Engineering. 81: 535-543.suitable conditions for light and camera height. The 4. Blasco, J., N. Aleixos, J. Gomez-Sanchis andalgorithm is written in such a way that is able to remove E. Motto, 2009. Recognition and classification ofthe background texture using filters. Less favorable external skin damage in citrus fruit using multicondition (Figure 10) makes the removal of background spectral data and morphological features. Biosystemstexture more difficult. When the background pixels are not Engineering. 103: 137-145.properly removed, the number of white pixels of the fruit 5. Blasco, J., S. Cubero, J. Gomez-Sanchis, P. Mira andappear greater than actual. E. Molto, 2009. Development of a machine for the

Diagrams a, b and c in figure11 shows the error automatic sorting of pomegranate(Punicafrequency of imaging with three intensities of 50.33, 95.92 granatum)arils based on computer vision. Journal ofand 153.34. Diagram(a) shows the total detected cases of food Engineering. 90: 27-34.intensity 50.33, while the Diagram(b) and the Diagram(c) 6. Brosnan, T. and D.W. Sun, 2002. Inspection andshow those of intensity 95.92 and 153.34 respectively. As grading of agricultural and food products byseen in this figures, due to change in colors, light type computer vision systems-a review. Computers andand camera height, some error fluctuations have appeared. Electronics in Agriculture. 36: 193-213. Where the color, light type and camera height create more 7. Cayuelaa, J.A. and C. Weilandb, 2010. Intact orangesuitable conditions, the error diagram is descending and quality prediction with two portable NIRotherwise it is ascending due to the presence of noise. spectrometers. Postharvest Biology and Technology.Table 4 shows the least errors brought about in 99 test 58: 113-120 cases. From Table 4, it is concluded that the errors 8. GHazanfari, A., J. Irudayaraj and A. Kusalik, 1996.resulted from the first light intensity are less than those of Grading pistachio nuts using a neural networkthe other two intensities, because, within the 8 cases approach. ASABE. 39: 2319-2324.

World Appl. Sci. J., 18 (10): 1435-1442, 2012

1442

9. Gonzalez, R.C., R.E. Woods and S.L. Eddins. 2004. 18. Omid, M., A. Mahmoudi and M.H. Omid, 2009.Digital Image Processing Using MATLAB An intelligent system for sorting pistachio nut(Second ed.). Prentice Hall. varieties.Expert Systems with Applications. 36: 9.

10. Hongmei Zhang, A., A. Jun Wang and Ye. A. Sheng, 19. Sahraroo, A., A. Khadivi Khub, A.R. Yavari and2008. Predictions of acidity, soluble solids and M. Khanali, 2008. Physical properties of Tangerine.firmness of pear using electronic nose technique. American-Eurasian J. Agric. and Environ. Sci.,Journal of Food Engineering. 86: 370-378. 3(2): 216-220.

11. Iqbal, S., D. Ganesan and Rao. P. Sudhakara, 2009. 20. Sharifi, M., S. Rafiee, A. Keyhani, A. Jafari, H.Mechanical system for on-line fruit sorting and Mobli, A. Rajabipour and A. Akram, 2007. Somegrading using machine vision technology. J. Instrum. physical properties of orange (var. Tompson). Int.Soc. India. 34(3): 153-162. Agrophysics. 21: 391-397.

12. Jimenez, A.R., R. Ceres and J.L. Pons, 2000. A 21. Sirisathitkul, Y., N. THumpen and W. Puangtong,survey of computer vision methods for locating 2006. Automated chokun orange maturity sorting byfruit on trees. Transactions of the ASABE. color grading.Walailak J. Sci and Tech., 3(2): 195-205.43(6): 1911-1920. 22. Topuz, A., M. Topakci, M. Canakci, I. Akinci and

13. Kavdir, I. and D.E. Guyer, 2002. Apple sorting F. Ozdemir, 2005. Physical and nutritional propertiesusing Artificial Neural Network and Spectral of four orange varieties. Journal of Food Engineering.imaging.ASABE. 45: 1995-2005. 66: 519-523.

14. Kleynen, O., V. Leemans and M.F. Beslain, 2005. 23. Vaysse, P., G. Grenier, O. Lavialle, G. Henry, S. Khay-Development of a multi-spectral vision system for Ibbat, C. Germain and J.P. Da Costa, 2005. Imagethe detection of defects on apples. Journal of Food processing as a tool for quality assessment of fruitsEngineering. 69: 41-49. in bulk shipping bins.Information and Technology

15. Leemans, V., H. Magein and M.F. Destain, 2002. for Sustainable Fruit and Vegetable Production.On-line fruit grading according to their external 05: 12-16.quality using machine vision.Biosystems 24. Xiaobo, Z., Z. Jiewen and L. Yanxiao, 2010.Engineering. 83: 397-404. Objective quality assessment of apples using

16. Liming, X. and Z,H. Yanchao, 2010. Automated machine vision, NIR spectrophotometer andstrawberry grading system based on image electronic nose. Transactions of the ASABE.processing.Computers and Electronics in Agriculture. 53(3): 1351-1358.71: 32-39.

17. Omid, M., M. Khojastehnazhand and A.Tabatabaeefar, 2010. Estimating volume and mass ofcitrus fruits by image processing technique. Journalof Food Engineering. 100: 315-321.