1 23

Precision AgricultureAn International Journalon Advances in PrecisionAgriculture ISSN 1385-2256Volume 11Number 6 Precision Agric (2010)11:684-702DOI 10.1007/s11119-010-9193-2

Multi-phase cross-correlation method formotion estimation of fertiliser granulesduring centrifugal spreading

1 23

Your article is protected by copyright and

all rights are held exclusively by Springer

Science+Business Media, LLC. This e-offprint

is for personal use only and shall not be self-

archived in electronic repositories. If you

wish to self-archive your work, please use the

accepted author’s version for posting to your

own website or your institution’s repository.

You may further deposit the accepted author’s

version on a funder’s repository at a funder’s

request, provided it is not made publicly

available until 12 months after publication.

Multi-phase cross-correlation method for motionestimation of fertiliser granules during centrifugalspreading

B. Hijazi • F. Cointault • J. Dubois • S. Coudert •

J. Vangeyte • J. Pieters • M. Paindavoine

Published online: 19 September 2010� Springer Science+Business Media, LLC 2010

Abstract Excessive fertiliser use has been a main contributor to the increasing

environmental imbalance observed in the past 20 years. Better accuracy in spreading

would limit excess fertiliser loss into the environment. Increased accuracy begins by

understanding the fertiliser spreading process from the vane to the soil. Our work con-

centrates on the use of centrifugal spreaders, as these are most commonly used in Europe.

Progress in imaging devices and image processing has resulted in the availability of new

technologies to use when describing the behaviour of fertiliser granules during ejection

from centrifugal spreaders. Fertiliser deposition on the soil can be predicted using a bal-

listic flight model, but this requires determination of the velocities and the directions of the

granules when they leave the spinning disc. This paper presents improvements to the high

speed imaging system that we had previously developed, i.e. enhancements to the illu-

mination and the image processing. The illumination of the previous system, which used

many separate flashes, did not give consistent illumination. We have improved it by using a

stroboscope with power-LEDs, located at 1 m height around the digital camera and

B. Hijazi (&) � F. CointaultAgroSup Dijon, UP GAP, 26, Bd Dr Petitjean, BP 87999, 21079 Dijon cedex, Francee-mail: b.hijazi@agrosupdijon.fr

J. DuboisLe2i UMR CNRS 5158, University of Burgundy, BP 47870, 21078 Dijon cedex, France

S. CoudertLML-UMR CNRS 8107, Cite Scientifique, 59655 Villeneuve d’Ascq cedex, France

J. VangeyteTechnology and Food Science Unit, Agricultural Engineering, Institute for Agricultural and FisheriesResearch (ILVO), Burg Van Gansberghelaan 115, 9820 Merelbeke, Belgium

J. PietersDepartment of Biosystems Engineering, Faculty of Bioscience Engineering, Ghent University,Coupure Links, 653-9000 Ghent, Belgium

M. PaindavoineLEAD UMR CNRS 5022, University of Burgundy, Pole AAFE, BP 26513, 21065 Dijon cedex, France

123

Precision Agric (2010) 11:684–702DOI 10.1007/s11119-010-9193-2

Author's personal copy

controlled by a Field-programmable gate array (FPGA) card. The image processing has

been improved by development of a multi-phase method based on a cross-correlation

algorithm. We have compared the cross-correlation method to the Markov Random Fields

(MRF) method previously implemented. These tests, based on multi-exposure images,

revealed that cross-correlation method gives more accurate results than the MRF tech-

nique, with guaranteed sub-pixel accuracy. Knowing that an error of one pixel can lead to a

prediction error between 200 and 500 mm on the ground, the latter method gives an

accuracy range between 0.1 and 0.4 pixels, whereas the MRFs technique is limited to 3 and

9 pixels for the vertical and horizontal components of the velocities, respectively. The sub-

pixel accuracy of the new method was proven by applying it on simulated images with

known displacements between the grains. By using a realistic spreading model, the sim-

ulated images are similar to those obtained with a high speed imaging system. This sub-

pixel accuracy now makes it possible to decrease the resolution of the camera to that of a

classical high-speed camera. These improvements have created an affordable and durable

system appropriate for installation on a spreader. Farmers could use this system to both

calibrate the spreader and verify the fertiliser distribution on the ground.

Keywords High-speed images � Fertiliser granules � Motion estimation �Cross-correlation

Context and objectives

In 2006, the Joint Research Centre of the European Commission published ‘‘An Atlas of

Pan-European Data for Investigating the Fate of Agrochemicals in Terrestrial Ecosystems’’

(FATE) (Mulligan et al. 2006). This document showed the application rates of nutrients

such as nitrogen and phosphorous, and concluded that the Netherlands, Belgium, Denmark

and France apply the highest nutrient pressure on the environment (an excess from 50 to

200 kg/ha). Applications were found at times to be twice as high as crop needs (Mulligan

et al. 2006). Sustainable nutrient and water management requires that these farming

practices be modified. Accurate fertiliser application is particularly important, as it affects

farmers’ profit margins and can cause environmental side effects. However, farmers do not

have the machinery, agronomic support or operator knowledge to apply fertilisers at the

correct rate and uniformity required to produce healthy crops on the one hand and prevent

environmental side effects and yield losses on the other hand.

The objective of this project was to combine efficiency (optimal crop growth), economy

(reduced fertiliser input and higher profit) and ecology (conformity to European standards).

Precision agriculture techniques call for closed loop regulation systems with appropriate

sensors in order to accurately manage the local fertilisation rate and control the fertiliser

distribution on the soil. Further, only a limited amount of precision fertilisation equipment

is currently available for purchase. In order to decrease the application rate, farmers need a

system capable of real time characterisation of the local fertilisation rate. This parameter

can only be accurately controlled through monitoring the distribution pattern on the

ground, particularly when using a centrifugal spreader. This last parameter depends on

many factors, such as construction and calibration of the machinery, particle types and

properties and field conditions.

The current system requires a tedious and laborious process: in-field setup of collection

trays, removing collected material from each tray, weighing the contents, entering the

Precision Agric (2010) 11:684–702 685

123

Author's personal copy

weight data into a computer program, and printing the output. In addition, the farmers do

not generally verify the correct adjustment of the spreader, which usually skews the data.

Other techniques have been investigated, particularly the use of a high-speed imaging

system and specific image processing developed at AgroSup, Dijon, in France (Cointault

et al. 2002). That system determines the initial conditions of flight of the granules in the

vicinity of the spreading disc. Improvement of this system could lead to an optimised

imaging device mounted directly on a spreader. Such a system would characterise the

fertiliser granules and offer an alternative to the ‘‘collection trays’’ method.

Imaging or optical techniques to characterise fertiliser granule parameters at ejection

In 1997, Grift and Hofstee’s research on optical methods showed that spatial distribution of

the particles on the ground can be estimated by calculating the ballistic flight of the

particles. They started from the particle’s initial flight conditions of velocity and direction

and their properties and geometrical parameters, e.g. topography, height and tilt of the

discs. The variables such as the direction of the horizontal outlet and velocity are used as

input for a ballistic model that estimates the distribution of fertiliser on the ground.

The success of the method is based on two components: a reliable model and the accurate

measurement of the variables. To determine these variables, Grift and Hofstee developed

an optical sensor. Then they used a ballistic model taking into account the friction of grains

with the air. However, this sensor cannot characterise a group of grains: it only measures

the velocity and diameter of an individual particle shortly after leaving the disc.

Later, a French spreader manufacturer, Sulky, developed the ‘‘Justax’’ system. This

consisted of two small aluminium rails equipped with piezoelectric sensors located on a

rotating arm that swept through the flow of fertiliser granules. This resulted in a Gaussian

curve indicating the location of the main granule ejection, but provided no information on

the angular mass flow and on the mean ejection angle. Moreover, this technique interfered

with the fertiliser distribution on the soil. The system also had to be frequently replaced

due to corrosion caused by the fertiliser.

Recently, another spreader manufacturer, Amazone S.A., developed a vision system

called the Argus camera, which automatically sets the fertiliser spreader. A camera system,

with pulsating IR radiation to reduce ambient light disturbance, records the distribution

pattern during the spreading. The recorded distribution is compared to distribution data

stored in the database of the onboard computer. Although the system is an important

improvement, it does not take into account the disc concavity and vane configurations. The

field of view is very small and little processing is done to define the spreading parameters.

Due to the relatively high speed (from 25 to 40 m/s) of the fertiliser granules, we

proposed a high-speed imaging system to determine the granule trajectories (Cointault

et al. 2002). The poor resolution of the resulting images at that time made the results

unacceptable. High-speed cameras with sufficient resolution were not common and too

expensive to be used in agricultural practice.

Then Cointault et al. (2003) and Vangeyte and Sonck (2005) presented alternatives to

high-speed cameras by combining a high-resolution monochrome CCD camera with strobe

systems (Cointault and Vangeyte 2005). Vangeyte and Sonck (2005) used a small field of

view of 0.10 m 9 0.10 m and a LED stroboscope to capture the grain flow while Cointault

et al. (2003) captured the total grain flow using flashes on a 1 m 9 1 m field of view as

shown in Fig. 1.

686 Precision Agric (2010) 11:684–702

123

Author's personal copy

The motion estimation proposed by Cointault et al. (2003) combined a theoretical model

of the granule distribution and MRF method, based on the determination of optical flow

characteristics on the images. The results for granule velocities were close to velocities as

visually evaluated from examination of the image. However, this system has only been

tested on a simplified spreader configuration (one single flat disc and radial vanes), whereas

classical centrifugal spreaders have two concave rotating discs and two or more pitched

vanes. This system cannot be installed on an actual spreader, owing to the:

• high cost of the camera flash system relative to the cost of a classical centrifugal

spreader,

• inconsistent illumination

• lack of robustness of the camera

• determination of only one spreading parameter (granule trajectories).

In response to these challenges, Villette et al. (2007) derived a simplified imaging

technique from the one described above. Based on simplified hardware, faster processing

algorithms and a simpler single-particle mechanical model, it provided average values of

basic spreading parameters such as the average velocity and the angular distribution of the

particle flow. This work significantly contributed to understanding of the centrifugal

spreading because it accounts for real configurations of centrifugal spreaders such as

concave disc and pitched vanes. This study has yielded preliminary information on friction

coefficients, but the angular distribution of the fertiliser cannot yet be accurately deter-

mined. Furthermore, this method does not determine the fertiliser’s granulometry and the

granule’s behaviour inside the vanes. The final results are still based on calculated speeds

rather than measured actual speeds.

Following these lines of study, there are three main approaches to create a robust on-

spreader image acquisition system:

• Motion-blurred images, resulting in a limited number of granule ejection parameters,

• Low-cost multi-exposure approach with the device based on a stroboscope (flashes or

LEDs) and a standard high resolution camera,

• Multi-exposure approach (only two successive images) with low-cost device based on a

stroboscope (power-LEDs), a high speed camera (low resolution) and high performance

algorithm.

This third approach optimizes the multi-exposure imaging system by first developing a

robust and easy-to-use power-LED-based stroboscope and combining it with a new motion

Fig. 1 Principle ofmulti-exposure imaging

Precision Agric (2010) 11:684–702 687

123

Author's personal copy

estimation method based on cross-correlation. The advantages of this approach are that it

provides sub-pixel accuracy in the velocity calculation, which allows the resolution of the

standard camera to be decreased and thus use low-cost high-speed cameras. In order to

validate the new motion estimation method, a comparison with concrete velocity values is

needed. To address this need, a simulator has been developed It creates images using

known grain velocities and trajectories as inputs of the simulation algorithm. These are

then used as reference to evaluate the results of the motion estimation algorithm applied to

the simulated images.

The main objectives are thus to propose a unique device capable of characterising the

fertiliser, to establish a typology of products (behaviour, friction resistance, ejection

characteristics, etc.), to account for the influence of the spreader type, and finally to install

it on a spreader for online fertiliser regulation.

Improvements of the current high-speed imaging system and associated processing

Illumination with power-LEDs

Accurate scene illumination requires a lighting system with the following characteristics:

high luminosity power, robustness, automated control and low cost. First tests have shown

that four 200 W spotlights furnish around 5,000 lx at the level of the spinning disc (dis-

tance is 200 mm between the spotlights and the luxmeter). With these lighting conditions

and the camera located at 1 m height, images of the fertiliser granules are over-exposed.

Around 1,000 lx gives a sufficiently clear image when the power-LEDs are used in a

stroboscopic configuration. To decrease the displacement of the granules in between

the 20 ls flashes, the stroboscopic system must work at very high frequencies

(1,000–10,000 Hz).

Stroboscopes work in the same way as photographic flashes. A condenser is discharged

through a transformer, which produces a luminous flash. The duration and intensity of the

flash depend on the characteristics of the electronics used. Degradation of the electronic

components limits the life of the flashes to around 10,000 flashes. The recycling time of the

classical stroboscope is difficult to modify and limits the frequency. The LED stroboscope

developed by Vangeyte et al. (2006) can work at high frequencies but does not provide

sufficient illumination for our application. The combination of frequency and illumination

necessary for our application cannot be obtained with the existing stroboscopes.

Power-LEDs appear to be a good alternative to the illumination systems mentioned

above. Experiments began with white power-LEDs because they have improved greatly

over the last 5 years and are relatively inexpensive. This system was controlled by a Field-

Programmable Gate Array (FPGA) card, allowing easy modifications of the illumination

configurations into several modes, i.e. sequence or single flash with different parameters

such as illumination time, inter-flash time and eventually inter-sequence flash. Figure 2

illustrates the entire system along with the stroboscope functioning sequence.

The output can be set from one to 32 flashes. Eight flashes were used, the number used

in the previous stroboscopic setup. Image acquisition was synchronised with the granular

flow: the passage of the vane triggers an external sensor to take the picture. The power-

LEDs must be positioned uniformly around the camera lens to fulfil two demands: suffi-

cient and homogeneous illumination. AgroSup Dijon has completed a preliminary study

characterising the illumination distribution of power-LEDs (Jaton and Biryukov, personal

communication, 2008).

688 Precision Agric (2010) 11:684–702

123

Author's personal copy

For Star Led1 power-LEDs with the following characteristics: 80� of angle of view,

3 W, 7000 K, the illumination (E) provided can be approximated by a polynomial equation

of 7th order:

Ei ¼X7

j¼ 1

kj � dð7� jÞi

� �ð1Þ

where Ei the illumination from a point i and kj is the coefficient of the jth term of

polynomial equation.

If Xc and Yc are the co-ordinates of the projection of the lighting source on the field of

view and Xi and Yi, the co-ordinates of the point i, the distance between the two points is:

di ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXc � Xið Þ2þ Yc � Yið Þ2

qð2Þ

If the lighting source is composed of n power-LEDs, a coefficient n which gives this

equation is introduced:

Ei ¼ nLED �X7

j¼ 1

kj � dð7� jÞi

� �ð3Þ

When combining the above Eqs. 2 and 3, we obtain the illumination of the point i

according to the co-ordinates and the location of the lighting source:

Ei ¼ n �X7

j¼ 1

kj �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXc � Xið Þ2þ Yc � Yið Þ2

q ð7� jÞ !

ð4Þ

If several lighting sources are placed above the field of view, each source creates an

illumination Epi for a point i. The entire field of illumination is then described by Eq. 5:

EP i ¼XP

p¼ 1

Epi ¼XP

p¼ 1

np �X7

j¼ 1

kj �ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXcp � Xi

� �2þ Ycp � Yi

� �2q ð7� jÞ

!" #ð5Þ

Fig. 2 The whole stroboscopic command and functioning sequence

1 Luxeon star Led (http://www.luxeonstar.com).

Precision Agric (2010) 11:684–702 689

123

Author's personal copy

where Xcp and Ycp are the co-ordinates of each power-LED and P, the number of lighting

sources.

Figure 3 shows the resulting illumination for two 3 W power-LEDs separated by

1.10 m and proves that the illuminations of two lighting sources can be added to provide

higher luminosity.

The second fundamental point concerns the homogeneity of the illumination. The

resulting illumination of the system depends on the characterisation of the luminosity of

each power-LED. All motion estimation methods rely on homogeneous illumination, but

this is particularly important for those using optical flow information. We calculated the

mean illumination of a grid of power-LEDs on a surface of 1 m2. The theoretical spread

angle of the granules is 180� but, in practice, the spread angle of interest is only 120�. The

lighting sources thus have to illuminate a scene corresponding to this angle.

In a second study, we proved empirically by simulation that two different arrangements

(both of which take the height of the camera into account) appear to be the best solutions

for our application (Fig. 4).

The lighting arrangement does not depend on any particular spreader, since it is placed

one metre above the spreader.

Fig. 3 Illuminations in l9 obtained for two power-LEDs separated by 1.10 m with a spatial sampling of50 mm

Fig. 4 Best illumination of the field of view with four power-LEDs located on a square (left) or six power-LEDs located on a hexagon (right)

690 Precision Agric (2010) 11:684–702

123

Author's personal copy

Arranging the power-LEDs to form a square creates a more homogeneous illumination

than the hexagonal arrangement. However, the maximum value of the average illumination

(51.18 lx) is half of the value for an arrangement in a hexagon (105 lx). Therefore, to

obtain around 1,000 lx illumination at the necessary 1 m height, a modified hexagonal

arrangement with several power-LEDs on each corner was used as shown in Fig. 5. Eight

power-LEDs positioned at each corner of a hexagon inscribed within a circle with a radius

of 700 mm were necessary to give constant illumination inside 1 m2.

In that case, the average illumination is 600 lx with 48 3 W power-LEDs (Fig. 6).

This value does not reach the 1,000 lx required to illuminate 1 m2 at 1 m height. But the

average illumination depends on the number of power-LEDs for each edge. For example, if

10 power-LEDs are used per edge, all 60 power-LEDs provide an average illumination of

750 lx. Therefore, to reach the 1,000 lx needed it is sufficient to increase the number of

LEDs on each corner of the hexagon.

As a follow-up study, we are comparing the illumination results for different kinds of

power-LEDs: 3 or 5 W; 3000 or 7000 K; and illumination angles of 80�, 120� or 140�.

In order to model these power-LEDs, the coefficient kj of the previous equations has to be

recalculated. Then, an experimental validation of the strobe system will be done.

Fig. 5 New hexagonalarrangement with eight power-LEDs at each location

Fig. 6 Illumination with 46.3 W power-LEDs arranged uniformly

Precision Agric (2010) 11:684–702 691

123

Author's personal copy

Granule velocity determination with cross-correlation method

Motion estimation techniques

Motion study through image processing can be broken down into three main activities:

detection, analysis and estimation. Estimation requires evaluation of different parameters

such as types of movement detected, techniques developed, size of the detected objects,

length of the displacements, management of the illumination problem, and tests done (as a

synthesis or on actual images).

In centrifugal spreading, the distribution of the fertiliser granules on the ground results

from two successive steps: first, the ejection and, second, the ballistic flight of the

granules. The motion of the granule is one of the most important parameters, particularly

the velocities of the granules after leaving the spinning disc. To evaluate these velocities,

several motion estimation methods can be used. The following characteristics of the

fertiliser granules and their motion need to be taken into account: size of the granules

(5 mm), motion discontinuity, effect of the centrifugal force and lift effect. The fertiliser

granules make very large displacements in pixels/image as compared to the displace-

ments generally estimated with classical motion estimation methods. The fertiliser dis-

placement cannot be detected directly by such methods as Markov Random Fields

(MRF) (Cointault et al. 2003), block matching or optical flow measurement (Barron and

Thacker 2005), even if we obtain a vector field describing the displacement of each point

between two successive images. Indeed, the maximum displacement which can be

detected is very small (\3 pixels/image). This can lead to some errors in our application.

Due to the necessity of estimating local motion, a possibility was to use Gabor filters

(Bruno and Pellerin 2000), or to use a combination of spatio-frequency methods. These

filters, used in a triad of controlled filters, have the double advantage of eliminating the

modelling and minimisation steps of the MRF technique. Unfortunately, Hijazi et al.

(2008) proved that this method does not accurately measure the displacements. The

second possibility was to use cross-correlation methods, considered as reference methods

for motion estimation.

Cross-correlation methods

The cross-correlation method can be used in signal processing as well as image pro-

cessing. In signal processing, cross-correlation is used to measure the similarity of two

waveforms. In image processing, it has applications in pattern recognition, single particle

analysis, PIV (Particle Image Velocimetry) (Fournel et al. 2003; Foucaut et al. 2003).

Processing images using the cross-correlation method is based on the correlation between

the same blocks in successive images. The similarity between blocks is given by the

difference between luminosities of the blocks using the Sum of Absolute Difference

(SAD).

Different variations of cross-correlation algorithms (CCA) have been used, such as

calculating the difference to replace multiplication, the mean squared error, the absolute

error, or the median squared error (Wesley et al. 2004; Traver and Pla 2005). A normalised

cross-correlation algorithm (NCCA) (Nillius and Eklundh 2002) gives better results by

using the local mean value and the local variance value. Since the images of fertiliser

granules are subject to noise and variation of luminosity, the use of NCCA algorithm is

more appropriate to our study.

692 Precision Agric (2010) 11:684–702

123

Author's personal copy

Motion estimation based on a single stage cross-correlation algorithm

For each of the algorithms mentioned above, cross-correlation motion-estimation methods

are generally based on a single stage algorithm, called ‘‘full search algorithm’’. For each

pattern in an image I1, taken at the instant t, the position that gives the maximum corre-

lation to an image I2, taken at the instant t ? Dt, is searched for. Hijazi et al. (2009) have

tested this algorithm; it does not give the expected results for our application. The shapes

of the fertiliser granules are very similar. The lift of the granules due to the centrifugal

ejection causes the granule shape to vary on the images between two exposures; thus, when

using a full search algorithm, there is a considerable possibility of finding one or multiple

correlation maxima. The specific motion of the granules calls for an improvement of the

standard methodology, namely estimation in two successive steps.

Motion estimation based on a two-stage cross-correlation algorithm

We thus developed a two-step algorithm. Both steps use cross-correlation to find the

velocity vectors. The first step calculates one total displacement vector for each fertiliser

throw, and the second one refines this vector to estimate the local motion for each pixel

(Fig. 7). This methodology has been used with relative success for the MRF technique, as

briefly presented above.

With this method, the modulus of the velocity vectors obtained for actual fertiliser

images are close to visually determined velocities of a few granules on the images. The

results are presented in Fig. 8.

However, the direction of resulting velocity vectors is the same for all pixels, which

does not correspond to reality. In fact, centrifugal force acts on the grains in fertiliser

throws, resulting in different directions for each grain depending on its location in the

throw. This scatters the throw. To take the centrifugal effect into account, we propose a

multiple global velocity vector.

The flow chart of the strategy for the motion estimation with cross-correlation technique

is described in Fig. 9a.

The fertiliser throw is first determined by detecting a Region Of Interest (ROI). By

focusing on this region of the image that contains the throw, we can reduce the data treated

in the following steps of the algorithm. Once the throw is detected, three of its points are

determined to calculate the circle that models the throw.

Fig. 7 Principle of the two-stepcross-correlation algorithm

Precision Agric (2010) 11:684–702 693

123

Author's personal copy

The ‘‘throw cut’’ step (Fig. 9b) cuts the fertiliser granule flow into several areas

according to the four following steps:

• Determination of three points in the throw: First, the throw is detected as a ROI. Then

two points, i.e. the top and the end of the throw, are determined. To find the third point,

we calculate the centre of gravity of a square in the centre of the throw to ensure the

passing of the modelling arc near by the higher concentration of the grains.

• Calculation of the centre of the circle: the three points determined above are sufficient

to calculate the centre and the radius of the circle that passes through those three points.

• Determination of the opening angle: this is the angle between the lines connecting the

centre of the circle with the top and the end of the arc (Fig. 10).

• Cutting points: the number of cutting points is fixed depending on the ratio of the

chords (l1 and l2) that link the top to the end of the arc modelling the throw in the first

and the second images (r = l1/l2). The lower the value of r, the higher the number of

cutting points. The cutting points on the arc are then calculated by dividing the opening

angle by the number of cutting points.

The ‘‘global motion calculation’’ step allows for extraction of one global motion vector

by area. To determine the global motion vector, we use cross-correlation to find the

0 20 40 60 80 100

40

60

80

100

120

140

0 20 40 60 80 100

40

60

80

100

120

140

400 450 500 550 600 650 700

700

750

800

850

900

950

400 450 500 550 600 650 700

700

750

800

850

900

950

(a) (b)

(c) (d)

Fig. 8 a Image of two granules, b corresponding velocity vectors obtained with cross-correlation, c realimage of two fertiliser throws, d corresponding velocity vectors obtained with cross-correlation

694 Precision Agric (2010) 11:684–702

123

Author's personal copy

position that gives the maximum correlation between the areas in the first image and the

second one.

Then comes the ‘‘local motion calculation’’. For each pixel of the throw, the local

motion vectors are calculated by refining the global calculation. That is done by fixing the

search window around the position predicted from the global estimation.

The last step is to perform a sub-pixel refinement. The cross-correlation pick in a

resulting sub-window can be approximated by different curves (Gaussian, parabolic). As

presented in Fig. 11, the interpolation of the cross-correlation pick enables sub-pixel

displacement to be determined. Different sub-pixel interpolation methods have been tested

and the most efficient is 2D-Gaussian Interpolation. Global review and comparison of these

methods is well described in Westerweel et al. (1997).

To test the previous algorithms on multi-exposure images, these last ones were

decomposed in sequences of X individual images (X corresponding to the number of

flashes used), each of them being dedicated to one throw. Then, motion estimation methods

False

Im1 & Im2

Test on the pass number

ROIs detection

Throw cut

Global motion calculation

Local motion calculation

True

Velocity matrix

ROI

Determination of three points in the

throw

Calculation of the radius and the centre of

the circle

Determination of the opening

angle

Cutting point

(a) (b)

Fig. 9 a Flow chart of the motion estimation based on a two-step cross-correlation algorithm. b Thealgorithm used to cut the throw in areas

Fig. 10 The left image is taken at the instant t and the right one is taken at the instant t ? Dt. The dottedcircle represents the model circle

Precision Agric (2010) 11:684–702 695

123

Author's personal copy

were used between two successive images among the X images. Since the acquisition time

between each throw was controlled, the evaluation of the granule velocities was directly

correlated to the evaluation of the displacements.

Results and discussion

Overview of MRF and cross-correlation methods

Since the new stroboscopic device is still under development, we tested the different

motion estimation methods on multi-exposure images obtained with the old high speed

imaging system. When comparing results on velocity vector fields obtained from different

image processing methods, Hijazi et al. (2009) concluded that either combining theoretical

granule distribution modelling with MRFs or a two step cross-correlation provide the best

results (Fig. 12).

When examined visually, even if the same velocity vectors are not displayed in Fig. 12,

the estimations using both techniques seem to give the same results. However, the MRF

method is highly time-consuming and the modelling used for the initialisation of the

velocity vector field needs an accurate determination of different spreader parameters such

as the centre of the spinning disc, the length of the vanes (in pixels), and other parameters.

These parameters can create errors on the motion estimation. Furthermore, this technique is

highly influenced by changes in illumination between the images. The cross-correlation

method is thus qualitatively a better solution. Since no reference method to determine the

actual velocities is available, until now validation has been done by comparing the results

with visual evaluation on actual images. This method is laborious and its accuracy depends

on the quality of those images and human precision. Therefore, we propose to

Fig. 11 Principle of the 2D-Gaussian interpolation method

696 Precision Agric (2010) 11:684–702

123

Author's personal copy

quantitatively validate the results of the MRFs method and the cross-correlation method by

using simulated images.

Validation of the results with simulated images

To compare bias error, error maximum and accuracy between MRF and cross-correlation

techniques, simulated images were used. Specific algorithms were developed to simulate

fertiliser granule images similar to real images obtained with a high-speed imaging system.

The simulated images are created in two steps. First, the first throw of granules is

created, then the displacement of that throw is created. The throw is modelled by an arc

and the granules by disks whose amplitude of pixel luminosity has a 2D Gaussian shape.

Granules are placed randomly around the arc, taking into account the friction effect that

causes bigger distances between the grains at the end of the throw.

For the displacement of the throw, the centrifugal effect was taken into consideration by

displacing each grain on an axis originating from the centrifugal centre. In addition to the

shape of the throws, the fertiliser mass flow (i.e. number of particles) is also simulated

based on the quantity of fertiliser generally spread in a field at 7 km/h (around 2 m/s) and a

working width of 24 m (classical working widths are included between 12 and 48 m).

Fertiliser applications between 100 and 500 kg/ha, with an average at 300 kg/ha were used.

450 500 550 600 650 700 750

700

750

800

850

900

950

450 500 550 600 650 700 750

700

750

800

850

900

950

400 450 500 550 600 650 700 750

650

700

750

800

850

900

950

1000

400 450 500 550 600 650 700

650

700

750

800

850

900

950

1000

Calculated vectors using MRFs Calculated vectors using cross-correlation

Fig. 12 Velocity vector fields obtained with MRFs (bottom left) and cross-correlation (bottom right)techniques, between two successive throws (top right and left)

Precision Agric (2010) 11:684–702 697

123

Author's personal copy

The difference of illumination is particularly obvious on the real image of Fig. 13. The

two images resulting from the subtraction, and the sum of the real and simulated images,

clearly show that the simulating algorithm provides simulated images that are highly

correlated to actual images. Evaluating this correlation could be a subject for future

research.

Tables 1 and 2 present examples of comparisons of velocities estimated with the two

motion estimation methods mentioned above. These comparisons are based on simulated

images from two different multi-exposure images (Fig. 14).

Throws are numbered from one to eight with regard to the instant of exposure,

e.g. throw number one is photographed at the first flash and the throw number 8 is

photographed at the eighth flash.

The first column of Tables 1 and 2 presents the following information:

• The number of the successive throws (‘‘T7 and T8’’ and ‘‘T5 and T6’’, respectively).

In the first four throws, the grains are too concentrated and a high number of grains are

occluded in the images. Besides, a large number of grains are still left in the vane.

Consequently, the predicted distribution pattern, if these throws are used, will be

incorrect. Therefore, in order to obtain a sufficient number of granules per throw, the

Fig. 13 Comparison between simulated and real images

Table 1 Comparison of velocities obtained with cross-correlation and MRF, based on simulated images forthrows T7 and T8

Velocity between T7and T8 2.048 ms A;V800; Pl; t = 25 mm

Mean velocitymodulus (pixel)

Bias error(pixel)

Error maximum(pixel)

Standarddeviation

Accuracy90% (pixel)

Cross-correlation Horizontal 76.509 0.064839 0.330551 0.059355 0.13392

Vertical 0.065808 0.277008 0.050016 0.12189

MRFs Horizontal 74.855 2.062399 7.966087 1.805508 4.54770

Vertical 5.172972 11.957868 3.008836 9.03480

698 Precision Agric (2010) 11:684–702

123

Author's personal copy

motion estimation of the displacements is always done from the throw no. 5 to the

throw no. 8.

• The delay between each flash, which indicates the value of the displacement to detect:

for t = 25 mm and a flash delay of 2.048 ms, there is a displacement of 77 pixels (ormm)/image; for t = 45 mm and a flash delay of 2.048 ms, a displacement of 63 pixels(or mm)/image.

• ‘‘A’’ corresponds to the type of fertiliser (Ammonium nitrate).

• ‘‘V800’’ corresponds to the rotational speed of the spinning disc (800 rpm).

• ‘‘Pl’’ and ‘‘Pm’’ correspond to the length of the vanes (325 or 275 mm).

• ‘‘t = 25 and 45 mm’’ correspond to the aperture of the hopper trap and define the

fertiliser mass flow (0.6 ¼) 125 or 1.4 ¼) 290 kg/ha). This mass flow affects the

concentration of the grains in images. With t = 25 mm, the throw no. 8 contains

around 250 grains and with t = 45 mm, it contains around 400 grains.

The accuracy was determined using the Mean Square Error (MSE) between the esti-

mated and the real velocity values. The last column shows the accuracy of the particle

speed: ninety percent of the detected particle speeds have an accuracy inferior to this value.

With the camera and lens located at a height of 1 m, 1 pixel corresponds to 1 mm.

The tables clearly show that the cross-correlation method very precisely determines the

fertiliser granule velocities with an average error of 0.1 pixel or less, and 90% of the

granule velocity with a rate of error less than 0.4 pixel.

The main advantages of the cross-correlation technique can be broken down into the

four following points:

• Only two successive images are needed and the distance between these two images

theoretically makes no difference;

• This is a semi-local motion estimation method and the two-step strategy developed is

ideal for our application and for non-uniform motion;

• It provides sub-pixel accuracy, which allows for a decrease in the resolution of the

camera used. This opens the possibility of using classical high-speed cameras. For

example, a precision of 0.2 pixel can allow division of the resolution of the camera by

Table 2 Comparison of velocities obtained with cross-correlation and MRF, based on simulated images forthrows T5 and T6

Velocity between T5and T6 2.048 ms A;V800; Pm; t = 45 mm

Mean velocitymodulus (pixel)

Bias error(pixel)

Error maximum(pixel)

Standarddeviation

Accuracy90% (pixel)

Cross-correlation Horizontal 62.402 0.085365 0.384418 0.073746 0.17261

Vertical 0.099817 0.330194 0.080768 0.21957

MRFs Horizontal 61.453 1.624881 5.549145 1.399179 3.65780

Vertical 0.800443 3.431636 0.834144 2.34400

T7 T8 T6T5

Fig. 14 The two 1000 9 1000pixel multi-exposure imagesprocessed for t = 25 mm (left)and t = 45 mm (right)

Precision Agric (2010) 11:684–702 699

123

Author's personal copy

5 in each dimension. Moreover, it can allow use of a higher focal length lens, which

will avoid image calibration due to distortion.

• In the next phase of the research, 3D information will be used and an appropriate

stereoscopic device developed. 3D motion is usually determined using both the cross-

correlation method and 3D calibration.

The sub-pixel accuracy obtained with the cross-correlation method is a function of

different image processing parameters: size of the search window and number of the zones

used for the dissociation of a fertiliser throw. Their influence on the accuracy of the

velocity measurement is currently being investigated.

The motion estimation method based on cross-correlation has been used successfully on

fertiliser granule images. The velocity vector obtained is more accurate, in all cases, than

with other methods such as MRF. Additionally, this method does not require precise

control of the luminosity.

From measuring the initial flight conditions of the granules, the cross-correlation

method simultaneously provides both the direction (not presented in this paper) and

velocity of each granule. The combination between these two parameters results in the

trajectory of each granule. The sub-pixel accuracy decreases errors on velocity and

direction determination at ejection, and in turn, increases accuracy of evaluation of fer-

tiliser distribution on the ground. Modelling of the pattern distribution of fertiliser on the

soil, using our results and a simple ballistic flight model, is under investigation.

Another possibility could be to substitute the theoretical model step in MRFs with the

cross-correlation technique, in order to avoid the use of spreader parameters. Research

involving the trajectory information obtained from a ballistic flight model could also be

useful; this information could be used to predict the distribution pattern on the ground and

to compare that pattern with real distributions.

Finally, a mobile low-cost measurement system must be developed. The sub-pixel

accuracy enables a system based on a low-resolution camera sensor (typically lower than

640 9 480 pixels) to be considered. Such a camera associated with the proposed power-

LEDs stroboscope can be considered as the first step for a low-cost embedded measure-

ment solution in a spreading context.

Conclusion

This paper began with a brief summary of previous work related to understanding the

centrifugal spreading process, particularly the determination of the fertiliser granule

parameters at the ejection. Then, the conception of a high-speed imaging device, the use of

image processing based on motion estimation methods, and the improvements to the

imaging system in depth were discussed.

Ways to avoid the variation of the luminance between two successive images were

studied, as errors arose when determining motion estimation using optical flow methods. A

new stroboscopic system is currently being developed based on the assembly of several

power-LEDs. The number required to obtain sufficient and homogeneous illumination at

1 m height (the height where the camera was placed) was determined. In addition, a

hexagonal repartition of LEDs shows the best configuration to provide adequate illumi-

nation for our purposes.

A new motion estimation method based on cross-correlation technique was adapted to

estimate the specific motion of the granules. The results on real images are very satisfying.

700 Precision Agric (2010) 11:684–702

123

Author's personal copy

The use of a two-step estimation was the best way to estimate motion of fertiliser granules.

The sub-pixel accuracy obtained with the cross-correlation method is sufficient to decrease

the resolution of the camera enough to use a classical high-speed camera. This greatly

decreases the cost of this imaging system. This last conclusion is an important one if the

imaging system is to be attached to existing centrifugal spreaders. The next step will be to

test a low-resolution high-speed camera and to estimate the fertiliser velocities.

Acknowledgments The authors thank Richard Martin for his help with the stroboscope concept, andGael Jaton and Alexey Byriukov for the first illumination study. Special thanks go to Miriam Levenson,English-language editor at the Institute for Agricultural and Fisheries Research (ILVO), for her languageassistance.

References

Barron, J. L., & Thacker, N. A. (2005). Tutorial: Computing 2D and 3D optical flow. In Imaging science andbiomedical engineering division (pp 1–12). Manchester: Medical School, University of Manchester.http://www.tina-vision.net/docs/memos/2004-012.pdf. Accessed July 07, 2010.

Bruno, E., & Pellerin, D. (2000). Robust motion estimation using spatial gabor filters. In G. Moncef &K. Paulti (Eds.), Proceeding of the 10th European signal processing conference (EUSIPCO)(pp. 1465–1468). Finland: Tampere. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.2170. Accessed July 07, 2010.

Cointault, F., Paindavoine, M., & Sarrazin, P. (2002). Fast imaging system for particle projection analysis:application to fertilizer centrifugal spreading. Measurement Science and Technology, 13, 1087–1093.

Cointault, F., Sarrazin, P., & Paindavoine, M. (2003). Measurement of fertilizer granules motion on acentrifugal spreader with a fast imaging system. Precision Agriculture, 4, 279–295.

Cointault, F., & Vangeyte, J. (2005). Development of low cost high speed photographic imaging systems tomeasure outlet velocity of fertilizer granules during spreading. In International fertiliser societymeeting, Proceeding 555, London, UK, 14 April. http://www.fertiliser-society.org/Proceedings/US/Prc555.HTM. Accessed July 07, 2010.

Foucaut, J. M., Miliat, B., Perenne, N., & Stanislas, M. (2003). Characterization of different PIV algorithmsusing the EUROPIV synthetics image generator and real images from turbulent boundary layer.In Recent improvements proceedings of the EUROPIV 2 workshop held in Zaragoza, Spain

Fournel, T., Coudert, S., Fournier, C., & Ducottet, C. (2003). Stereoscopic particle image velocimetry usingtelecentric lenses. Measurement Science and Technology, 14(4), 494–499.

Grift, T. E., & Hofstee, J. W. (1997). Measurement of velocity and diameter of individual fertilizer particlesby an optical method. Journal of Agricultural Engineering Research, 66(3), 235–238.

Hijazi, B., Cointault, F., Dubois, J., Villette, S., Vangeyte, J., Yang, F., & Paindavoine, M. (2009). New highspeed image acquisition system and image processing techniques for fertilizer granule trajectorydetermination. In E. J. van Henten, D. Goense & C. Lokhorst (Eds.), Precision agriculture ‘09proceedings of the 7th European conference on precision agriculture (pp. 639–647). Wageningen, TheNetherlands: Wageningen Academic Publishers.

Hijazi, B, Cointault, F., Yang, F., & Paindavoine, M. (2008). High Speed Motion Estimation of FertilizerGranules with Gabor Filters. In H. Kleine & M. Guillen (Eds.), 28th International congress on highspeed imaging and photonics proceedings, SPIE, Canberra, Australia.

Mulligan, D., Bouraoui, F., Grizzetti, B., Aloe, A. & Dusart, J. (2006). An atlas of Pan-European data forinvestigating the fate of agrochemicals in terrestrial ecosystems. http://www.environmental-expert.com/sign_in.asp?vienede=http://www.environmental-expert.com/articleemailformbd_login.asp?cid=27957&codi=26379. Accessed July 07, 2010.

Nillius, P., & Eklundh, J. O. (2002). Fast block matching with normalized cross-correlation using Walshtransforms. Computational vision and active perception laboratory (CVAP). Stockholm, Sweden,Tech. Rep. KHT-NA-P02/11, September 2002.

Traver, V. J., & Pla, F. (2005). Similarity motion estimation and active tracking through spatial-domainprojections on log-polar images. Computer Vision and Image Understanding, 97(2), 209–241.

Vangeyte, J., Baert, J., Scheers, B., Maertens, W., & Sonck, B. (2006). Analyzing particle trajectories:assessing the distribution of the grains in the circumference of the disk. In The 2nd internationalsymposium on centrifugal fertiliser spreading, Montoldre, France (pp. O1–O4).

Precision Agric (2010) 11:684–702 701

123

Author's personal copy

Vangeyte, J. & Sonck, B. (2005). Image analysis of particle trajectories. In B. Tijskens, & H. Ramon (Eds.),Proceedings of the 1st international symposium on centrifugal fertiliser spreading. KULeuven:Leuven, Belgium.

Villette, S., Cointault, F., Piron, E., Chopinet, B., & Paindavoine, M. (2007). A simple imaging system tomeasure velocity and improve the quality of fertilizer spreading in agriculture. Journal of Electronicimaging, 17(3), 1109–1119.

Wesley, E., Snyder, W. E., & Qi, H. (2004). Machine vision. Cambridge, UK: Cambridge University Press.Westerweel, J., Dabiri, D., & Gharib, M. (1997). The effect of a discrete window offset on the accuracy of

cross-correlation analysis of digital PIV recordings. Experiments in Fluids, 23, 20–28.

702 Precision Agric (2010) 11:684–702

123

Author's personal copy