Laser-to-Radar Sensing Redundancy for Resilient Perception inAdverse Environmental Conditions

Marcos P. Gerardo Castro and Thierry PeynotAustralian Centre for Field Robotics

The University of Sydney, NSW 2006, Australia{m.castro@acfr.usyd.edu.au, tpeynot@acfr.usyd.edu.au}

Abstract

This paper presents an approach to promotethe integrity of perception systems for outdoorunmanned ground vehicles (UGV) operating inchallenging environmental conditions (presenceof dust or smoke). The proposed techniqueautomatically evaluates the consistency of thedata provided by two sensing modalities: a 2Dlaser range finder and a millimetre-wave radar,allowing for perceptual failure mitigation. Ex-perimental results, obtained with a UGV oper-ating in rural environments, and an error anal-ysis validate the approach.

1 Introduction

This work focuses on the development of reliable per-ception systems for outdoor unmanned ground vehicles(UGV), in particular in adverse environmental condi-tions (e.g. presence of airborne dust, smoke, thick fogor rain). The problem of modelling and mitigating sys-tematic errors in perception models, such as sensor mea-surement errors or sensor misalignment, has been ex-tensively studied by robotics researchers and thoroughsolutions have been proposed (e.g. [Underwood et al.,2010] for range sensors such as laser range finders (LRF)or radars). However, the main remaining challengelies in interpretation errors. These errors can be ran-dom, are difficult to predict, and can often be orders ofmagnitude larger than the systematic errors mentionedabove. Arguably, a reliable perception system shoulduse different sensor modalities [C. Thorpe et al., 2001;C. Urmson et al., 2008], especially for outdoor opera-tions. As these modalities sense the environment us-ing different physical processes, they also respond dif-ferently to environmental conditions. For example, amm-wave radar has excellent properties of penetrationthrough heavy dust and smoke in contrast to a laser, andan infrared camera can see through smoke, contrary toa visual camera. Therefore, a more reliable perception

system can be obtained by intelligently combining thedata provided by such different sensing modalities [A.Kelly et al., 2006].

While the fusion of observations made by a laser anda radar in clear conditions, e.g. without the presence ofchallenging conditions such as dust or smoke, is straight-forward when a good sensor error model is available [Un-derwood et al., 2010], it relies on the assumption (or pre-condition) that the two sensors actually detect the sametargets in the environment. If, for example, a LRF doesnot see through a heavy dust cloud while a radar does,this assumption does not hold any more. Therefore, insuch a situation data fusion should not be executed, atleast not in its traditional form. Consequently, to be ro-bust to adverse environmental conditions, the perceptionsystem should have the ability to verify this assumptionof data consistency prior to fusion. Another advantageto this ability is that the data provided by a LRF can beconveniently filtered, separating points returned becauseof dust or smoke that a radar would hardly be affectedby. The radar could then ensure that detection of actualobstacles and terrain modelling remains operational, al-beit less accurate (since the radar accuracy is typicallynot comparable to the laser’s, as described in Table 1).

Recently, laser range finders capable of returning mul-tiple echoes for each emitted pulse have been introducedcommercially (e.g. the Sick LMS5xx series [SICK Inc.,2012b] or LD-MRS [SICK Inc., 2012a] for automotiveapplications). Although this ability has made such lasersensors more robust to adverse environmental conditions(e.g. compared to the LMS2xx series), they cannot pro-vide a full solution of the problem. Because of the levelof attenuation of the laser signal, a mm-wave radar willstill be able to penetrate better through obscurants suchas heavy dust that would eventually block laser signals[Brooker, 2009; Ryde and Hillier, 2009]. Moreover, ananalysis of pre-conditions for laser-radar fusion and forseparating dense objects from such obscurants would stillbe required to obtain a resilient navigation of the UGV.

The idea of using laser-radar data comparison for per-

Proceedings of Australasian Conference on Robotics and Automation, 3-5 Dec 2012, Victoria University of Wellington, New Zealand.

Figure 1: The Argo UGV and its sensors.

ception in the presence of airborne dust was introducedin previous work by the authors, using the UGV shownin Fig. 1. However, if [Peynot et al., 2009] delivered aproof of concept with promising initial results, this workhad several limitations: 1) the two sensors were consid-ered perfectly aligned, allowing for a direct comparisonof the measured ranges they provide for each bearing an-gle, 2) the laser-radar data comparison was specificallydesigned and used as an airborne dust filter, 3) this fil-ter was demonstrated on only one particular dataset. Inpractice, not only is the alignment assumption a strongconstraint on the system, but such alignment is prac-tically nearly impossible unless the two sensors use thesame mirror and scanning mechanism. The techniqueproposed in this paper does not require that the sensorsare perfectly aligned, instead it uses a 6-DOF calibra-tion allowing for the correction of the mis-alignment ofthe sensors. The comparison of the data can then berealised in a coordinate frame related to the body of thevehicle (instead of one of the sensor frames as in [Peynotet al., 2009]), accounting for the extrinsic calibration ofthe sensors. In this paper we also exploit more informa-tion from the spectrum provided by the radar, allowingfor a closer comparison between the two types of data.Finally, if the technique can also be used as a dust fil-ter, it is not designed as such specifically, so that othercauses of inconsistencies can be detected as well. Someof these causes will be discussed below.

The paper is organised as follows. In Sec. 2, we discussthe methodology to perform the laser-radar consistencytest. Sec. 3 presents an experimental study to charac-terise the laser-radar distance. In Sec. 4, we describeresults measuring the consistency test in scenarios thepresence of airborne dust, smoke or none of the above.Finally, in Sec. 5, an error analysis is presented to eval-uate the proposed method.

2 Laser-Radar Redundancy

In order to compensate for the mis-alignment of the laserand radar sensors, we need to perform an extrinsic cal-ibration of the relative transformation between the twosensors (or the transformation between each sensor anda frame linked to the body of the vehicle, which we willcall the body frame). In this paper we use the calibrationtechnique described in [Underwood et al., 2010], whichcan achieve a joint extrinsic calibration of multiple exte-roceptive range-based sensors such as lasers and radar.Since the configurations of the sensors are different, onlya (common) part of a synchronised pair of laser-radarscans contain points that can be considered consistent1.This part can be seen as a “common footprint” (or “foot-print overlap”) of the two scans and can be convenientlyexpressed as a range of bearing angles for each type ofscan. Hereafter, all comparison of laser and radar pointsis made within this common part of the scans. Anotherimportant thing to consider during this comparison isthe range resolution of the two sensors. As described inTable 1, radar resolution is much bigger than the laserresolution.

Table 1: Range Sensor Specifications

Sensor Maximum Range Horizontal Angular Scanning(model) range resolution 2 FOV resolution rate

Horizontal Laser(Sick LMS291) 80 m 0.01 m 180◦ 0.25◦ ≈18 Hz

Radar(Custom built

at ACFR) 40 m 0.2 m 360◦ ≈1.90◦ ≈3 Hz

The rest of the process can happen systematically on-line. Sec. 2.1 describes how target data points are ex-tracted from the radar raw data (i.e. noise removal).Then, Sec. 2.2 shows how radar and laser points are ef-fectively compared after their transformation into thebody frame.

For each bearing angle the radar provides an FFT(Fast Fourier Transform) spectrum. Using the “radarequation” [Brooker, 2009] this spectrum can be mappedto a function of intensity vs. range. Most robotics ap-plications only use the highest peak of that spectrumas a range value provided by the radar (such as in ourprior work in [Peynot et al., 2009]). However, this leadsto the loss of a significant amount of useful informationcontained in the rest of the spectrum. As an example,[Reina et al., 2011] exploited the shape of this spectrumto estimate a model of the ground. The resulting ground

1The adjective consistent will be used to refer to the localagreement between laser and radar observations.

2Note that radar and laser manufacturers use a differentdefinition for resolution. The radar precision when observinga flat plate actually approaches that of the laser.

Proceedings of Australasian Conference on Robotics and Automation, 3-5 Dec 2012, Victoria University of Wellington, New Zealand.

(a) Radar Spectrum and Laser scan

(b) Laser scan projected in the Camera frame

Figure 2: (a) Radar Spectrum, coloured by intensities fromblack to white.The corresponding laser points are showed ingreen. (b) Laser scan (in cyan) projected on a visual imageon the same area, taken from the platform.

estimation was significantly more accurate than when us-ing the highest peak of the spectrum only. However, thisparticular technique can only be used if a model of thespectrum profile obtained for a given target (such as aroughly flat piece of ground) is known a priori.

2.1 The Radar DataIn order to make a “fair” comparison of the radar pointswith observed laser points, in this paper we extract otherpeaks (local maxima) from the spectrum, in addition tothe highest peaks (the global maxima), see Fig. 2(a).This will provide us with a better resolution in the dis-crimination of laser-radar data. First, for each radarbearing angle, all intensity peaks above a threshold of in-tensity are extracted from the radar spectrum (note thatthis includes the highest peak). This threshold is definedin order to minimise the radar noise. Then, given that:a) the laser provides much more accurate data than theradar, b) we know that generally both sensors detect thesame targets in clear conditions, c) we have an accuratecalibration of the sensors and a very accurate localisa-

tion on our robot, we used the laser data as a referencein large datasets of a rural static environment to deter-mine a relevant criteria for an automatic extraction ofthe peaks from the noise in the radar data.

For our radar, extracting peaks that have an inten-sity above 55% of the intensity of the highest peak wasfound to be appropriate. Let us call these peaks pR.For each pR, we find its closest laser point by comput-ing the minimum 3D Euclidean distance to neighbouringlaser points. This distance can be called dR. Similarly,DR is defined as the distance between the radar highestpeak and its closest laser point. A radar peak pR is thenconsidered as a candidate peak if dR ≤ DR.

The existence of a laser point at close proximity indi-cates that most likely this candidate radar peak does cor-respond to a return from an actual target. Besides, thisalso indicates that the extracted candidate peak will con-tribute to a more accurate perception of this target thanthe highest peak alone. The resulting radar candidatepeaks are used to perform a consistency test betweenthe two sensing modalities. This process is described inthe next section.

2.2 Laser-Radar Comparison (ConsistencyTest)

The actual comparison between laser points and can-didate radar peaks relies on the computation of the 3DEuclidean distance between each laser point and the clos-est radar peak found in the synchronised scan, whichwill be called the laser-radar distance. A model of thelaser-radar distance based on 3D distance comparison(which will be described below) between laser and radarpoints was used to decide whether the laser and radarare observing the same target. The main reasons for notobserving the same target (i.e. laser-radar measurementdiscrepancy) are the following:

• the laser actually detects dust, smoke or rain parti-cles that the radar waves penetrate through,

• the perception is inconsistent because of the mate-rial the target is made of (e.g. the radar may detectthe presence of a window that the laser sees throughand therefore does not detect),

• the relative extrinsic calibration between the laserand the radar is wrong,

• the echo returned by the sensor is the result ofa multi-path effect (see [Brooker, 2009; Ryde andHillier, 2009]).

To determine an appropriate threshold on the 3D dis-tance between comparable laser and radar points, weused a dataset in clear conditions in a rural environment(see Fig. 3), limiting the risks of multi-path or distinctreaction of the radar and the laser to particular materi-als. Since in these conditions a close match should always

Proceedings of Australasian Conference on Robotics and Automation, 3-5 Dec 2012, Victoria University of Wellington, New Zealand.

(a) Laser data

(b) Radar data

Figure 3: View from the top of the scene observed in clearconditions by the four lasers (a) and the radar (b) on theArgo UGV. Points are coloured by elevation. We can see theposts of a fence at the bottom and a shed on the left of (a).The area is about 56× 55 m2.

be found, the dataset (containing about 1.7 million laserpoints) could be used as a reference.

Fig. 4 shows the number of inconsistent points for avarying value of distance threshold δ (i.e. number oflaser points for which the closest radar peak was at adistance superior to δ). A distance threshold of δ = 0.8mwas found to be appropriate. With this threshold in thestatic environment used as reference, only about 0.5% ofthe points were inconsistent.

Section 3 and 4 show an experimental study to char-acterise the laser-radar distance and different examplesof application of the laser-radar comparison.

3 Experimental Setup

The experiments were conducted with the Argo UGV, an8-wheel skid-steering platform (see Fig. 1) equipped witha navigation system composed of a Novatel SPAN (Syn-chronised Position Attitude & Navigation) System and aHoneywell Inertial Measurement Unit. This unit usuallyprovides a 2-cm accuracy localisation, with a constantupdate of the estimated uncertainties on this solution.

The following exteroceptive sensors were mounted onthe vehicle (Fig. 1):

Figure 4: Percentage of inconsistent points vs. threshold onthe laser-radar distance (in metres). A 0.5% error was foundwith a threshold at 0.8m (red cross).

• 4 Sick LMS291/221 laser range scanners, with 180◦

field of view (FOV), 0.25◦ angular resolution, and arange resolution of 0.01m.

• a 94GHz Frequency Modulated Continuous Wave(FMCW) Radar, custom built at ACFR for envi-ronment imaging, with 360◦ FOV, 2◦ angular reso-lution and a range resolution of 0.2m,

• a visual camera and an infrared camera.The Laser indicated in Fig. 1 was only roughly alignedwith the Radar to have a similar perspective of the envi-ronment, therefore this laser was chosen to provide thedata to be compared with the radar data 3. Fig. 5 showsan example of scans provided by these two sensors.

The experiments were conducted with the MarulanDatasets described in [Peynot et al., 2010]. We usedvarious datasets with the vehicle driven around two dif-ferent areas. Each dataset featured the presence of air-borne dust (Fig. 8), smoke (Fig. 9), or none of the above(i.e. clear conditions). The environment was not knownby the vehicle a priori.

4 Results

In these experiments, synchronised pairs of laser andradar scans were compared to separate consistent andinconsistent points. In practice, since the laser scan-ner has a higher scanning rate than the radar scanner(see Table 1), for each laser scan the closest radar dataavailable in time was used for the comparison and theconsistency check.

Fig. 8 shows an experiment realised in the same areaas in Fig. 3 but with presence of heavy airborne dust.We can see that most dust points in the laser data havebeen well cleaned out from the dataset, after being foundinconsistent with the radar data.

3Recall that only a rough physical alignment is sufficient,as mentioned earlier, as long as an extrinsic calibration be-tween the two sensors is available

Proceedings of Australasian Conference on Robotics and Automation, 3-5 Dec 2012, Victoria University of Wellington, New Zealand.

Figure 5: Example of laser and radar scans displayed as range(m) vs. bearing angle (degree). Red points are laser returnswhile blue points are radar peaks (the highest peaks for eachbearing angle are shown in dark blue). Note the laser returnsdue to dust at shorter range, which are clearly inconsistentwith the radar measurements.

Figure 6: Experiments with adverse environmental condi-tions: presence of airborne dust.

However, some dust points returned by the laser haveremained, as they were too close to the ground, whichwas still seen by the radar, to be called inconsistent.

Fig. 9 shows another experiment, conducted in a dif-ferent area (a more natural and unstructured environ-ment with surrounding trees), with presence of smoke.It shows how smoke also significantly affects the laserdata and how the consistency test with the radar dataallows for an effective separation of the smoke cloud.

5 Error Analysis

This section shows an analysis of results and errorsobtained from the Laser-to-Radar consistency test de-scribed previously. In order to compare the errors inthe consistency test we used static datasets where theplatform and the objects detected are not moving (see[Peynot et al., 2010]). A reference scan correspondingto data in clear conditions was compared against suc-cessive laser points. The inconsistencies found with this

Figure 7: Sample visual Image from the platform perspective,acquired during the experiments shown in Fig. 8.

comparison were used as ground truth data. Fig. 10(b)shows a comparison between the reference scan (in blue)and a scan taken when dust is present. Note that dust isentering the scene from left to right, which is illustratedby the red points representing inconsistencies with thereference scan. The corresponding visual image is shownin Fig. 10(a).

One important aspect to consider for error measure-ments is the number of points classified as inconsistentfor each laser scan (en), which can tell us the concentra-tion of dust/smoke found. Fig. 11 shows the number oflaser points labelled as inconsistent (ene) as estimatedby the Laser-to-Radar consistency test, compared withthe number of inconsistent points obtained based on theground truth data in clear conditions (enr). Both sce-narios started in clear conditions. Approximately at scannumber 200 smoke (resp. dust) was released. Note thatthe estimation (ene) follows a very similar pattern com-pared with the ground truth (enr). Nevertheless, sincewe used a defined threshold, as explained in Sec. 2.1,differences with the ground truth inconsistency test areexpected to be found, specially when dust/smoke parti-cles are close to obstacles.

We computed the ratio λn of the estimated number ofinconsistent points ene to the ground truth enr:

λn =Σene

Σenr(1)

We obtained λn = 89% for the dust scenario andλn = 84% for the smoke scenario. In addition to en,we also computed the number of points classified as con-sistent by the proposed approach: ane. We defined falsepositives (fp) as the number of laser points found incon-sistent by the test but not by the ground truth. Sim-ilarly, we defined false negatives (fn) as the number ofpoints found to be consistent by the test but not by theground truth. The resulting precision rate of the incon-sistency test (α) was 97% for the dust scenario and 95%

Proceedings of Australasian Conference on Robotics and Automation, 3-5 Dec 2012, Victoria University of Wellington, New Zealand.

(a) All Laser points, bird’seye view.

(b) Without the inconsistentpoints

(c) Side view. Top: all points. Bottom: consistent pointsonly.

Figure 8: Experiment with heavy airborne dust (see Fig. 6).Points are coloured by elevation. The laser points found to beconsistent were coloured from green to red, while inconsistentpoints were coloured from yellow to white. The blue lineshows the path followed by the platform while collecting thisdataset.

(a) All Laser points. (b) Consistent points only.

Figure 9: Experiment with smoke, bird’s eye view. Points arecoloured by elevation. The laser points found to be consistentwere coloured from green to red, while inconsistent pointswere coloured from yellow to white.

for the smoke.α =

Σene − Σfp

Σene(2)

The accuracy rate of the inconsistency test (β) was 88%and 84% for dust and smoke respectively.

β =Σene − Σfp + Σane − Σfn

Σene + Σane(3)

(a) Image from the platform perspective.

−30 −20 −10 0 10 20 30 402

4

6

8

10

12

14

Angle (degrees)

Range

(m)

Laser Reference Scan (Dataset 05)

(b) Laser scan.

Figure 10: (a)Visual image of the scene. Note the presenceof some light dust on the left hand side. (b) The blue line isthe reference laser scan taken in clear conditions. In dottedred, the current laser scan, taken when dust started to passin front of the platform (as image above). Points in red areinconsistent with the ground truth, whilst points in green areconsistent.

Similarly to λn, β is expected to increase in cases wheremost of the inconsistencies are found in the proximitiesof obstacles or ground.

6 Discussion

The method presented in this paper enables to main-tain the safe operation of a UGV in the presence of ad-verse environmental elements such as airborne dust orsmoke, which are strong obscurants for common roboticsensing modalities such as a laser or a visual camera.When dust or smoke are present and block the laser per-ception, the UGV may still go through the obscurantcloud, with the radar allowing for persistent obstacle de-tection. On the other hand, when no obscurant cloudis present, laser perception will be preferred since it is

Proceedings of Australasian Conference on Robotics and Automation, 3-5 Dec 2012, Victoria University of Wellington, New Zealand.

0 100 200 300 400 500 600 700 800 900 10000

50

100

150

200

250

300

Number of inconsistencies: Dust

scan number

Num

ber

ofin

consi

sten

cies

Laser inconsistencies (ene)

Ground truth (enr)

(a) Inconsistent points en (presence of dust).

0 100 200 300 400 500 600 700 800 900 10000

20

40

60

80

100

120

140

160

180

200

Number of inconsistencies: Smoke

scan number

Num

ber

ofin

consi

sten

cies

Laser inconsistencies (ene)

Ground truth (enr)

(b) Inconsistent points en (presence of smoke).

Figure 11: In red, number of laser points labelled as incon-sistent (ene) for each laser scan in the presence of dust (a)and smoke (b) respectively. In blue, number of inconsistentpoints compared with the ground truth data in clear condi-tions (enr).

more accurate compared with radar perception. In theexperiments presented in this paper we have observedthat some dust/smoke points may not be labelled as in-consistent when they are too close to dense obstacles, astheir discrimination is limited by the resolution and thenoise of the radar data.

Another situation that this method may not be able toidentify is when airborne dust or smoke particles are de-tected by the laser in the immediate proximity of radarreturns due to multi-path effect. In such situation thesystem will consider these radar returns as a confirma-tion that the target detected by the laser is in fact a denseobject (therefore a potential obstacle for the UGV). Toovercome this situation another sensing modality suchas visual or infrared cameras could be used.

The proposed method relies on the availability of anaccurate exteroceptive calibration between the laser andthe radar. If the calibration is jeopardised during a mis-

sion of the UGV (for example one of the sensors is ac-cidentally displaced), the consistency test might rejecta large part of the laser data even in clear conditions.Consequently, the UGV would have to rely entirely andsystematically on the radar data (which is typically lessaccurate). However, such situations could be recognisedover time since the inconsistency between the laser andradar data would then be very stable and geometricallyconstant. This could let the system distinguish this casefrom the presence of dust or smoke for example. A sensormodel that accounts for uncertainties will be introducedin future work. Uncertainties in the comparison test willalso be accounted for.

Acknowledgements

This work was supported in part by the Australian Cen-tre for Field Robotics (ACFR) and the NSW State Gov-ernment. This material is based on research sponsoredby the Air Force Research Laboratory, under agreementnumber FA2386-10-1-4153. The U.S. Government is au-thorized to reproduce and distribute reprints for Govern-mental purposes notwithstanding any copyright notationthereon.

References[A. Kelly et al., 2006] A. Kelly et al. Toward reliable

o road autonomous vehicles operating in challengingenvironments. International Journal of Robotics Re-search, 25:449–483, May/June 2006.

[Brooker, 2009] Graham Brooker. Sensors for Rangingand Imaging. SciTech Publishing, Inc., 2009.

[C. Thorpe et al., 2001] C. Thorpe et al. Dependableperception for robots. In Proceedings of InternationalAdvanced Robotics Programme IEEE, Seoul, Korea,May 2001. Robotics and Automation Society.

[C. Urmson et al., 2008] C. Urmson et al. Autonomousdriving in urban environments: Boss and the urbanchallenge. Journal of Field Robotics, 25(8):425–466,2008.

[Peynot et al., 2009] T. Peynot, J. Underwood, andS. Scheding. Towards reliable perception for un-manned ground vehicles in challenging conditions. InIEEE/RSJ Int. Conf. on Intelligent Robots and Sys-tems (IROS), 2009.

[Peynot et al., 2010] T. Peynot, S. Scheding, andS. Terho. The Marulan Data Sets: Multi-SensorPerception in Natural Environment with Challeng-ing Conditions. International Journal of Robotics Re-search, 29(13):1602–1607, November 2010.

[Reina et al., 2011] G. Reina, J. Underwood,G. Brooker, and H. Durrant-Whyte. Radar-based per-ception for autonomous outdoor vehicles. Journal of

Proceedings of Australasian Conference on Robotics and Automation, 3-5 Dec 2012, Victoria University of Wellington, New Zealand.

Field Robotics, 28(6):894–913, November/December2011.

[Ryde and Hillier, 2009] Julian Ryde and Nick Hillier.Performance of laser and radar ranging devices inadverse environmental conditions. Journal of FieldRobotics, 26(9):712–727, September 2009.

[SICK Inc., 2012a] SICK Inc. SICK LD-MRS LaserMeasurement Technology. https://www.mysick.com/partnerPortal/ProductCatalog/DataSheet.aspx?ProductID=34057#, 2012.

[SICK Inc., 2012b] SICK Inc. SICK LMS5xx LaserMeasurement Technology. https://www.mysick.com/partnerPortal/ProductCatalog/DataSheet.aspx?ProductID=75420#, 2012.

[Underwood et al., 2010] J. P. Underwood, A. Hill,T. Peynot, and S. J. Scheding. Error modeling andcalibration of exteroceptive sensors for accurate map-ping applications. Journal of Field Robotics, SpecialIssue: Three-Dimensional Mapping, Part 3, 27(1):2–20, January/February 2010.

Proceedings of Australasian Conference on Robotics and Automation, 3-5 Dec 2012, Victoria University of Wellington, New Zealand.