Institut für Nachrichtentechnik

Automobil-Radar

Fax: +49 (40) 42878-2281

Eißendorfer Straße 40

D- 21073 Hamburg

Version 09.02.2006

Signal Processing Strategies for a MultiSensor Pre-Crash ApplicationStephan Müller, Henning Ritter, Hermann Rohling; TU Hamburg-Harburg

Marc-Michael Meinecke, Mark Gonter; Volkswagen AG

Introduction

System Setup

In today's cars a crash is detected by observing the signals of the mi-

cro-mechanical acceleration sensors (so-called crash-sensors) of the

car. If the analysis of the deceleration signal (e.g. by using signal inte-

gration and neural networks) exceeds a defined value a crash is as-

sumed and the safety measures like activating passenger airbags

are triggered. Standard airbags are triggered normally 10ms-30ms

after the first contact with the obstacle and they are fully blown up

after further 30ms. Due to the fact that the collision has already hap-

pened there is only a short time frame available for triggering actua-

tors. In order to improve this situation, the car's surrounding can be ob-

served by sensors like radar or cameras. This gives the opportunity to

get an overview of the situation around the car and therefore detect

certain crash situations even before the car is hitting the obstacle.

The automotive industry spends for the time being a large amount of

effort for developing pre-crash systems. The gain of such systems is

obvious. In case the system can detect a highly risky situation some

milliseconds in advance of a collision, active safety measures (so-

called actuators) can be triggered early and the safety for the car

driver and occupants can be increased.

An additional and very important future application may be the ac-

tive protection of vulnerable road users, like pedestrians or cyclists.

This could increase road safety; however, such system is not feasible

with sensors available today.

To detect a situation of an inevitable accident in time, a sensor ob-

servation of the car environment is needed. The sensor based on its

ability to analyze the situation will make a decision whether an acci-

dent is about to occur and whether some actuators have to be trig-

gered or if the driver can still avoid the accident by starting an eva-

sive maneuver.

Since even non-reversible safety measures are taken into account

the situation analysis and the decision procedure of the pre-crash sys-

tem requires an extremely high reliability and therewith very low false

alarm rates. This is not easy to achieve with today's technology. And

even with extensive testing it is difficult to cover and simulate all the

different kinds of situations which a large number of vehicles

equipped with the system may be subjected to when driven on the

road.

To test and validate the developed pre-crash system on a quantita-

tive basis a research car Audi A8 has been built up which is

equipped with three different sensor systems. A 77 GHz long range

radar with 160 m maximum range (standard ACC sensor), a 24 GHz

radar sensor (narrow band) with 70 m maximum range and a PMD

camera (3D camera) with a maximum range of 40 m are integrated

into the Audi A8. Fig 1 shows the test vehicle with the mounted sen-

sors.

The observation area of the three sensor types is much larger com-

pared to the safety area where the decision (trigger for actuators)

about an unavoidable accident situation must be taken. Within the

limits of the system all targets inside the observation area are sup-

posed to be detected early enough so that a reasonably reliable situ-

ation analysis can be processed. The tracked targets will show a rela-

tively high measurement accuracy which is an advantage for the ac-

cident situation analysis.

Due to the different physical measurement principles each sensor

has its individual measurement characteristics and accuracy as well

as limits. Each sensor has its own individual performance which is in-

vestigated in this paper. But nevertheless a data combination can in-

crease the system performance. The radar sensors have advantages

in terms of being nearly independent of weather conditions and

have a relatively high accuracy in range and radial velocity mea-

surement. The PMD camera has the advantage to measure the tar-

get details like target extension or size in the three dimensions width,

height and distance. With a data combination procedure it is possi-

ble to combine these sensor advantages and improve the situation

analysis in general, measure the target positions more accurate and

finally improve the accident detection procedure.

Fig1: Research car Audi A8 equipped with 24 GHz and 77 GHz radar sensors and PMD camera

Institut für Nachrichtentechnik

Automobil-Radar

Fax: +49 (40) 42878-2281

Eißendorfer Straße 40

D- 21073 Hamburg

Version 09.02.2006

Signal Processing Strategies for a MultiSensor Pre-Crash ApplicationStephan Müller, Henning Ritter, Hermann Rohling; TUHH

Marc-Michael Meinecke, Mark Gonter; Volkswagen

Tested Sensors for Pre-Crash Situations

The 24 GHz radar sensor

The 77 GHz radar sensor

The PMD Camera

Three different types of sensors were investigated during the project. A

24 GHz near field radar sensor, a 77 GHz long range radar sensor

and a PMD camera have been installed in an Audi A8 experimental

vehicle. As shown in Fig 1 the radar sensors are mounted behind the

front bumper and the camera is mounted behind the front shield.

The 24 GHz radar and the PMD camera have a very broad beam

compared to the 77 GHz long range radar sensor. These two sensors

are covering the important area in front of the car, as to be seen in

Fig 2, whereas the 77 GHz radar sensor has a very long beam.

The 24 GHz radar sensor is able to measure the target range and

angle very precisely. Because of its FMCW (Frequency Modulated

Continuous Wave) waveform it can also determine the radial com-

ponent of the target speed.

Since the measurement frequency is

24 GHz the radar is highly independent

on weather conditions and very reli-

able. In addition this sensor uses only a

small bandwidth of about 150 MHz

within the 250 MHz ISM band and

therefore no restrictions of the so-

called “SARA package solution” are ap-

plied. In Table 1 the technical details of the sensor are listed.

The 77 GHz radar sensor from Bosch is a stan-

dard ACC long range sensor. Like the 24 GHz

radar sensor it is capable of measuring the

target position and speed. In Table 2 the

technical specifications of the sensor are

given.

The PMD (Photonic Mixing De-

vice) camera shown in Fig 5 pro-

vides a gray scaled image like a

mono camera. In addition in in-

tensity modulated infrared light

from two external light sources is

used to measure the range and

angle for every camera pixel. By combining the two measurement

principles it is possible to measure not only the target's position but

also its extension in width and height. In Table 3 the technical details

for the camera are shown.

Fig 3: 24 GHz radar sensor

Fig 2: Sensor field of view

Fig 4: 77 GHz radar sensor

Fig 5: PMD camera

Observation area 50° in azimuth

Angular accuracy 2°

Maximum range 70m

Range accuracy 0.20m

Velocity range > ±22m/s

Velocity accuracy 0.3m/s

Table 1: 24GHz radar sensor parameter

Observation area ± 8° in azimuth

Angular accuracy ±0.1° 0.4°

Maximum range 150m

Range accuracy 0.50m

Range resolution 2m

Velocity range -60m/s - +20m/s

Table 2: 77GHz radar sensor parameter

Observation area 55° in azimuth

Resolution H x V 64 x 16 Pixel

Angular accuracy 2°

Maximum range 40m

Range accuracy 0.02m @ d=0m

0.40m @ d=25m

Table 3: PMD camera parameter

Institut für Nachrichtentechnik

Automobil-Radar

Fax: +49 (40) 42878-2281

Eißendorfer Straße 40

D- 21073 Hamburg

Version 09.02.2006

Signal Processing Strategies for a MultiSensor Pre-Crash ApplicationStephan Müller, Henning Ritter, Hermann Rohling; TUHH

Marc-Michael Meinecke, Mark Gonter; Volkswagen

Pre-Crash Theory

Stationary targets

Moving Targets

Currently there are a lot of different safety measures in discussion;

starting with seatbelt pre-tensioners, up to active hood concepts

and innovative airbag technologies. From a deployment point of

view they can be divided into two large categories: First reversible

and second non-reversible actuators. Since the developed system

is intended to increase safety special care has to be taken to se-

cure deployment reliability.

To detect an upcoming crash it has to be known if the car is still able

to avoid the collision either by braking or by a steering maneuver or

not. Since for relevant speeds and small target sizes the braking ma-

neuver always consumes more time than the steering maneuver

here only the steering maneuver is considered. In case the collision

cannot be avoided the reduction of crashworthiness is of high inter-

est as well.

If the driver wants to avoid the collision with a stationary target he

can drive to the left or to the right hand side, as shown in Fig 6. Nev-

ertheless a certain area will be overdriven by the car regardless of

the driver's maneuver.

This area is called the crash area and is marked in red in Fig 6.

Under the assumption of a single track model the crash area is only

dependent on the speed of the car and the static friction between

the tires and the ground which includes also the condition of the

road (dry, wet, icy, etc.). This assumption gives a good approximation

of the vehicle behavior, while the complexity of calculations is still rea-

sonable.

Since any obstacle inside the crash area will be hit by the car (crash

is unavoidable) it is now possible to identify a pre-crash situation

under the assumption that the situation around the car is known.

When the pre-crash situation is detected the corresponding time to

crash (Time to collision) at this moment can be estimated by equa-

tion (1).

The model of the safety area in the chapter above is only valid for sta-

tionary targets and has to be extended for trigger decisions with mov-

ing objects. The prediction that an object inside the safety area will

definitely be hit by the car is not sufficient because the object might

move away from this zone in the time the own car is reaching the de-

picting point.

Therefore the trigger algorithms for moving targets not only have to

predict the positions of objects but also the superposition in time do-

main. This extension of the theory by the time domain can be con-

sidered from two different points of view:

A moving target (regardless if extended in x and y or a point target)

has to be at certain point of time on a finite area if the accelera-

tion is limited. This area can be calculated by using the equation of

motion for a given maximal acceleration (positive and negative).

For an extended (in x and y) and moving target it is possible to cal-

culate the period of time in which this object is placed over a cer-

tain point in the x-y-plane. This time period gets smaller the higher

the assumed dynamics of the considered object.

One major difference arises in the data processing for both points of

view. In the first case a trigger decision would mean to test if the

whole set of whereabouts for the object is inside the safety area. If

this is the case a crash is inescapable. This test is admittedly of a high

calculation complexity and therefore not feasible for multi target situ-

ations. The second case is to be evaluated more easily.

At first the movements for the own car as well as for the adversarial

object have to be predicted and possible intersection points are cal-

culated which could end up in an accident. A second step is to cal-

culate the time periods in which the own car is placed over the set of

intersection points and the certain points of time when the detected

object is to be there. If an overlap in time of the position of the own

car and the object's position occurs a crash will definitely happen

and can be predicted by this method.

In either case the knowledge of the velocity components in direction

of x and y is indispensable for the data processing algorithms as they

are the basis for the prediction of the movements. A radial velocity di-

rectly to be measured by a radar sensor is not sufficient, but the ve-

locity in direction of x and y can be estimated by tracking algorithms.

�

�

Fig 6: Geometrical view of the maneuver crash area

(1)target distance

ttcrelative targetspeed

=

Institut für Nachrichtentechnik

Automobil-Radar

Fax: +49 (40) 42878-2281

Eißendorfer Straße 40

D- 21073 Hamburg

Version 09.02.2006

Signal Processing Strategies for a MultiSensor Pre-Crash ApplicationStephan Müller, Henning Ritter, Hermann Rohling; TUHH

Marc-Michael Meinecke, Mark Gonter; Volkswagen

Performance of single sensor and combined

sensor set-up

Safety on public roadsConclusion

Each individual sensor provides a certain performance in terms of

correct deployment probability and false alarm rate. In case that

more then one sensor is available in a car the idea comes up to

combine this entire individual trigger signal to increase reliability of

the predictions. Both concepts are analyzed in this chapter.

For the acceptance of the system by the customer an extremely low

false alarm rate depending on the concrete actuator is required. It

varies from approximately 10-6 /km for reversible actuators to approx-

imately 10-12 /km for non-reversible actuators. These numbers can

be derived from the acceptance by the driver as well as from prod-

uct liability constraints of the manufacturer.

To maintain this false alarm probability for the reliable deployment of

non-reversible actuators the detected pre-crash situations from each

sensor are combined to a single alarm trigger. The architecture of the

deployment algorithm is as follows. All targets seen by the individual

sensors are tested if they are present in the crash zone. If so, a trigger

is send by the risk assessment/ deployment algorithm of the individual

sensor. All the outputs of the individual deployment signals are com-

bined in a statistical manner. The combined deployment signal is

connected to the (virtual) actuator in the experimental car.

The first approach in combining the individual trigger signals is the

conjunction. In this case the alarm signal is only generated if all sen-

sors have detected a pre-crash situation. The combined false alarm

probability of the system is also called the false decision probability

because it describes if the situation is interpreted correctly. Addition-

ally this also leads to a reduced detection probability.

For testing the performance of the sensor and the trigger algorithms

selected standard road situations were taken. In one of these situa-

tions the test car was driving straight against an obstacle. Taking this

as an example the system performance shall be shown. To test if the

algorithms are detecting a crash, the car is hitting the obstacle. With

this test also the time to collision at the moment when the alarm is is-

sued can be measured. To test if the algorithms are not triggering an

alarm in near crash situations now the car is making an evasive ma-

noeuvre such that it is not hitting the obstacle but passing it at a dis-

tance of a few centimetres.

These numbers should be handled with care since the data base is

very small (only 33 trials in this example) and an artificial situation is

considered.

Another important figure is the time to collision at the moment when

a pre-crash situation has been detected. The time to collision gives

the maximum time that an actuator has to react to the detected situ-

ation. In Fig 9 the achieved TTCs for the front crash situation are

shown as a histogram for the single sensor and the combined alarm.

In this diagram it was assumed that the given actuator has an activa-

tion time of 100 ms.

As it can be seen in Fig 9 in the most situations the deployment time

abides with the requirement of 100 ms. There are only two exceptions

in which a lower deployment time occurred. This was caused by ex-

treme driving manoeuvres on a proving ground which is not typical

for an ordinary driver. In these situations the target entered the safety

area from the side and the therefore the time to collision was not suf-

ficient for the actuators described above. Nevertheless the majority

of the alarm triggers have been sent with sufficient time.

It can be seen that under “typical” public road situations the system

still has a level of false alarm rate which is too high in the context of a

system intended to increase the safety. Nevertheless it is capable to

detect a pre-crash in standard situations reliable. Furthermore the ma-

jority of the alarm triggers were sent at the required time and there-

fore the safety actuators could have reacted in a sufficient way.

The future work will emphasis on the reduction of the false alarm rate

by optimizing the single sensor tracking and analysing the combina-

tion of the single sensor alarms to improve the robustness of the

alarm decision.

Fig 9: Achieved TTCs in a front crash situation for the combined alarm.