3D Imager

AMAA 2006 [email protected]

Three-Dimensional CMOS Image Sensor

for Pedestrian Protection and Collision

Mitigation

Peter Mengel Siemens AG

Co - Authors:

L. Listl, B. König, C. Toepfer, M. Pellkofer, Siemens AG

W. Brockherde, B. Hosticka, O. Elkhalili, O. Schrey, W. Ulfig, Fraunhofer IMS


Content

• Introduction and motivation

• 3D sensor technology

• Road safety applications

• Car evaluation of first prototype

• Conclusion & outlook


Environmental Sensing Overview

LDW

Night Vision

Lane Keep

Assistant

ACCParking

Aid

Stop

& Go

Pre- Crash

Parking

Aid

Back-up

Aid

3) Far Distance Range

! Radar (77/79 GHz)

! Lidar

2) Medium Distance Range

! Video

1) Near Distance Range

! Radar (24 GHz)

! Video

! 3D-Kamera

Blind Spot Detection

Lane Change Assistant

Blind Spot Detection

Lane Change Assistant

Intersection

Intersection


TOF Principle

Multiple Double Short Time Integration (MDSI)

Transmitted

Laserpulse

Reflected

Laserpulse

Integrator

T0 T1

!!"

#$$%

&'=

2

11

2 U

UT

vd

PW

C

U1

T2

U2

U

3D-Scene

Trigger

Laserpulse

CMOS SensorWith High

Speed Shutter

d

d

VC = Velocity of light

TO = Time of Flight

TPw= Laser Pulse Width


• Available technology at start of PReVENT

• Sensor used for first prototype evaluation

64 x 4 Pixel ToF Line Sensor

3D CMOS Image Sensor Evolution


3D CMOS Image Sensor Evolution

New ToF 64 x 8 Pixel ArrayPixel pitch:

130!m!300!m

Chip size:

10.58 ! 9.4mm_

Sensitive area:

8.39 x 2.40mm_

Process:

IMS 0.5!m CMOS


ToF 64 x 8 Pixel Array

Key Specification Data

– Pixel count/size 64x8 = 512 / 130µm x 300µm

– Pixel principle photo diode, single buffer type

– NEP (= smallest 2,3 W/m2

detectable signal)

– Minimum shutter 30 ns

– Dynamic range 75 dB (distance & reflectance range)

– Image acquisition 50…200 fps (1…128 pulses/frame)

non-destructive readout

– Background light suppr. > 40dB

– Operating wavelength 850….910nm

– Chip size < 100mm2 (IMS 0.5µm CMOS process)


Microsystems Architecture

Sensor BoardSensor Board

ADCADC

3D-

CMOS

sensor

3D-

CMOS

sensorFPGAFPGA

FlashFlash RAMRAM

•Sensor Control

64 x 8 Pixel Array

•Data Acquisition

•Distance Calculation

•Application Software

Interface

Board

Interface

Board

RS-232RS-232

LAN 100LAN 100

User I/OUser I/O

Laser ModuleLaser Module

Laser

Source

Laser

Source

DC/DC-

Converter(not included into

camera housing)

DC/DC-

Converter(not included into

camera housing)

12 V

Power

Supply (Car)

12 V

Power

Supply (Car)


Customized Laser Modules

Image of completed base module Optical output pulse

Modular design depending on application

– High Output energy up to 400 !Ws (Multilayer design)

– Selectable pulse width 40 - 200 ns, 40 kHz repetition rate

– Laser class 1 (module + illumination optics)


Road Safety Application- Objectives

One Active 3D Range Sensor for a Variety of Different

Functions and Applications:

• The near to intermediate side

and front distance range of

cars up to 25 m

• Blind area surveillance

• 3D- algorithms for traffic

object detection and

classification

• Performance investigation

with car/truck demonstrators

Front view

Blind Area Surveillance

Side View


Requirements and Specifications

• Front View Perception (BMW & CRF):

– Pedestrian and rear end car collision mitigation in

urban areas

– Protection by means of autonomous and semi-

autonomous braking and/or conditioning of other

active components


Requirements and Specifications

Side Crash Perception

for Cars (Renault)

Blind Area Surveillance

for Trucks (VTEC)


3D Test System in Experimental Car


Scenarios for Vehicle Tests

Camera

frontal view, mounted at hood

frontal view, mounted at hood

frontal view,mounted at hood

lateral view,mounted at side window

Laser 1 module (75 µWs), isotropic illumination

2 modules (2 x 75 µWs), anisotropic illumination

2 modules,isotropic illumination

2 modules,anisotropic illumination

2 modules,isotropic illumination

lateral and frontal view,

collisionmitigation

pedestrianprotection

side crashprediction

blind areasurveillance

Application


Distance Calibration in a 20 m Stage

Target (reflector, dark felt, plywood)2 4 6 8 10 12 14 16 18 20 22

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10 12 14 16 18 200

2

4

6

8

10

12

14

16

18

20

0

2

4

6

8

10

12

14

16

18

20

0 2 4 6 8 10 12 14 16 18 20true distance [m]

2 4 6 8 10 12 14 16 18 20 22true distance [m]

-0.4

0

0.4

0.8

calibration

function

spatial

standard

deviation

= ~0.3m

20 m track (computer controlled position) plywood target


Test Scenario: 2 cars

! both cars (Renault and Mercedes) are visible to the camera

Mercedes at 11 m Renault at 5 m

range data


Test Scenario:

Approaching Pedestrian

• Pedestrian detection from 18 m to 4 m

Image scene with a pedestrian Range data with 64 pixel line sensor

0 10 20 30 40 50 600

2

4

6

8

10

12

14

16

18

20true distance 12 m

measured distance/m

pixel

true distance = 12m

me

as

ure

d d

ista

nc

e [

m]

pixel number


Test Scenario:

Laterally Passing Pedestrian

range data for the marked (x)

pedestrian positionbirds-eye view

camera mounted

on side window

0 10 20 30 40 50 600

2000

4000

6000

8000

10000

1200012000

10000

8000

6000

4000

2000

00 10 20 30 40 50 60

# pixel

testcar

pedestrian# pixel0

63

30


0 10 20 30 40 50 600

2000

4000

6000

8000

10000

1200012000

10000

8000

6000

4000

2000

00 10 20 30 40 50 60

# pixel

Test Scenario: Side Crash Protection - Tree

at Lane Side

! tree trunk is well visible at 5 m distance

tree at 5 m

range data


Test Results

" The sensor performance was successfully

investigated in different test scenarios,

including moving objects

" !"#$%#&'()*(+*&,#++'-*.-/)#,'(.*.0(1%./+%$*,#)2/*3#&#*%4*&(*56*7

" 89.&/7*-#)*-(4/*1'&0*(%&3((,:.'&%#&'().;')-$%3')2*<,'20&*.%)*$'20&

" =(,*')&/,'(,*7(%)&')2*>?:&,#).4#,/)&1')3(1.*#,/*)/-/..#,9


"Improve detection and enable classification:increase image resolution to 64*8 pixels

"For detection of dark obstacles at higher distances:increase sensitivity by lowering NEP

">)-,/#./*3'.&#)-/*,#)2/*<9*7/#).*(+*#3#4&'"/.'2)#$*#--%7%$#&'()

">74$/7/)&*$#./,*7(3%$/.*1'&0*0'20/,*(%&4%&/)/,29*#)3*0'20/,*,/4/&'&'()*+,/@%/)-9*AB6*CDEF#)3*-%.&(7'E/3*(4&'-.*1'&0*$(1*+:)%7</,

"!)0#)-/*./).(,*4#-C#2/*+(,*-#,*/)"',()7/)&

Ongoing Camera Enhancements


Principles of Image Processing

(images composed of

line measurements)

• Goal: Reliable and predictable 3D

position and velocity estimation for

traffic objects (VRU’s, vehicles,

obstacles)

• Sources of information:

– Depth image:

• Segmentation

• Object classification

• 3D position and motion prediction

(Kalman filter)

– Intensity image:

• Confidence estimate

Intensity image

Depth Image


Discussion

Thanks for your attention

Date post:	10-Apr-2015
Category:	Documents
Upload:	greta-garbo
View:	20 times
Download:	2 times

3D Imager

Documents