8/12/2019 7.2 - VerstocktS_ISFEH

1/22

ELIS Multimedia Lab

VIDEO FIRE DETECTIONUSING NON-VISIBLE LIGHT

Steven Verstockt,R. Dekeerschieter, A. Vanoosthuyse,B. Merci, B. Sette, P. Lambert, and R. Van de Walle

Ghent University IBBTFaculty of Engineering

Department of Electronics and Information SystemsMultimedia Lab

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

2/22

2/22

ELIS Multimedia Lab

OVERVIEW

Introduction

- he need for VFD in non-visible light (~ thermal IR imaging)?

LWIR-based flame detector

- Hot object segmentation- Low-cost visual flame features

Multi-sensor fire detection

Conclusions / Q&A

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

3/22

3/22

ELIS Multimedia Lab

The need for VFD ?

Effective response to fire requiresaccurate & timelyinformationof its evolution.

Limitations of traditional sensors

Require close proximity to the fire (~ transport delay)

(Mostly) cannot provide additional information about fire location, size, Generally limited to indoors and are not able to understand the scene

VFD promises fast detection andaccurate localization

VFD can be a viable alternative or complement

for the more traditional sensors and can help shorten the timeline[ Verstockt et al.: State of the art in vision-based fire and smoke detection.

Proc. 14th International Conference on Automatic Fire Detection. (AUBE 2009) ]

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

4/22

4/22

ELIS Multimedia Lab

The need for VFD in non-visible light ?

Use of IR cameras can be of added value

Limitations of Ordinary VFD

-> need for sufficient and specific lighting conditions

-> vulnerable to false alarms due to variability of shape,motion, colors, and patterns of smoke and flames

Combination of IR and ordinary VFD is considered to be a win-win!

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

Project Eagle - univ. Coimbra

8/12/2019 7.2 - VerstocktS_ISFEH

5/22

5/22

ELIS Multimedia Lab

Related work

IR thermal imaging already used successfully in manyvideo surveillance applications

traffic safety (~ pedestrian detection)

airport security, detection of elevated body temperature (swine flu)

material inspection

Due to steady price-reduction, #IR imaging applications

.,but the number of IR-based VFDs is still limited.

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

[Owrutsky et al., Long wavelength video detection of fire in ship compartments][Toreyin et al., Fire Detection in Infrared Video Using Wavelet Analysis]

[Bosch et al., Object discrimination by infrared image processing]

8/12/2019 7.2 - VerstocktS_ISFEH

6/22

6/22

ELIS Multimedia Lab

LWIR-based flame detector

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

7/22

7/22

ELIS Multimedia Lab

Selection of the spectral range

SWIR, MWIR of LWIR ?

SWIR -> near the visible bands -> behavior similar to visible light

-> needs (some) external illumination (~ reflected energy)

MWIR/LWIR -> radiated energy -> no need for external illumination

Flames -> best visible + less disturbed by other objects in LWIR

LWIR is not sensitive to dust, smoke, and fog(~ looking through smoke)

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

The further we go in the infrared spectrum,the more the visual perceptibility decreasedand the thermal perceptibility increased

8/12/2019 7.2 - VerstocktS_ISFEH

8/22

8/22

ELIS Multimedia Lab

Dynamic background subtraction

Moving object detection(with quantized shifted luminosity)

Accelerates and optimizes IR image segmentation

Subtract frame Fn+1with everything that remains constant over time(=> estimated background BGn+1 , which is updated dynamically)

Back-step correction fixes errors of previous BG estimations

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

[ Verstockt et al.: AUBE 2009]

8/12/2019 7.2 - VerstocktS_ISFEH

9/22

9/22

ELIS Multimedia Lab

Hot object segmentation

Automatic histogram based image thresholding [Otsu et al.]

Segments 2 classes : hot foreground and cold(er) background

Treshold-based minimalization of intra-class variance

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

10/22

10/22

ELIS Multimedia Lab

Low-cost LWIR flame features

1. Bounding Box Disorder (BBD)

BB of flames varies considerably over time in both directionsand have a high degree of disorder

Flames -> BBD close to 1; more static objects -> BBD near to 0

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

11/22

11/22

ELIS Multimedia Lab

Low-cost LWIR flame features

2. Turbulence Variance (TVar)

Indicator for area change / boundary roughness

Flames -> TVar close to 1

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

12/22

12/22

ELIS Multimedia Lab

Low-cost LWIR flame features

3. Principal Orientation Disorder (POD)

Disorder in principal orientation for flames >> static objects

POD based on orientation angle between x-axis and major axis

Flames -> POD close to 1

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

13/22

13/22

ELIS Multimedia Lab

LWIR FLAME FEATURES

4. Histogram Roughness (HR)

Histograms of flame regions are very rough

Flame region intensities range almost over whole histogram

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

14/22

14/22

ELIS Multimedia Lab

Global classifier

LWIR-based flame feature -> probability between 0 and 1

Global classifier = mean of 4 low-cost flame feature probabilities

Overall flame probabilityP, is compared to alarm threshold tfire

IF P(Flames) > tfire -> raise fire alarm

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

Tfireis an experimentally determined threshold (= 0.7) alternative ?

future work

8/12/2019 7.2 - VerstocktS_ISFEH

15/22

15/22

ELIS Multimedia Lab

Experimental setup

Xenics Gobi-384 LWIR camera -> 8 14m spectral range

Xenics Xeneth software -> extract appropriate grayscale videoimages out of thermal imaging camera

Own detection algorithm (written in Matlab)+ add-ons for extrema detection+ add-ons for histogram analysis

Performance evaluation framework (GT creation,)

Fire/non-fire real case scenarios

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

16/22

16/22

ELIS Multimedia Lab

Experimental results

8 14 m

spectral range

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

17/22

17/22

ELIS Multimedia Lab

Experimental results: problems

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

Distance Temperature IR reflections

IR blocking

(e.g. glass, water)

8/12/2019 7.2 - VerstocktS_ISFEH

18/22

18/22

ELIS Multimedia Lab

MULTI-SENSOR FIRE DETECTION

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

19/22

19/22

ELIS Multimedia Lab

Low-cost visual flame features

Spatial Flame Color Disorder (SFCD)

Flame colors belong to red-yellow color range

Flame color does not remain steady > spatial disorder

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

20/22

20/22

ELIS Multimedia Lab

MULTI-SENSOR FIRE DETECTION

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

21/22

21/22

ELIS Multimedia Lab

CONCLUSIONS

Proposed LWIR-based flame detector yields good results

-> further testing on a broader range of video sequences isnecessary for a more adequate performance evaluation.

Systems that combine visible and LWIR fire detectionare proven to be more accurate and sensitive than either alone.

-> Combination of IR and ordinary VFD is a win-win!

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.

8/12/2019 7.2 - VerstocktS_ISFEH

22/22

22/22

ELIS Multimedia Lab

Qu

estions

?

VIDEO FIRE DETECTION USING NON -VISIBLE LIGHTSteven Verstockt et al.