Image Processing in Infrared Cameras - Meetupfiles.meetup.com/4379272/BIV20160602_Norvig.pdfBoston...

Boston Imaging and Vision Group Infrared Vision

Image Processing in Infrared Cameras

Marc Norvig Principal Engineer FLIR Systems, Inc.


[email protected] 06/02/2016 Image Processing in Infrared Cameras

Hashagen, J. (September 2014). SWIR Applications and Challenges: A Primer. EuroPhotonics http://www.photonics.com/Article.aspx?AID=56646

Reflected IR Thermal IR



Reflected Infrared

Photos by Nick Spiker https://www.facebook.com/invisiblelightimages/



Long-Wave (8 – 14 μm)

Photos Courtesy FLIR Systems, Inc.

Thermal Infrared Camera Comparison Mid-Wave (3 – 5 μm)



Long-Wave (8 – 14 μm) + Room temperature operation

Photos Courtesy FLIR Systems, Inc.

Thermal Infrared Camera Comparison Mid-Wave (3 – 5 μm) – Cyrogenic (77˚K to 155˚K)



Long-Wave (8 – 14 μm) + Room temperature operation + Compact + Lightweight + Lower power + Lower price + Instant imaging + 20-plus year service interval + Very good radiometric accuracy

Thermal Infrared Camera Comparison Mid-Wave (3 – 5 μm) – Cyrogenic (77˚K to 155˚K) – Size – Weight – Power – Cost – Cooldown time – 10 to 15,000 hour service interval + Best thermal image quality + Best sensitivity + High contrast + Long distance viewing + Higher frame rates



offset(x,y) gain(x,y)

Sensor

Temp

Image

Memory

Vshutter (x,y)

OV(T3)

1

GV(T2)

Raw

Image

Data-

+

+

+ -Corrected

Infrared

Image

Shutter

Sensor Corrections

- Mechanical shutter captures “dark image” for subtraction

- Each pixel has a different bias point and sensitivity due to manufacturing process variation

- Corrected using per-pixel offset and gain tables

- All pixels now have uniform response to relative temperatures

- For accurate temperature measurement we need to correct for sensor temperature variation

Corrected Image

Sensor Corrections

Raw Image



Sosnowski, T. et al. (2010). Processing of the Image from Infrared Focal Plane Array Using FPGA-based System. MIXDES Proceedings of the 17th International Conference, 581-586

Corrected Image

Sensor Corrections

Raw Image



Budzier, H. & Gerlach, G. (2015). Calibration of uncooled thermal infrared cameras. Journal of Sensors and Sensor Systems,4(1), 187-197

Defective Pixel Replacement

- Some pixels can’t be made to match others (dead, excessive gain, hot, blinking)

- These pixels are marked as defective at factory calibration

- The bad pixels must be replaced with a better value for image viewing

- Approach this as either a denoising problem or an inpainting problem

- Simplest solution is nearest neighbor replacement - Need method to choose between equidistant neighbors

- Spatial filter over a small local neighborhood - If any neighborhood pixel is also defective it must be excluded (remember to renormalize the kernel)

- Simple mean of a few neighbors is surprisingly effective

- Median filter never seems to turn out as well as you want it to

- Adaptive, gradient-based, and patch-based methods preserve image structure - Probably not be worth the computational effort



Image Enhancement

- Corrected sensor output is 14-bit but display is 8-10 bits

- Typical scenes have high dynamic range (hot car engine on a winter day) - Most of the pixels in this scene will be the same temperature (low contrast)

- Combined bit length reduction and contrast enhancement is required

- Contrast Limited Adaptive Histogram Equalization (CLAHE) and its derivatives have emerged as the best solutions.



Image Enhancement



Image Enhancement



Original Histogram EQ

Image Enhancement



Original CLAHE



Visible–Infrared Image Fusion: Registration

- Image-driven registration requires features that appear in both visible and IR images - Finding good joint feature detectors is an open problem

Lui, F. & Seipel, S. (2015). Infrared-visible image registration for augmented reality-based thermographic building diagnostics. Visualization in Engineering,3(16)



Visible–Infrared Image Fusion: Image Combining

- Combining can occur at various levels - Pixel-level

- Feature-level

- Object-level

- Transform Domain Techniques - Pyramids (Gaussian, Laplacian, Gradient)

- Wavelets, Curvelets

- Spatial Domain Techniques - Weighted blending (global or locally adaptive weights)

- PCA

- Gram-Schmidt

- High-pass filter



Feature-based Fusion Example



Infrared Camera 50% Blend Visible Camera



Vis + IR Features

Infrared Camera Shared Features

Visible Features Infrared Features

Visible Camera



Infrared Camera 50% Blend


Visible Camera

Grayscale Fused



Infrared Camera False-Color Fused


Visible Camera

Grayscale Fused



High-pass Filter Fusion Example (FLIR MSX®)



Visible Camera Infrared Camera High-pass filter



Visible Camera Infrared Camera MSX

®

Visible Camera Infrared Camera MSX

®


Thank you


Date post:	23-Jul-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Image Processing in Infrared Cameras - Meetupfiles.meetup.com/4379272/BIV20160602_Norvig.pdfBoston...

Documents