Removing Weather Effects fromMonochrome Images
Srinivasa Narasimhan and Shree Nayar
Computer Science DepartmentColumbia University
IEEE CVPR ConferenceDecember 2001, Hawaii, USA
SponsorsDARPA HID, NSF
How does scene contrast degrade in bad weather ?
How can scene contrast be restored from bad weather images ?
Contrast Degradation in Bad Weather
RainFog
Weather Effects are Depth Dependent
Image Processing Does Not Suffice
Histogram Equalized Images
Prior Methods for Contrast Restoration
Yitzhaky, Kopeika [98]
Oakley, Tan, Satherley[98,01]
Nayar, Narasimhan [99]Narasimhan, Nayar [00]
Scene DepthWeather
Information
RequiredPredicted Weather PSF
Required
ComputedWavelengthIndependentScattering
Not Required
MethodClear-day SceneIntensity/Color
Computed
Gaussiandistribution
assumed
Computed(Color Images Required)
OUR GOAL : ComputedNot Required Computed
Scattering Models : Attenuation and Airlight
Object
Observer
d
Attenuation
d
SunlightDiffuseSkylight
DiffuseGround Light
Airlight
E
d
deEE )1( deEE
Contrast Degradation in Bad Weather
Irradiance = Attenuation + Airlight
)(dE += deE )1( deE
HorizonBrightness
Depth Reflectance ScatteringCoefficient
Contrast Decay : Exponential in Scene Depth
)1(2)2()1(
)2()1(
)2()1(
)2()1(
12
deEE
EEC
Contrast between Iso-Depth points , P and P :(1) (2)
Mild Fog Denser Fog
Depth Edge Reflectance Edge
Depth Edges vs. Reflectance Edges
( 2) ( 2)Normalized SSD of
Reflectance Edge NeighborhoodNormalized SSD of
Depth Edge Neighborhood
Edge Classification from Weather Changes
Mild Fog Denser Fog Edge Classification
Reflectance Edge : Depth Edge :
Scene Structure from Weather Changes
Irradiance under versus Irradiance under : Linear 1 21E 2E
2
1
2
1 lnln)(2
112
E
E
EE
EEdScaled Depth :
1E
2E
1E
2E
)1( )()( 12
2
12
1
2 dd eEeE
E
1E2E
All Scene pointsat Depth 1
All Scene pointsat Depth 2
Depth Map from Two Weather Conditions
Mild Fog, 5 PM (Input)
Denser Fog , 5: 30 PM (Input)
Computed Depth Map (Output)
Comparing with Prior Methods:
• Color Images Not Needed
• Works for Wider Range of Weather Conditions
Weather Removal Using Scene Structure
Contrast Restored Image (Output)Dense Fog, 5:30 PM (Input)
Computed Depth Map (Input) Histogram Equalized Image (For comparison)
Different amounts of Fog removed from different Depths
A De-Weathering System
DetectSignificantWeatherChange
Video Frame (Weather 1)
Video Frame(Weather 2)
Scene Structure
System Initialization : Computing Scene Structure
Continuous De-Weathering Using Scene Structure
Scene Structure
ContrastRestored
Video Frame
