Satellite image deconvolutionusing complex wavelet packets
André Jalobeanu, Laure Blanc-Féraud, Josiane Zerubia
ARIANA research groupINRIA Sophia Antipolis, France
CNRS / INRIA / UNSA
www.inria.fr/ariana
Satellite image deconvolution / CWP 2
• Problem statement• Efficient representations and basis choice• Complex wavelet transform• Complex wavelet packet transform• Transform thresholding
– Different methods– Parameter estimation– Two algorithms : COWPATH 1 and 2– Results
• Conclusion and future work
Satellite image deconvolution / CWP 3
Observed images are corrupted :
Y = h * X0 + N Noise :White and Gaussian(known variance)
Observation equation
Convolution kernel(known PSF)
Original imageObserved image
Satellite image deconvolution / CWP 4
Problem statement
Ill-posed inverse problem [Hadamard 23]
• existence,• unicity,• stability of the solution ?
Inversion à noise amplification
Small errors of Y à high errors of X
Satellite image deconvolution / CWP 5
Introduction
• Monoscale methods [Geman & McClure 85, Charbonnier 97, …]
Regularization + edge preservation
Find X by minimizing U(X) :
U(X) = ||Y-h*X||2 / 2σ2 + Φ(X)
Data term Non-quadraticregularization term
• Multiscale methods [Mallat 89, Bijaoui 94, …]Multiresolution analysis à wavelets• Regularization of classical iterative methods (statistics)
(shift invariant wavelet transform thresholding)
• Multiresolution variational models
Satellite image deconvolution / CWP 6
Introduction
• Filtering after inversion [Donoho, Mallat, Kalifa 99]
• Non-regularized inversion (Fourier domain)• Transform (change the basis)• Coefficient thresholding• Inverse transform (return to image space)
Satellite image deconvolution / CWP 7
Representationsfor efficient filtering
Efficient separation of the signal and the deconvolved noise :- compact reprensentation of the signal- efficient compression of the noise in high frequencies
Image deconvolvedwithout regularization
Transform
signal
noise
Satellite image deconvolution / CWP 8
Filtering the deconvolved noise
• Cancel the coefficients corresponding only to the noise• Thresholding the coefficients corrupted by noise
In the new basis, the coefficients of the noise transformmust be independentà enable separate thresholding
è noise covariance « nearly diagonalized » [Kalifa 99]
The deconvolved noise is colored !!
Satellite image deconvolution / CWP 9
Choice of the basis
• Compact representation • « nearly diagonal » noise covariance
The thresholding estimator is optimal [Donoho, Johnstone 94]
T-T
x
θT(x) Thresholding function
Coefficientin the new basis
Satellite image deconvolution / CWP 10
Algorithm design
Choice of the basis :• compacity
• diagonalization• reconstruction
• invariance properties
Choice of thethresholding function
Optimal threshold value ?
Roughdeconvolution
Directtransform
thresholding Inversetransform
Satellite image deconvolution / CWP 11
Complex wavelets
¶ Shift invariance¶ Directional selectivity¶ Perfect reconstruction¶ Fast algorithm O(N)
Properties :
• quad-tree (4 parallel wavelet trees) [Kingsbury 98]
• filters shifted by ½ and ¼ pixel between trees• combination of trees à complex coefficients
• filter bank implementation • biorthogonal wavelets
Properties :
Satellite image deconvolution / CWP 12
Quad-tree : 1st level
a0(image)
a1A
d21A
d11A
d31A
a1B
d21B
d11B
d31B
a1C
d21C
d11C
d31C
a1D
d21D
d11D
d31D
a1
d21
d11
d31
Non-decimated transform Parallel trees ABCD
AB
CD
AB
CD
AB
CD
AB
CD
AA A
A
AA A
A
AA A
A
AA A
ABB B
B BB B
B
BB B
B BB B
BCC C
CCC C
C
CC C
CCC C
CDD D
D
DD D
D
DD D
D
DD D
D
Perfect reconstruction :
mean (A+B+C+D)/4
Satellite image deconvolution / CWP 13
â2e
â2e
he
ge
aj,A
he
ge
â2e
â2e
he
ge
â2e
â2e
aj+1,A
d1j+1,A
d2j+1,A
d3j+1,A
Quad-tree : level j
different length filters : ho, go, he, ge à shift < pixel
â2e
â2e
he
ge
aj,B
ho
go
â2o
â2o
ho
go
â2o
â2o
aj+1,B
d1j+1,B
d2j+1,B
d3j+1,B
â2o
â2o
ho
go
aj,C
he
ge
â2e
â2e
he
ge
â2e
â2e
aj+1,C
d1j+1,C
d2j+1,C
d3j+1,C
â2o
â2o
ho
go
aj,A
ho
go
â2o
â2o
ho
go
â2o
â2o
aj+1,D
d1j+1,D
d2j+1,D
d3j+1,D
Satellite image deconvolution / CWP 14
Complex coefficients
M
dkj,A
dkj,B
dkj,C
dkj,D
4 real subbands
Zkj+
Zkj-
2 symmetricComplex subbands
Z + = (A - D) + i (B + C)Z - = (A + D) + i (B - C)
!The wavelet function is not a complex function.Not exactly ‘complex’ wavelets !
Satellite image deconvolution / CWP 15
Necessity of the packets
Compact representation
Poor representation of the deconvolved noise
Complexwavelets :
TransformImage deconvolvedwithout regularization High frequencies
not recoverable
Satellite image deconvolution / CWP 16
Wavelet packets
j=0
j=1
è decomposethe detail spaces[Coifman et al. 92]
j=2
Satellite image deconvolution / CWP 17
Deconvolved noise power spectrum
0 1/2Normalizedfrequency
energy
Choice of the tree
• no unicity of the decomposition tree
• application dependent
• deconvolution : must adapt to the deconvolved noise
wavelets packets Limit thegrowth of the noise variance
Satellite image deconvolution / CWP 18
Complex wavelet packets (CWP)
image
φ2φ2φ2φ2
è decompose the detail spaces of thecomplex wavelet transform
for each tree A,B,C,D
TransformOriginal image
Satellite image deconvolution / CWP 19
Complex wavelet packets (CWP)
Compact representationComplexWaveletPackets :
TransformImage deconvolvedwithout regularization
High frequenciesrecoverable
Nice representation of the deconvolved noise
Satellite image deconvolution / CWP 20
Frequency plane partition
Satellite image deconvolution / CWP 21
Directional selectivityimpulse responses – real part
Complex wavelets
Complex wavelet packets
Satellite image deconvolution / CWP 22
Directional selectivityimpulse responses – imaginary part
Complex wavelet packets
Complex wavelets
Satellite image deconvolution / CWP 23
Comparison with real wavelet packets
No shift invarianceè artefacts (mean over translations)
No rotation invariancePrivileged directions : horizontal / verticalè poor texture representation
variously oriented (diagonals)Impulse
responses
Satellite image deconvolution / CWP 24
Example
Test image, 512x512
Satellite image deconvolution / CWP 25
Example
Complex wavelet packet transform, level 6
Satellite image deconvolution / CWP 26
Transform thresholding
Filter only the magnitudeè enable shift invariance
recall : observed images are corrupted :Y = h * X0 + N
each coefficient of the subband kof the CWP transform is corrupted :
x = ξ + n
Deconvolved noisestandard dev. σk
Original coefficientunknown
(original imageCWP transform)
Observed coefficient(CWP transform of
the deconvolved image)
( ) ( )xaxxx̂ TT =θ=
Satellite image deconvolution / CWP 27
Thresholding functions
Data : image deconvolved without regularization
Fix a thresholdingfunction θT
Optimal threshold computation :minimize the risk
•Minimax risk [Donoho 94]• subband modeling
Bayesian methods :
Coefficient estimation byMAPè function θT
Models for the subbands :• Homogeneous
generalized Gaussian• Inhomogeneous Gaussian
Satellite image deconvolution / CWP 28
Deconvolved noise variance
σk
min
max
Estimation of σk :• simulation (CWP transform of a white Gaussian noise)• direct computation, with known h and σ
[ ][ ]
2
j,i ij
ijk
22k hFFT
RFFT∑σ=σ Impulse response
for subband k
Satellite image deconvolution / CWP 29
Optimal risk• Impose a thresholding function θT
• Minimize the risk of the thresholding estimator
−=
200 XX̂E)X,X̂(r ( )
ξ−θ= ∑
m
2T xE
• Theoretical results [Donoho, Johnstone 94]not useful in practice (too large threshold) [Kalifa 99]
• Subband modeling (Generalized Gaussian [Mallat 89])à model parameters estimation
Satellite image deconvolution / CWP 30
Subband modeling
Generalized Gaussian : ( ) p/
p,eZ
1P αξ−
α=ξ
Experimental study :
ReIm
ReIm
Histograms :original image
CWP transform subbands
α,p modelparameters
Satellite image deconvolution / CWP 31
Bayesian methods
• Subband modelingà parameter estimation
• No arbitrary choice of thresholding function
• estimate x by Maximum A Posteriori (MAP)( ) ( ) ( )ξξ=ξ PxPMaxxPMax
( ) p/eP αξ−∝ξ
( ) 22 2/xexP
σξ−−∝ξ
p22 /2/xMinx̂ αξ+σξ−= ξ
( )xx̂ θ=
Estmation of themodel parameters α,p : Maximum Likelihood,…
pk=1 pk=0,2
Classical thresholdingfunctions for particularvalues of pk
Satellite image deconvolution / CWP 32
COWPATH 1.0« COmplex Wavelet Packets Automatic Thresholding »
Satellite image deconvolution / CWP 33
Inhomogeneous Gaussian ModelInsufficiency of homogeneous models
(constant areas / edges / textures)
( ) 2ij
2ij s2/2ij
ij es21P ξ−
π=ξ
Parameter sij : depends on the location of the coefficient ξij
!Estimation problems for the model parameters sij(not enough data / number of unknown parameters !)
Hybrid method : parameter estimationfrom a ‘good’ approximation of the original image
Complete Data Maximum Likelihood
Satellite image deconvolution / CWP 34
Roughdeconvolution
YDeconvolution(Non-quadratic ϕvariational model)
COWPATH 2.0
CWP
CWPParameterestimation
(ML)
thresholding(MAP)
Noisevariances
computation
X̂CWP-1
Computation time3,5s on PII 400(quadratic ϕ)
Satellite image deconvolution / CWP 35Nîmes, original image 512 x 512 © French Space Agency (CNES)
Satellite image deconvolution / CWP 36Nîmes, blurred and noisy image (σ~1.4)
Satellite image deconvolution / CWP 37Nîmes, COWPATH 1 result
Satellite image deconvolution / CWP 38Nîmes, COWPATH 2 result
Satellite image deconvolution / CWP 39Nîmes, COWPATH 2 result - enlarged
smoothhomogeneous
areas
sharptextures
sharp andregular
diagonal edges
Satellite image deconvolution / CWP 40Nîmes, deconvolution using real wavelet packets [Kalifa, Mallat 99]
Satellite image deconvolution / CWP 41Nîmes, deconvolution using RHEA (non-quadratic ϕ function regularization [Jalobeanu 98])
Satellite image deconvolution / CWP 42Nîmes, deconvolution using a quadratic regularization (~Wiener)
Satellite image deconvolution / CWP 43Result comparison
X0 Y ϕ func
COWPATH 1 COWPATH 2 WienerReal packets
Satellite image deconvolution / CWP 44
Results / Astronomy (Hubble PSF)
Deconvolutionusing quadraticregularization
(~Wiener)
Deconvolution using CWP
X0
Y
Satellite image deconvolution / CWP 45
Y
X0
Wiener
COWPATH 2
Satellite image deconvolution / CWP 46
Conclusion and future work
Providing better results by
ð Adapting the structure of the tree to the problemà taking into account images and PSFs
ð Better subband modelingà inhomogeneous Generalized Gaussian model ?
ðMore accurate data termà noise transform coefficients not fully independent
ð Taking into account the interactions between scalesà Hidden Markov Trees [Nowak et al. 98]
Hybrid method : DEPA [Jalobeanu et al. 00]
Result of COWPATH à estimation of the parametersof an adaptive regularizing model