Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 216 times |
Download: | 1 times |
ON THE IMPROVEMENT OF IMAGE REGISTRATION FOR HIGH
ACCURACY SUPER-RESOLUTION
Michalis Vrigkas, Christophoros Nikou, Lisimachos P. KondiUniversity of Ioannina
Department of Computer ScienceIoannina, Greece
• Objective– Reconstruct a high-resolution image from a
sequence of low-resolution images.• Improve spatial resolution.
• Constraints on low-resolution images– Motion– Rotation– Blurring– Subsampling– Additive noise
MOTIVATION
• MAP scheme for image super-resolution.
• Registration in two parts– At first, the LR images are registered by
establishing correspondences between robust SIFT (Scale-Invariant Feature Transform) features.
– In the second step, the estimation of the registration parameters is fine tuned along with the estimation of the HR image.• Mutual Information Criterion: maximize the mutual
information between HR image and each of the upscaled LR images.
APPROACH
• Let the high-resolution image
where
• The set of LR images is described as
We consider p LR images each of size
FORMULATION MODEL
1 2 [ , , , ]TNz z zz 1 1 2 2N L N L N
1 2[ , , , ]T T T Tpy y yy
1 2M N N
• Observation model:– All images are ordered lexicographically
– represents zero-mean additive Gaussian noise,
– is the degradation matrix, performing the operations of:• motion• blur• down-sampling
FORMULATION MODEL (cont.)
y = Wz +n
nW
( )k k kW = DB M s
• The Gaussian prior for the HR image is:
– is the Laplacian of the image z– controls the precision and the shape of the
distribution
• The likelihood of the LR images is Gaussian:
MAP ESTIMATOR
/2
/2
( | |) 1( ) exp ( ) ( )
(2 ) 2
T NT
Np
Q Qz Qz Qz
Qz
22
1 ( ) ( )( | ) exp
2(2 )
T
pMpM
p
y Wz y Wzy z
• MAP approach– Maximize – Which leads to a MAP functional to be minimized with
respect to HR image z and the transformation parameters s:
• Use gradient descent method– The update equation is given by:
where εn is the step size at the n-th iteration.
MAP ESTIMATOR (cont.)
( | ) ( | ) ( )p p pz y y z z
2 2 2
1
( , ) | | ( ) || || where =||p
k k kk
L
z s y W s z Qz
1( , ) | n
n nn L
z z z
z z z s
• Objective: independently detect corresponding keypoints in scaled versions of the same image.
• Idea: Given a keypoint in two images, determine if the surrounding neighborhoods contain the same structure up to scale.
• SIFT features are invariant to:– Image scale and rotation– Affine transformations– Changes in illumination and noise
[D. G. Lowe. "Distinctive image features from scale invariant
keypoints.”International Journal of Computer Vision 60 (2), pp. 91-110, 2004.]
SCALE INVARIANT FEATUTE TRANSFORM - SIFT
• Basics: the mutual information is maximized when the two images are correctly registered.
• The mutual information between two images A and B is:
– H(A) and H(B) are the marginal entropies of the random variables A and B.
– H(A,B) is the joint entropy.
MUTUAL INFORMATION CRITERION
( , ) ( ) ( ) ( , )
( , ) ( , ) log
( )· ( )AB
ABa b A B
I A B H A H B H A B
p a bp a b
p a p b
• Normalized Mutual Information:– Robust measure in order to provide invariance to the
overlapping areas between the two images.
• Problem: – If mutual information is not initialized close to the
optimal solution it is trapped by local maxima.• Good initialization is important.
• Solution:– Initialization using SIFT descriptors.
MUTUAL INFORMATION CRITERION (cont.)
( ) ( )( , )
( , )
H A H BNMI A B
H A B
• Estimation of registration parameters in two steps.– First step, LR images are registered by
employing SIFT features.• Minimization of mean square error between the
locations of features between the reference image and the LR images.
• Provides good initialization.
IMAGE REGISTRATION
– Second step, the estimation of the registration parameters is fine-tuned along with the estimation of the HR image, by maximization of mutual information criterion.• Iterative scheme.
• Contribution:– The registration accuracy is improved at each
iteration step.– Refinement of the mutual information
registration.
IMAGE REGISTRATION (cont.)
• Synthetic data sets.
• LR images were created by rotating, translating, blurring, down-sampling and degrading by noise.– Translation: uniformly selected in [-3, 3] (in pixels)– Rotation: uniformly selected in [-5, 5] (in degrees)– Down-sampling factor: 2 (4 pixels to 1)– Blurring: 5x5 Gaussian kernel, standard deviation
of 1– Additive noise: AWGN to obtain SNR of 30 dB and
20 dB
EXPERIMENTAL PARAMETERS
• First estimate of the HR image– Bicubic interpolation
• Total number of realizations for each case: 10
• Convergence: or 70 iterations reached.
• Quantitative evaluation: peak signal to noise ratio
EXPERIMENTAL PARAMETERS (cont.)
15/ 10
n n n z z z‖ ‖ ‖ ‖
2
10 2
255PSNR 10log
ˆ|| ||z z
COMPARE METHODS
• Books (PSNR = 26.06 dB)
• 4 LR images used
EXPERIMETAL RESULTS
LR image
Reconstructed HR image
• Front page (PSNR = 26.14 dB)
• 6 LR images used
EXPERIMETAL RESULTS (cont.)
LR image
Reconstructed HR image
• Car (PSNR = 28.13 dB)
• 5 LR images used
EXPERIMETAL RESULTS (cont.)
LR image
Reconstructed HR image
• Eye chart (PSNR = 27.33 dB)
• 4 LR images used
EXPERIMETAL RESULTS (cont.)
LR image
Reconstructed HR image
EXPERIMETAL RESULTS (cont.)
• Statistics for the compared SR methods
+1.5 dB on average better results than SIFT.
• Hybrid registration approach– SIFT-based image registration combined with
the maximization of mutual information.– High precision registration
• High accuracy super-resolved image.– Improvement is 1.5 dB on average higher for
both 30 dB and 20 dB.
• Proposed algorithm converges faster than the standard solution.
CONCLUTIONS
QUESTIONS?
?
THANK YOU ALL FOR YOUR PARTICIPATION AND
PATIENCE!