Qiaochu Li, Qikun Guo, Saboya Yang and Jiaying Liu*
Institute of Computer Science and TechnologyPeking University
Scale-Compensated Nonlocal MeanSuper Resolution
2013
2
Outline Introduction
Multi-frame SR Nonlocal means SR (NLM SR)
Our Algorithm Scale-detector Scale-Compensated NLM Experimental results
Conclusion & Future work
3
Outline Introduction
Multi-frame SR Nonlocal means SR (NLM SR)
Our Algorithm Scale-detector Scale-Compensated NLM Experimental results
Conclusion & Future work
4
Multi-Frame SR Converge low resolution images into a high
resolution image Direct motion estimation
INVALID in complex situation
5
Nonlocal Means SR Image content repeats in neighborhoods
In temporal and spatial domains Probabilistic motion estimation Weighted average
NLM weight distribution. The weights go from 1 (white) to 0 (black).
6
Problem Scale may be varied in frames by zooming.
Camera motion Object motion
Scale changing effects in adjacent frames. (a) Two adjacent frames, (b) some critical areas of the frames.
7
Outline Introduction
Multi-frame SR Nonlocal means SR (NLM SR)
Our Algorithm Scale-detector Scale-Compensated NLM Experimental results
Conclusion & Future work
8
Scale-Detector Using SIFT descriptor to compute scales
Partial matched keypoints and the corresponding scale values.
9
Verification Verification of scale-detector
Always appears region
The performances of scale-detector in different standard scales and different resolutions,(a) average error by frame scale, (b) average error by frame resolution.
(a) (b)
10
Scale-Compensated NLM SC NLM finds more similar patches
Comparison of unmodified and modified patch-extractor in patch matching.
11
Procedures Overview of SC NLM
Scale-detector
Patch extraction
&modification
NLM SR
12
Experimental Results Downsample
Blurred using 3×3 uniform mask Decimated by 3× factor Additive noise with standard deviation 2
Objective measurement Subjective measurement
13
Experimental Results 3×, Objective measurement (PSNR)
Sequence NLM ARI-SWR SC-NLMForeman 31.15 30.96 31.27Tempete 22.85 22.74 23.00Text 29.23 30.06 30.11Man 27.14 27.02 27.29
14
Experimental Results 3×, Subjective measurement (SSIM)
Sequence NLM ARI-SWR SC-NLMForeman 0.8109 0.8001 0.8151Tempete 0.6927 0.6737 0.7013Text 0.8592 0.8512 0.8633Man 0.7780 0.7617 0.7831
15
Experimental Results
a) Result of whole frame. b) High resolution frame. c) NLM SR. d) SC NLM.
16
Experimental Results
a) Result of whole frame. b) High resolution frame. c) NLM SR. d) SC NLM
17
Outline Introduction
Multi-frame SR Nonlocal means SR (NLM SR)
Our Algorithm Scale-detector Scale-Compensated NLM Experimental results
Conclusion & Future work
18
Conclusion When patches are convert into
SAME SCALE, we can find more
SIMILAR PATCHES, we can use more
COMPLEMENTARY INFORMATION to reconstruct a
HIGH RESOLUTION & QUANLITY IMAGE.
19
Future Work More accurate scale-detector
Segmentation based scale-detector
Combination of rotation and translation-invariant algorithm Rotation-invariant measurement Translation-invariant measurement
20