Photomosaic Image Stitching Using SIFT Features
Liz Del Cid
EGGN 512 Computer Vision
Colorado School of Mines
Image Mosaics (stitching) Objective:
• blend together several overlapping images into one
mosaic (composite)
Inspiration:
• Multiple images can give us depth and 3D structure
o Stereo Vision
o Shape carving
o Structure from motion
Image Stitching Algorithm Overview
For each pair of images:
1. Extract SIFT features
2. Match Features
3. Estimate homography
4. Transform 2nd image
5. Blend two images
6. Combine images
Blending Images Problem:
Seams are produced when adjacent pixel intensities differ in stitched images due to changed lighting conditions or the
vignette effects. (a reduction of an image’s brightness or
saturation at the periphery compared to the image center)
Solution:
Image pyramids
Pyramid Blending
Laplacian pyramid is an algorithm
using Gaussians to blend the image
while keeping the significant features. It
resizes the image into different levels
(sizes) with Gaussian. Then expands the
Gaussians in to the next level and subtracts from the image in that level
to acquire the Laplacian image.
• After creating the Laplacian pyramids for each image, combine the two images in different Laplacian levels by combining partial images from each of them
Pyramid Blending (continued)
Pyramid Blending (continued)
Chaman Singh Verma and Mon-Ju. “Panoramic Image Mosaic”. University of Wisconsin, Madison. Computer Sciences. http://pages.cs.wisc.edu/~csverma/CS766_09/ImageMosaic/imagemosaic.html William Hoff. “Multi-resolution Image Processing”. Colorado School of Mines http://egdegrees.mines.edu/course/eggn510/lectures/19-Multiresolution.pdf
Pyramid Blending Test
Image splicing: no blending
Image Splicing: Laplacian Pyramid
Combining Images After transforming the second image
• Canvas Creation and Placement: o The images are placed on identical sized canvases and then combined
to create the final image.
[transimg xdata ydata] = imtransform(img2, TFORM);
o xdata and ydata are both vectors of two integers. The first integer of each
array is the offset of the initial point of the transformed image. This
information along with the image sizes allowed for not only the summation
of the transformed and static images, but also for an adequately sized
canvas for the resultant image.
• Michael Carroll and Andrew Davidson. “Image Stitching with MatLab”. Louisiana State University.
Department of Electrical and Computer Engineering
Combining Images (continued) • For proper image alignment, chose one of four
different cases to construct a dynamic canvas. The
four cases are:
Combining Images (continued) • Final step:
-imadd()/imsubtract()
• These allow for the combining of the transformed
and static image to create the output image
canvas.
• imsubtract() is first used to clear out the
overlapping area of the final image.
• imadd() then appends the difference between the
images to the area that was overlapping while
keeping the result from imsubtract().
My Results
16 images
≈22 images
7 images
8 images
Recognizing Panoramas
Recognizing Panoramas
System components
• Feature detection and description
• Fast matching (hash table)
• RANSAC filtering of matches
• Intensity-based verification
• Incremental bundle adjustment
M. Brown, R. Szeliski, and S. Winder. Multi-image matching using multi-scale oriented patches, CVPR'2005
Matching Mistakes
• Accidental alignment o repeated / similar regions
• Failed alignments o moving objects / parallax
o low overlap
o “feature-less” regions
More Panoramas
Bibliography • Chaman Singh Verma and Mon-Ju. “Panoramic Image Mosaic”. University of Wisconsin, Madison.
Computer Sciences. http://pages.cs.wisc.edu/~csverma/CS766_09/ImageMosaic/imagemosaic.html
• William Hoff. “Multi-resolution Image Processing”. Colorado School of Mines http://egdegrees.mines.edu/course/eggn510/lectures/19-Multiresolution.pdf
• Michael Carroll and Andrew Davidson. “Image Stitching with MatLab”. Louisiana State University. Department of Electrical and Computer Engineering
• Matthew Brown, Richard Szeliski, and Simon Winder. “Multi-image matching using multi-scale oriented patches”. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), volume I, pages 510-517, June 2005.
• Richard Szeliski. Computer Vision: Algorithms and Applications. Springer, New York, 2010.
• Matthew Brown and David Lowe. “Automatic Panoramic Image Stitching Using Invariant Features”. International Journal of Computer Vision, Vol, No. 3, January 2006 2
Questions?