Date post: | 26-Dec-2015 |
Category: |
Documents |
Upload: | mark-cameron |
View: | 213 times |
Download: | 0 times |
MACHINE VISION GROUP
Graphics hardware accelerated panorama builder for mobile phones
Miguel Bordallo López*, Jari Hannuksela*, Olli Silvén* and Markku Vehviläinen**,
* University of Oulu, Finland** Nokia Research Center, Tampere, Finland
MACHINE VISION GROUP
Contents
IntroductionDocument Panorama Builder
• Interactive user interface• Quality evaluation and frame selection• Re-registration and blending• Computation times
Scene Panorama BuilderGraphics processor as a computing engineGPU accelerated panorama builder
MACHINE VISION GROUP
IntroductionDocument capture with single snapshot
ViewFinder Image Real size section
MACHINE VISION GROUP
IntroductionPanorama Builder
Panorama builder• Used to combine several images into one wide-angle mosaic image• Traditionally image mosaic is formed from a few separate images• Video sequences can also be used as a source
MACHINE VISION GROUP
IntroductionVideo Mosaicking
• Videos contain lots of information – most of it redundant• Video mosaicking vs. image mosaicking
– image registration is easier due to smaller motions between frames
– some frames can be discarded without losing information
MACHINE VISION GROUP
Document Panorama BuilderInteractive user interface
•VGA video analysis
•Motion estimation system calculates shift, rotation and scale in real time
•When frame is suitable for stitching (high quality), the user receives feedback and instructions
MACHINE VISION GROUP
Document Panorama BuilderQuality determination and frame selection
MACHINE VISION GROUP
Document Panorama BuilderRe-registration and blending
Selected frames
Final panorama
MACHINE VISION GROUP
Document Panorama BuilderComputation times
• 1. Registration with SIFT feature based RANSAC to find the warping and stitching parameters– approx. 6 frames/minute in software
• 2. Image Correction: warping with interpolation – approx. 2 frames/s in software
• 3. Quality determination and frame selection during the capture stage – approx. 10 frames/s in software
• 4. Motion estimation during capture stage – approx. 30 frames/s in software
MACHINE VISION GROUP
Scene Panorama Builder
• >360 degrees panoramas from landscape• Real-time registration• Real-time frame evaluation and selection• Real-time frame correction• On-the-fly stitching• Less memory requirements
MACHINE VISION GROUP
GPU as a computing engine
• Newer phones include a GPU chipset
• OpenGL ES as a highly optimized and attractive accelerator interface
• Compatible data formats for graphics and camera sub-systems desirable
• Emerging platforms (CUDA , OpenCL) facilitate using the GPU as a computing resource
MACHINE VISION GROUP
Fixed pipeline (OpenGL ES 1.1) vs. programmable pipeline (OpenGL ES 2.0)
MACHINE VISION GROUP
Stream processing (OpenGL) vs. shared memory processing (CUDA)
MACHINE VISION GROUP
OpenCL (Embedded Profile)
• Emerging platforms will offer needed flexibility
• Code executed concurrently on CPU & GPU (even DSP)
• High parallelization on image processing computations . High efficiency
MACHINE VISION GROUP
GPU-accelerated panorama builder
• Open GL ES 1.1:– Warping with interpolations:
• >4 times faster
– Pixel blending:
• Open GL ES 2.0:– Image features (SIFT, Haar,...) extraction:
• 10 times faster on PC, CPU load decreased 50%
– Frame quality evaluation
• Emerging platforms:– RANSAC estimator– Better efficiency– Enabling the use of other chipsets
MACHINE VISION GROUP
Image warping with interpolation on OpenGL ES 1.1
Create quad
Transform
Render texture
MACHINE VISION GROUP
Summary
•Document mosaicing allows feedback to the user•Quality can be measured while capturing•Mobile phones are an excellent platform for this method
•GPUs can be used as a general purpose procesors•New platforms will offer more efficiency and flexibility
•Not optimized interfaces include excesive overheads•Lack of RAM in small devices sitll a problem
MACHINE VISION GROUP