Interactively Modeling with Photogrammetry

Pierre PoulinMathieu Ouimet

Marie-Claude Frasson

Dép. Informatique et recherche opérationnelle

Université de Montréal

Motivation

• Photo-realism is difficult to achieve

• Important recent progress in rendering

• Acquiring realistic 3D models is still a major hurtle

• Important needs for realism, special effects in movies, CAR, etc.

Extracting 3D models from photographs

Computer Vision / Robotics

• 3D models do not satisfy most of the visual accuracy necessary in graphics

• Fully automatic systems are challenging :– false correspondences– missed edge detections– noise– textures

• Provide much inspiration in our system

Our Interactive Reconstruction System

• User knows the 3D models / textures

• User is responsible for everything

• User interactions :– User draws 2D primitives– User puts the 2D primitives in correspondences– User adds 3D constraints– User extracts a unified texture

Drawing 2D Primitives

Correspondences

3D Constraints

Perpendicularity

Parallelism

Co-planarity

Extracted Textures

Reconstruction Process

• Incremental

• Robust

• Intuitive

• Provides good graphics models

• Labor-intensive

The Camera

• Our camera is a transformation matrix

• No explicit need for real camera parameters

1

0000

1098

7654

3210

TTT

TTTT

TTTT

Reconstructing a Camera

• 6 or more 2D-to-3D point correspondences

(0,1,0)

(0,1,1)

(1,1,0)

(0,0,1)

(1,0,0)

(1,0,1)

Reconstructing a Camera

• Least-squares to compute all Ti

• Solution with SVD– Fast– Robust– Always provides a solution– Conditions for accuracy similar to non-linear

Reconstructing a 3D Point

• Incidence of 3D point on planes

• Least-squares to compute each (x,y,z)

• Polygons as set of 3D points

Reconstructing a 3D Line

• Plücker coordinates of a 3D line

Additional 3D Constraints

• Co-planarity

• Parallelism

• Perpendicularity

• Weights can be used to alter the importance of certain constraints

Weights

Iterating

• Better cameras give better 3D geometry

• Better 3D geometry give better cameras

• Iterations between the two improve both

Convergence

Recovering Texel Colors

u

v

u

v

t

s

Texture map 3D Polygon

2D Images

t

t

s

s

Occlusion Testing

Zones of Occlusion3D Model

2D Image

Linear Fit

• Misalignments due to imprecisions in the 3D model and its cameras

• 2D transformation matrix using least-squares

Unifying Texel Criteria

• Clustering to discriminate view-dependent colors for a texel

• Other metrics used to weight valid texels :– Projected area (adaptive sampling)– Texture quality

Two Scenes with Cubes

Desktop

Lego Tower

Coffee Pot

Conclusions

• User knows best

• Satisfying 3D models and extracted textures

• Labor-intensive

Future Work

• Better user interface

• Error detection

• Radiances, reflectances, and global illumination

• Displacement maps on 3D primitives

• Bounds on reconstructed information