A Unified Multi-Surface, Multi-Resolution Workspace with

Camera-Based Scanning and Projector-Based Illumination

Tyler Johnson and Henry FuchsUniversity of North Carolina – Chapel Hill

IPT/EGVE July 18, 2007 - Weimar, Germany

2 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Multi-Projector Display

3 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Previous Work

Projected desktopInteraction with virtual

documents Camera-based scanning

Digital Desk[Wellner ’93]

Multi-resolution workspaces Focus Plus Context Displays [Baudisch ’01], Escritoire [Ashdown ’03]

Multi-surface projected workspace

[Ashdown ’04]

4 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Workspace Configuration

5 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Calibration

Cameras pre-calibratedProject structured light to determine

Projector calibrationWorkspace geometry • RANSAC-based plane-fitting. [Quirk EDT ’06]

Also, display structure light with LCDCalibrate as a projector

Workspace Walls

LCD PanelEncoded Structured Light

6 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Two-Pass Rendering

Display Surface

Projector

Viewer

Determine image warping to remove distortion caused by surface geometry

7 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Single vs Multi-Surface Content

Two methods of displaySingle vs. multi-surface

8 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Single Surface Content

User selects region where the window should appear

Compute P-Matrix from 3D->2D corner correspondences• Choose arbitrary COP

Two-pass rendering determines what to project

9 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Multi-Surface Content

Same as for single surface, except that COP is user viewpoint

Viewpoint can be obtained from separate tracking system

10 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Projector-Based Illumination

Projection of imagery onto certain objects is often undesirable

Simple interface allows imagery to be replaced with illumination

11 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Camera-Based Scanning

User selects copy area with selection window

Selection window rendered as a single-surface window

12 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Camera-Based Scanning

Remove perspective distortion using two-pass rendering

Selection window P-matrix

Camera

Selection window

13 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Video

14 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Contributions

Two-pass rendering as a unifying framework for digital workspaces

Image generation• Rectified single surface imagery• Perspectively correct multi-surface imagery

Camera-based scanning• Removal of perspective distortion

LCD panel as a projectorSimplifies calibration/rendering

15 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Future Work

Automatic object illuminationCamera-based detection of objects occluding projection

Improved scanning resolutionTrigger optical zoom, multi-camera super-resolution etc.

Continuous calibrationOnline calibration monitoring and refinement

16 T. Johnson, H. Fuchs, “A Unified Multi-Surface, Multi-Resolution Workspace”

Thank You

Funding support: ONR N00014-03-1-0589Funding support: ONR N00014-03-1-0589DARWARS Training Superiority program DARWARS Training Superiority program

VIRTE – Virtual Technologies and Environments programVIRTE – Virtual Technologies and Environments program