Scaleable 3D modelsScaleable 3D modelsLuc Van Gool

Large-scale 3D modeling

� Clear trend towards ever more massive 3D models, and

dynamic capture (4D)

� SecondLife, Google Earth, games like the Runaway, 3D

navigation systems, Digital 3D Cartography (AGIV),

digitalisation of museum content (EDL), …digitalisation of museum content (EDL), …

� Developers of acquisition, editing, transmission, rendering,

and presentation technologies are struggling to match

� Problem is huge, as massive, high-quality modelling also

requires bridging the infamous semantic gap

Large-scale 3D modeling

� ACQUISITIONACQUISITION

3D acquisition taxonomy

uni-directional multi-directional

passive

uni-directional multi-directional

active

3D acquisition methods

LIDARStereo

Str. from motion

Shape-from

- texture

- contour

- silhouettes

- defocus

- shading

Line scanning

Structured light

Photom. stereo

3D acquisition taxonomy

uni-directional multi-directional

passive

uni-directional multi-directional

active

3D acquisition methods

LIDARStereo

Str. from motion

Shape-from

- texture

- contour

- silhouettes

- defocus

- shading

Line scanning

Structured light

Photom. stereo

Underlying principles

Structure from motion

Structured light

TRIANGULATION

Structured light

LIDAR TIME-OF-FLIGHT

Stereo

The underlying principle is “triangulation” :

�

Triangulation � 3D measurements

Active triangulation

8

Projector

Camera

Active triangulation

Camera image

Active triangulation

Need for registration...

Has to be automatic for

large-scale work

Time-of-flight

Cyrax

Distance = C x T ÷2

Time-of-flight

Cyrax

Laser sweeps over surface

No correspondences needed

Large-scale 3D capture

� Speed becomes essential

� … fast calculations (e.g. helped by GPU)

� … but also easy setup and data capture

� Increasingly systems that are based on comsumer HW� Increasingly systems that are based on comsumer HW

� … and intelligent modeling tools

� … and automatic data capture

Tracking and

Calibration

Densedepth

estimation

3Dsurface

modeling

3D acquisition / structure-from-motion

Points and cameras 3D modelsDepth map

Shape

3D acquisition / structure-from-motion

Shapefrom Video(KUL)

3D acquisition / structure-from-motion

ShapeShapefrom Stills(KUL)

ARC – Free 3D webservice

ARCARCARCARCAutomatic

Reconstruction Reconstruction

Conduit

non-commercial

use is free

www.arc3d.be

Automatic photo mining

18

(kooaba)

Use in movies / e.g. King Arthur (2003)

Dublin: The Hollywood Valley (Eyetronics)

19

Area B

Use in movies / e.g. King Arthur (2003)

20

Combined areas

Use in movies / e.g. King Arthur (2003)

21

Use in movies / e.g. King Arthur (2003)

Use in movies / e.g. King Arthur (2003)

One-shot

ShapeCam

3D acquisition / structured light

3D acquisition / structured light

Jackie Chan

Latest Latest RushHour Movie

(Eyetronics)

3D acquisition / structured light

MLB

1200 players1200 playersfor gaming

(Eyetronics)

3D acquisition / structured light

MLB

1200 players1200 playersfor gaming

(Eyetronics)

3D acquisition / structured light

MLB

1200 players1200 playersfor gaming

(Eyetronics)

4D acquisition / structured light

3D snapshots in fast succession (KUL)

4D acquisition / structured light

30

(KUL &

ETH Zurich)

4D acquisition / structured light

Real-time 3Dextraction, incl.texture

(KUL &

ETH Zurich)

3D acquisition / structured light

LIDAR

Chevron hydrocarbon plant

LIDAR

Cyrax / Leica

LIDAR

Large-scale 3D modeling

� EDITING / REPRESENTATIONEDITING / REPRESENTATION

LIDAR

(Cyrax / Leica)

LIDAR

39

LIDAR

� 400’x500’ area

� 10 vessels; 5 pumps

� 6,000 objects

� 81 scans from 30 tripod locations

� Cyrax field time = 50 hrs

Measuring for cartographic surveying

Large-scale 3Dmeasurements

Efficiency of the interface !

Procedural modeling

Modeling using rules from a style-specific shape grammar

(ETH Zurich & Procedural)

Large-scale 3D modeling

� STREAMING / RENDERINGSTREAMING / RENDERING

QoS

• Bandwith of channel and type of platform!

• Local, over network, wireless, ...?

• PC cluster, PC, PDA, mobile phone, ...?

• Level-of-Detail mechanisms• Level-of-Detail mechanisms

• Compact representations

• Exploit differential sensitivity HVS

• 3D meshes or... point splashing, IBR, ...?

A4MC3

WIFI

connected

community community

with PDAs

(EDM, IMEC, VUB)

MobiComp

(Un. Kent &&

Bologna )

MeshGrid

A scalable coding scheme for 3D models that:• enables progressive transmission• offers quality, resolution scalability and region-of-interest coding• guarantees a minimum distortion at any bit-rate• provides resilience against transmission errors

MeshGrid

A MeshGrid model consists of a connectivity-wireframe (CW) and a reference-grid (RG) that acts as the reference-system (RS) of the model.

MeshGrid

G1

V

G2

G1

G2

V

1

1 2

, with [0,1)G V

offset offsetG G

= ∈

uuuur

uuuuur

Vertex offset is a relative value:

1 1 2G V G G offset=

uuuur uuuuur�

Update vertex position whengrid is deformed or animated:

MeshGrid

� Volumetric Morphing

� Volumetric Animation

� Scalability, Regions Of Interest Coding

(a) (b) (c)

� Volumetric Animation

Image-based Rendering

(KUL)

(Un. Bonn)

51(EDM)

Large-scale 3D modeling

� PRESENTATIONPRESENTATION

Shuttered Walls & Synchronized Cameras

Blue-C project

walls transparent –acquisition, no projection

walls opaque –no acquisition, projection

(ETH Zurich)

Blue-C project

CyberWalk project

Towards holistic vision – 3D & recognition

56