Sukhanov S. 3D content production 1

3D content production

St. Petersburg Electrotechnical University, St. Petersburg, Russia

Sergey A. Sukhanov

Sukhanov S. 3D content production 2

Outline

1.What is 3D

2.3D Technology

3.The need for 3D video content

4.Live camera capture

5.2D to 3D conversion

6.Depth maps and depth image

based rendering (DIBR) method

7.Conclusion

Sukhanov S. 3D content production 3

What is 3D?

Sukhanov S. 3D content production 4

3D Technology

Sukhanov S. 3D content production 5

3D Content Production

Three main approaches to creating 3-D content:

• live camera capture

• computer generated imagery (CGI)

• 2D to 3D conversion

Sukhanov S. 3D content production 6

Live camera captureLive camera capture: recording stereo sequences of images simultaneously

A depth-range camera Special 3D cameras (single body)

Special 3D cameras (single body)side-by-side rig

side-by-side rig

Sukhanov S. 3D content production 7

Depth map

Regular 2D color videoAccompanying 8-bit depth-images with the

same spatio-temporal resolution

The depth-images are normalized to a near clipping plane Znear and a far clipping plane Zfar.

Sukhanov S. 3D content production 8

2D to 3D conversion2D to 3D conversion: using monocular depth cues obtained froma 2D sequence to generate the equivalent 3D sequence

1. Cutting out pieces of the image and nudging

them around

2. Projecting an image on a 3D model

3. Turning motion parallax into stereoscopy

4. Using depth maps and DIBR process

- challenging task because of automatic segmentation

- main purpose: to generate additional views

Sukhanov S. 3D content production 9

Depth maps and depth image based rendering (DIBR)

Sukhanov S. 3D content production 10

Depth maps generation

Block-matching based motion estimation4x4 block

Using previous frame as reference

Sukhanov S. 3D content production 11

Depth maps generation

- identifies the chromatic homogeneous regions- group together pixels basing on their likeness

Color Segmentation

Sukhanov S. 3D content production 12

Depth maps generationFusion

- to enhance the block-based depth map- to eliminate the staircase effort- to assign better depth values in each region

Sukhanov S. 3D content production 13

Depth image based rendering

Camera configuration for generation of virtual stereoscopic images.

3D Image Warping

- projects each pixel of the real view into the 3D world- re-project these pixels back to 2D image of the virtual view

Sukhanov S. 3D content production 14

Hole-Filling

Depth image based rendering

- occlusion: two different pixels of thereal view image are warped to the same location in the virtualview- disocclusion: is due tothe occluded area in the real view may become visible in thevirtual view

Left-eye view image created by 3D image warping with holesdue to disocculsion

Enlarged left-eye view image with stripe distortion after linearinterpolation based hole-filling

Sukhanov S. 3D content production 15

Conclusion

Rising demand on new 3D content

• live camera capture

• 2D to 3D conversion

Advantages of proposed method:- Automation- use of a single view of the scene- effectiveness

Drawbacks:- high computational requirement- not suitable for real-time applications

Sukhanov S. 3D content production 16

Thank you for your attention!