Nikolai Daskalov n Ivan Pushkarov n Angel Gavrailov n Atanas Boev

First version of terminal device

MOBILE3DTV

Project No. 216503

First version of terminal device

Nikolai Daskalov, Ivan Pushkarov, Angel Gavrailov, Atanas Boev

Abstract: This report describes the design and implementation of the first version of the terminal device. While not having the final form factor of a mobile device, this version includes key components of the final system. Namely, the chosen processing platform has been coupled with two types of auto-stereoscopic LCD. Key SW components for the processing and playing stereo video have been targeted as well. From platform perspective the goal of this version is to help the team to assess the system architecture of the proposed device, the feasibility of implementation, the performance of targeted components and the interoperability between them from both HW and SW point of view. The second goal is to verify the system interfaces (again both HW and SW) and to establish some performance metrics so to quantify the performance of the integrated displays and developed player.

Keywords: OMAP, DVB-H, auto-stereoscopic LCD

MOBILE3DTV D6.2

2

Executive Summary

The first version of the terminal device (prototype) has been developed and implemented. At that stage no final form factor has been targeted. Instead, key HW components which will be used in the final version of the terminal device, such as the processing platform and the auto-stereoscopic LCD have been coupled together.

Key SW components and tools have been identified and used to support the first version of the mobile terminal device. Those are the platform-specific operating system, file system, tool-chain, and a H.264 decoder.

Two different auto-stereoscopic LCDs from two different vendors have been connected to the system. For the first LCD, parallel 24bit interface has been used. For the second LCD, standard DVI interface has been used.

New HW and SW components have been developed in order to accomplish the targeted version. These include a new LCD daughter-card for interfacing the selected LCD to the platform; EVM and corresponding SW support for the two particular displays, and stereo image renderers. The completed version is capable of decoding and rendering stereoscopic video content. The input is stereo video streams, encoded by H.264 simulcast, and stored in file and the output is rendered on any of the integrated auto-stereoscopic LCD.

Performance metrics for the used components and encoding methods are defined so quantify the performance of the integrated components. This information will be used in the next stages for testing and performance validation.

MOBILE3DTV D6.2

3

Table of Contents

Platform and tools selection .......................................................................................... 4

Operating system ................................................................................................... 4 Video decoder application ...................................................................................... 4 Video renderer ........................................................................................................ 5 Utilities .................................................................................................................... 6

Selection of autostereoscopic display ........................................................................... 8

Mobile 3D display technologies .............................................................................. 8 Selection of display model ...................................................................................... 9

Stereo-video encoding method and decoding tools modifications ................................ 10

Decoder modification ............................................................................................ 10 Additional software components ............................................................................ 11

1.1.1 Video renderer module .......................................................................... 11

Interfacing the MasterImage auto-stereoscopic LCD .................................................... 12

System requirements ............................................................................................ 12 1.1.2 General information ............................................................................... 12 1.1.3 Description of interface and data transfer ............................................. 13 1.1.4 Design of interface daughter-card .......................................................... 14

Bring-up SW Project ............................................................................................. 18 OS Dependant modifications and new components ............................................. 24 Results.................................................................................................................. 24

Interfacing the NEC auto-stereoscopic LCD ................................................................. 25

System requirements ............................................................................................ 25 Interface................................................................................................................ 25 OS Dependant modifications and new components ............................................. 25 Results.................................................................................................................. 25

Performance metrics definition and measurements ...................................................... 26

Optical parameters of the displays – a comparative analysis ............................... 26 Video decoding performance ................................................................................ 29

References ................................................................................................................... 30

MOBILE3DTV D6.2

4

Platform and tools selection

Operating system

Currently, two major high-level operating systems are dominating the smart-phones

market: Symbian and Windows mobile. However, they both imply high cost of starting the

development, especially for the kernel level components such as display driver, DVB-H

receiver driver, etc. At the same time, embedded Linux is getting more and more popular in

the smart-phones market. It is even more adopted in portable devices such as internet

tablets and MIDs. Vendors such as Nokia, SEMC, and Motorola have announced support

of or migration to Linux-based devices.

For the MOBILE3STV project Linux seems to be the obvious choice as operating system

for the mobile demonstrator based on the following benefits:

Fast ramp-up of the SW team;

Abundance of example software;

Availability of reference drivers for display, SPI, I2C and other peripheral interfaces

to be used in the project;

Interoperability between different project work packages and their outcomes;

Open source community support

Video decoder application

3430 SDP is bundled with video decoder test application, capable to decode MPEG4,

H.264, MPEG2 and WM9 video streams with no audio synchronisation. The platform,

denoted as OpenMAX (OMX) implements optimized decoder running on the DSP and

using some vendor-specific hardware accelerators. OpenMAX is standard for components

implementing different multimedia processing functions. It is managed by Khronos group,

which manages also several other standards such as OpenGL. OpenMAX is well accepted

by major multimedia platforms and frameworks vendors. The major purpose of it is to

create unified environment for creating multimedia frameworks. The block diagram of the

decoder application and its components is shown i Figure 1. It allows decoding video-

streams read from files and rendering on the LCD display. Typical example of the

command line to start the decoding is shown below:

./VideoDisplayTest 6 omx/patterns/*.h264 blank.vop 640 480 4 1 90 0 0 100 100 1 70 LCD

MOBILE3DTV D6.2

5

Figure. 1: Block diagram of video decode architecture in OMAP3430 SDP.

Video renderer

The video decoder application has no support for auto-stereoscopic LCD and the LCD

controller of OMAP3430 has no internal logic to interleave the data from the left and right

channels of the decoded sequence in order to be displayed on the selected LCD.

Therefore, an additional software video renderer has been developed. We have avoided

the straightforward implementation of this interleaving on the CPU, as such

implementation would result in high CPU load and subsequent system performance

degradation. Instead, we opted for a separate solution, as described in Chapter 3.3. The

integration of this interleaving implementation into the Linux frame buffer is in progress.

MOBILE3DTV D6.2

6

Utilities

The utilities include conversion software between different stereo video formats. The first

selected stereoscopic display supports and interleaved input frame while the second

selected display supports a side-by-side format. Therefore, the following conversion

formats are supported: from two-channel to side-by-side and back. From two-channel to

interleaved (intedigitized) frames and back. From side-by-side to interleaved frames and

back. A screen shot of the program interface is shown on Figure 2.

Figure 3 illustrates converted screen shot of the converted video for the NEC display

module while Figure 4 illustrates converted screen shot of the converted video for the

MasterImage display module.

The software takes the two videos and can create double non interlaced video (Figure 3),

or interlaced (Figure 4). See Figure 21 for an illustration of the interdigitization process.

Figure 2: Interface of the converter tool

MOBILE3DTV D6.2

7

Figure 3: Converted video screen shot for NEC module.

Figure 4: Converted video screen shot for MasterImage module.

MOBILE3DTV D6.2

8

Selection of autostereoscopic display

Mobile 3D display technologies

Currently, there is a wide range of 3D display technologies [1], [2], but not all of them are

appropriate for mobile use. For example, wearing glasses to aid the 3D perception of a

mobile device is highly inconvenient. The limitations of a mobile device, such as screen

size, CPU power and battery life limit the choice of a suitable 3D display technology.

Another important factor is backward compatibility – a mobile 3D display should have the

ability to be switched back to “2D mode” when 3D content is not available.

Autostereoscopic displays form a class of 3D displays which create 3D effect without

requiring the observer to wear special glasses. Such displays use dedicated optical

elements aligned on the surface of the screen so to ensure that the observer sees different

images with each eye. Typically, autostereoscopic displays are capable of presenting

multiple views to the observer, each one seen from a particular viewing angle along the

horizontal direction. However, the number of views comes at the expense of resolution and

brightness loss – and both are limited on a small screen, battery driven mobile device. As

mobile devices are normally watched by only one observer, two independent views are

sufficient for satisfactory 3D perception. At the moment, there are only a few vendors with

announced prototypes of 3D displays, targeted for mobile devices [3], [4], [5]. All of them

are two-view, TFT-based autostereoscopic displays [6].

The basic operational principle of an autostereoscopic display is to “cast” different images

towards each eye of the observer. This is done by a special optical layer, additionally

mounted on the screen surface which redirects the light passing through it. There are two

common types of optical filters – lenticular sheet [7] which works by refracting the light,

and parallax barrier [8] which works by blocking the light in certain directions. In both

cases, the intensity of the light rays passing through the filter changes as a function of the

angle, as if the light is directionally projected. These two technologies are shown in Figure

5.

MOBILE3DTV D6.2

9

a) b)

Figure 5: Technologies for redirecting the light in an autostereoscopic display. a) lenticular sheet, b) switchable parallax barrier

The main advantage of parallax barrier is the ability to switch it off, so the display works in

2D mode, thus providing backwards compatibility with 2D content. The main disadvantage

of the parallax barrier is that it blocks part of the light, resulting in a lowered brightness of

the display. In order to compensate for that, one needs an extra bright backlight, which

decreases the battery life. Comparing the cost of developing and manufacturing parallax

barrier and lenticular sheet, the former is much cheaper than the latter.

Selection of display model

Following the requirement for backwards compatibility, we initially focused on a parallax

barrier display. One of the few commercially available parallax barrier models was

Stereoscopic 3D LCD model MB403M0117135 produced by MasterImage, which is a

4.3”WVGA (800px x 480px) transmissive LCD display. Additional feature of this display is

the ability to switch the barrier between horizontal and vertical mode, which allows

landscape and portrait mode of 3D operation.

Due to the operation principle, a parallax barrier reduces the horizontal resolution twice

when operating in 3D mode. For the MasterImage display this results in the following set of

resolutions: 800x480 in 2D mode, 400x480 in landscape 3D mode, and 240x800 in portrait

3D mode.

In the mean time, we also considered another display – 3D HDDP LCD produced by NEC

[5]. It uses lenticular sheet, but allows switching of the display between 2D and 3D mode,

by simply sending the same 2D image to both channels. Also, the selection of size and

order of the subpixel components results in the same resolution both in 2D or 3D mode.

The colour components of a pixel are ordered vertically, instead of horizontally, which

suppresses angle-dependant colour artefacts.

R G B R G R G B

1 2 1 2 1 2 1R 1R

B R G B R G B R G

RR RR

LL LL

MOBILE3DTV D6.2

10

The pixels are twice more dense in horizontal than in vertical direction, thus called by the

developers from NEC Lab, Horizontal Double Density Pixel (HDDP) arrangement. It has

been described in [5]. Additionally, the display is able to work both in transmissive or

reflective mode, which ensures 3D operation in wide range or lighting condition. The

HDDP display is not yet commercially available. However, we received a pre-production

sample from NEC, which is 3.1 inch, with 427x240 resolution both in 2D and 3D mode.

As each 3D display model has advantages and disadvantages of their own, it was decided

to mount and interface both of them and to compare their optical quality as part of a

working system.

Stereo-video encoding method and decoding tools modifications

Decoder modification

Based on the results from D 2.2 report and considering capabilities of the chosen

developing platform and the existing H.264 decoder, at that stage we have focused on two

encoding approaches and respectively their decoding parts implementations, namely

H.264 Simulcast and H.264 SEI Message.

In the simulcast approach, we put left and right part of the stereo image side by side. The

result is H.264 stream with double frame size. Decoding this stream is equivalent to

decoding the stream with double frame size. If we are able to double the existing decoder

frame size, we can easily implement H.264 Simulcast.

When using H.264 SEI message, left and right views are interlaced, resulting in a stream

with double frame rate. From implementation point of view, decoding such stream is

equivalent to decoding mono video stream with double frame rate, and de-interlacing each

two frames in order to create stereoscopic view.

To implement H.264 Simulcast we need to have a decoder able to decode frames with

double frame size, preserving frame rate while implementing H.264 SEI message we need

a decoder able to decode with double frame rate, preserving frame size.

MOBILE3DTV D6.2

11

In order to implement H.264 SEI Message we need to modify these features of the existing

decoder:

decode SEI message

de-interlacing left and right views

constructing side-by-side frame for the display subsystem According to the comparative study of different encoding approaches described in

Deliverable 2.2, H.264 SEI Message gives about 35% bitrate saving vs. H.264 Simulcast

and is almost identical to MVC coding, which makes H.264 SEI message a strong favorite

for practical implementation of decoding stereo video streams.

For the first version of the terminal device, the H.264 simulcast approach has been

implemented, while the SEI message implementation is still under investigation. Though

inferior, the simulcast approach has been chosen due to the purpose of that version, i.e. to

be used in the subjective tests. Simulcast allows manipulating and experimenting with

videos of different quality (varying quality factors such as bitrate, framerate, transmission

modes, etc.) – simply code and decode the streams and put the left and right channels

side by side to be played further in the subjective tests.

Additional software components

1.1.1 Video renderer module

This component combines left and right views from the output of the H.264 decoder to

produce the format required for driving the respective auto-stereoscopic LCD display. In

the case of the MasterImage display this means converting the side by side video frames

to spatially interleaved (interdigitized) frames. In the case of NEC display driven by DVI

input the side by side frames are send directly with no format conversion.

MOBILE3DTV D6.2

12

Interfacing the MasterImage auto-stereoscopic LCD

System requirements

1.1.2 General information

One of the two interfaced displays is produced by the Korean company MasterImage. It

uses a parallax barrier to implement the auto-stereoscopic effect. The display consists of

two LCD modules, assembled together in one compound 3-D LCD module. The back

module (briefly “Main display”) is a common color LCD display of certain resolution. In front

of it, there is the second LCD serving as an optical filter (briefly “filtering display”). While

switched on, its role is to block the light to some directions and thus to cast two different

views to the eyes (left and right views) – hence the name „parallax barrier‟. When switched

off, it plays no role and the display module works in 2D mode. Figure 8 illustrates the

positioning of the two displays.

Figure 8: Arrangement of main and filtering displays

The main display characteristics are as follows:

Size 4.3 inches;

Resolution WVGA (800 x 480 pixels);

RGB Interface with 1 pixel / clock;

8-bit color depth, 16777216 colors;

10 LEDs back-light;

High luminance and contrast ratio, low reflection.

MOBILE3DTV D6.2

13

The filtering display features are as follows

Target display size 4.3 inch;

Target display resolution WVGA (800 x 480 pixels);

1.1.3 Description of interface and data transfer

The interface is 24-bit parallel data transfer interface. We connect the daughter board to

the OMAP3430 gpios trough the OPAM3430SDP.

Each pixel consists of three dots for each color - Red, Green and Blue. Each dot is

controlled by 8 bits (i.e. 8-bit color depth). This yields 16777216 different color states per

pixel. The data is transferred parallel trough 24 lines, controlled by the PCLK (pixel clock).

Figure 10 shows the time diagram, and the data transferred for one pixel clock pulse. For

each pixel clock pulse 24 bits are transferred.

Figure 10: Data transfer

MOBILE3DTV D6.2

14

The input data format of the 3D LCD is shown in Figure 11.

The main requirements concerning the timing of the 3-D LCD are shown in Table 1.

Table 1: Main Timing Requirements

Parameter Min Typical Max Unit

VSYNC - 60 75 Hz

HSYNC - 31,5 39,4 KHz

PCLK frequency 10 33,5 50 MHz

PCLK pulse width 8 - - ns

The targeted platform supports the upper values in order to be compatible with the 3-D

LCD. There are few more timings, but they are directly connected to the main listed above.

Figure 11: 3-D LCD input data format

1.1.4 Design of interface daughter-card

MOBILE3DTV D6.2

15

The daughter-card must provide the proper functionality of the 3-D LCD.

The main goals in the design of the daughter-card design are as follows

To connect the 5V supply voltage to the requested components;

To generate the supply voltage for the back-light of the 3-D LCD (30 – 34V);

To provide the proper power on/off sequence of the 3-D LCD;

To level-shift the signals from the platform to the Main display (1.8 to 3.3 V);

To generate the supply voltage of 3.3V (for the PIC controller, level-shifters,

power supply of the logic of the main display, etc);

To provide popper control of the PIC controller;

To fit the existing platform hardware.

A simple block diagram is depicted in Figure 12. The shown blocks are as follows

Board connector schematic – this is the connector to the platform interface.

Level shifter schematic – this block level-shifts the signals coming from the platform (the

platform display gpios) from 1.8 V to 3.3 V for logical high level.

LCD connector schematic – this block consists of the connector to the 3-D LCD and the

scheme that generates the power supply voltage of the back-light from 5 V input voltage.

3V3 power schematic – this block generates the 3.3 V voltage needed for the schematic.

LCD power sequence enabling schematic – this part provides the proper on/off power

sequence to prevent bad/faulty functionality or damage to 3-D LCD.

TN LCD control schematic – this part controls the filtering display, therefore controls the

mode of the 3D LCD.

MOBILE3DTV D6.2

16

Figure 12: Block diagram of the daughter-card

The board has been designed so to fit to the existing platform hardware. It has been

provided with stable fixation between the different connectors, and stable and safe

positioning of the 3-D LCD considering the manufacturer recommendation concerning the

exploitation.

The technology of the MasterImage display module is illustrated by the following photos.

Figure 13 shows a stereo frame to be displayed on the interfaced card. The picture is

interlaced so to create the desired 3D effect on the 3D module. Figure 14 and Figure 15

show the daughter-card hosting the 3D LCD as connected to the OMAP3430SDP.

MOBILE3DTV D6.2

17

Figure 13: Interlaced photo for the test.

Figure 14: The display working in 3-d mode

MOBILE3DTV D6.2

18

Figure 15: The display showing 3-d Picture with turned off 3-D mode

In Figure 14, the 3D effect has been activated (parallax barrier switched on) while in Figure

15 the barrier has been switched off.

Bring-up SW Project

The bring-up software project was developed in order to implement initial testing of the

display without all the complexity of Linux operating system. It could be divided into two

major parts:

Part A: Software, related to the TFT display

This software provides the following functions:

Basic system initialization (OMAP3430 SDP)

Display subsystem initialization

Create and display a test pattern

Picture loading

Picture display – Landscape and Portrait mode

MOBILE3DTV D6.2

19

Part B: Software, related to the stereoscopic overlay

This software is intended to provide the correct driving signals for the stereoscopic layer. It

is run on a separate embedded microcontroller in order to simplify the main software and

to provide independent and steady operation of the stereoscopic overlay.

Part A

The software for TFT display bring-up was created in the environment of CodeComposer

Studio. Although Linux would provide more flexibility in terms of interfaces, file loading,

etc., it is much easier to control certain subsystem (that is, Display Subsystem) parameters

without it. Additional benefit in this approach is that we can use JTAG emulator for

debugging.

There are two modules of the TFT bring-up software

Basic functional evaluation: Mainly for test of colours and display timings

“Normal use” functional evaluation: Display of (processed) stereoscopic images

The two block diagrams are shown in Figure 16 and 17 respectively

Figure 16: Block diagram of the Basic functional evaluation

Start

Initialize SDP system

Initialize Display Subsystem

Create & isplay test patterns

End

MOBILE3DTV D6.2

20

Figure 17: Block diagram of the “Normal use” functional evaluation

The blocks illustrate the following functions Initialize SDP subsystem

This functional block takes care of basic initialization steps, including but not limited to:

OMAP3430 low-level initialization: clocks, stacks, interrupts, etc.

OMAP3430 pads initialization

OMAP3430 SDP power supply initialization – i.e. Triton3 set up

Initialize Display subsystem

This functional block takes care of the OMAP3430 DSS initialization. Main functions

performed here are:

Display controller initialization: Type of display, Display size, Number of colors, Data

bus width, Pixel clock rate, VSYNC/HSYNC Timings

Graphics subsystem initialization: Number of layers, Transparency, Image

dimensions, Rotation, etc.

Start

Initialize SDP system

Initialize Display Subsystem

Load image

End

Process & Display Stereo image

MOBILE3DTV D6.2

21

Load image

In order to simplify the code, the image is hard-coded in the form of a pre-initialized C

array and is used as-is inside the test software. For the purpose of creating the C-array, a

small console application was created, which converts a JPG file into pre-initialized C-

array

Create and display test patterns

In order to strictly check any identify possible problems with the TFT Display, its settings or

settings of the OMAP3430 DSS, test patterns with the appropriate parameters are

generated (Figure 18):

Figure 18: Test patterns

Using these patterns, problem in routing of the TFT control signals (PCLK, VSYNC, DE)

and routing of colour driving signals were corrected.

Process and display of a Stereo Image

This functional block is responsible for processing the image data in a form, suitable for

displaying.

If the input is an interlaced Image, i.e. two images – for the left and the right eye

respectively, are already interlaced, there is no need of further processing. Example of

such an image is shown in Figure 19.

MOBILE3DTV D6.2

22

Figure 19: Interlaced left and right views to be displayed on the MasterImage display

If the input image is a side-by-side image of the left and right views, it has to be processed

and the two parts to be interlaced. Such image is shown in Figure 20.

Figure 20: Left and right views as put side-by-side

For the purpose of processing, two DMA channels were used in order to simultaneously

transfer the two halves of the original picture into a final interlaced image (see Figure 21).

MOBILE3DTV D6.2

23

Figure 21: DMA transfer of data from side-by-side format to interlaced frame

By using two DMA channels, the two halves are automatically transferred to the correct

places with no complex loops. The DMA channels use Double-indexing (i.e. indexing

within an Element Frame and an indexing of Frames), which is a key feature to automate

the entire process. The DMA indexing parameters are shown in Table 2:

Table 2: DMA indexing parameters

where:

X – Horizontal resolution of the output image (800 pixels in this particular case)

Y – Vertical resolution of the output image (480 pixels in this particular case)

DMA2 Channel Settings

Source Start Address Address of the (400, 0) pixel

Source Element indexing X

Source Frame indexing -(Y-1)*X+401

Destination Start Address Address of the (1, 0) pixel

Destination Element indexing X

Destination Frame indexing -(Y-1)*X+2

MOBILE3DTV D6.2

24

Part B

The stereoscopic overlay (the parallax barrier) is basically an LCD display, whose pixels

are driven in a specific way in order to achieve a row visibility blocking effect either in the

horizontal or in vertical directions.

OS Dependant modifications and new components

Regarding the Linux OS, changes on kernel display drivers and frame buffer have to be

accomplished. These changes have been implemented for the L12.20 baseline release of

the platform software.

Results

The MasterImage auto-stereoscopic display was successfully integrated to OMAP3430

SDP platform. A special interface daughter card, containing all needed logic and level

translation was developed. Stand-alone bring-up test project was developed in order to

verify the HW and SW components in the system. This stand-alone project allows loading

and display auto-stereoscopic images on the display, prepared by the interleaving tool,

developed for the project. The perceived quality of the stereo images using MasterImage

LCD is good. In addition, the team has been modifying the 3430 Linux frame buffer and

display driver to be included in the next release of the platform software.

MOBILE3DTV D6.2

25

Interfacing the NEC auto-stereoscopic LCD

System requirements

The NEC module is a lenticular-based auto-stereoscopic display, delivered as stand- alone

module, interfacing with the platform trough a DVI interface. The display consists of two

parts: ordinary LCD with certain resolution and micro-lenses filter positioned over the LCD.

That filter (i.e. the lenticular sheet) separates the left and right channels for each eye. 2D

mode is achieved by delivering the same content to the two channels. Otherwise, by

delivering stereo content, the display works in 3D mode. A sample video frame is shown in

Figure 25.

Figure 25: Side by side left and right frames for the NEC display

The input frame to be displayed is of resolution 852 x 240 pixels formed by two side-by-

side pictures with resolution 427 x 240 pixels each.

Interface

At that stage, we have access to the display module trough a DVI interface.

OS Dependant modifications and new components

In order to run NEC Module on SDP (OMAP3430 Zoom) a new kernel image was created

supporting the 852 x 240 pixels resolution. A Linux-based player application working on the

SDP deliver the video frames to the display.

Results

As the OMAP3430 Zoom platform supports the DVI interface to connect the NEC display

module no additional HW components were needed. After modification in the kernel driver

and frame buffer, stereoscopic images can be shown on the display. The perceived image

MOBILE3DTV D6.2

26

quality can be rated as excellent. In the next section, the two displays have been

compared for their 3D optical characteristics.

Performance metrics definition and measurements

Optical parameters of the displays – a comparative analysis

The measurement of crosstalk and angular luminance profile was based on the

methodology, proposed in [9]. Two displays were measured - MB403M0117135 from

MasterImage (designated here as “MasterImage”) and a 3.1” prototype of HDDP 3D LCD

produced by NEC (designated here as “NEC”). A number of test images were prepared,

visualized on each display, and photographed from different angles using digital camera.

The parameters of the camera were set in such a way, so the intensity of the captured

images does not cause saturation and is mostly in the linear range of the camera sensor,

as seen in Figure 31.

Two experiments were made, and a number of observation points were chosen for each

experiment. A set of test images were prepared. Each test image is a stereo-image where

each channel contains all pixels set at a certain brightness level. For example, in test

image “L0R64” all pixels from the left channel have value 0 and all pixels in the right

channel have value 64. Each test image was photographed from each point. The

observation points were restricted onto a plane perpendicular to the display surface as

shown in Figure 26. When using a 3D display, the eyes of an observer usually appear

close to that plane.

Figure 26: Selection of plane for measurements

At each observation point, all measured results were scaled from 0 to 1, where 0

corresponds to the value measured when the display was completely black, and 1

Plane of the measurements

MOBILE3DTV D6.2

27

corresponds to the value measured when all pixels from both channels had maximal

brightness. Additionally, the measurements were performed in a dark room.

In the first experiment we measured the luminance profile or each display. The observation

points were chosen along a line at typical observation distance from the display as shown

in Figure 27. Test images L0R0, L0R255, L255R0 and L200R255 were photographed.

Figure 27: Observation points for measuring angular dependant luminance profile

The results, shown in Figure 28, illustrate the main difference between the display

technologies used. The left and right channel of the NEC display have clearly separated

observation zones, where each channel is predominantly seen from a set of angles to the

left and to the right of the display respectively. The MasterImage display uses parallax

barrier, which creates a number of interleaved “left channel” and “right channel” visibility

zones.

a) b)

Figure 28: Luminance profiles: a) NEC, b) MasterImage

The approximate position of visibility zones of each channel is shown in Figure 29. For the

NEC display, stereoscopic image can be perceived from a set of angles, as long as each

eye of the observer appears in the designated zone. The observation zones of

MasterImage are much narrower, and as a result, 3D perception is possible from a number

of tight “sweet spots” in front of the display. The advantage of the second configuration is

that the MasterImage display can provide 3D image to a number of users simultaneously.

However, in order to have a satisfactory 3D perception, a user of MasterImage has to

Front of Display

Back of display

Observation distance(~30cm)

Observation points

-50 … 50 degrees

MOBILE3DTV D6.2

28

carefully choose a position in front of the display, and try to avoid positions where a

preudoscopic (reversed stereo) image will be seen.

a) b)

Figure 29: Vizibility zones left and right channels: a) NEC, b) MasterImage

The second experiment measured the crosstalk of each display. Two observation points,

mimicking the typical position of the eyes of an observer, as depicted in Figure 30. It is

assumed, that this is the “best case” scenario, which should result in the lowest crosstalk.

We prepared 34 test images for each display – L0R0, L0R16, L0R32…L0R240, L0R255,

L16R0, R32R0…L240R0, L255R0, L255R255.

Figure 30: Observation points for measuring the crosstalk

The results are shown in Figure 31. The crosstalk is nearly symmetrical across channels

for each display. The relative crosstalk of NEC is about 4%, while the crosstalk of

MasterImage is more that twice of that, 9%. It should be noted, however, that due to the

narrower observation zones of MasterImage, the measurement of crosstalk for that display

requires higher precision and might be less accurate. Still, the visual appearance of the

displays suggests lower crosstalk values for NEC.

RL

screen

right

eye

left

eye

R

L

screen

right

eye

left

eye

L LRL

R

R

Front of Display

Back of display

Observation distance (~30cm)

Interpupillary distance (~6.3cm)

Typical observationpoints

MOBILE3DTV D6.2

29

a) b)

c) d)

Figure 31: Crosstalk measurements for 3D displays: a) left channel crosstalk of NEC, b) right channel crosstalk of NEC, c) left channel crosstalk of MasterImage, d) right channel crosstalk of MasterImage

We summarize the results of the optical measurements in the following table:

Table 3: summary of the optical measurement results.

Parameter MasterImage NEC

3D crosstalk 9,00% 4,00%

Viewing distance ~10cm-90cm ~20-40cm, 30cm optimal

Viewing freedom ~7 degrees (for interpupilar

distance of 6.5cm)

1.2 degrees

Luminance profile See Figure 28b See Figure 28a

Video decoding performance

According to the conclusion of the video encoding method (chapter 2.1), the most

promising methods are H.264 simulcast or H.264 SEI method. For both methods, knowing

that for the moment the best performance we can get out of the decoder is VGA@30fps,

we were able to decode stereo sequences at HVGA@30fps, which is better than the

minimum requirements defined in the D6.1 report (QWVGA@25fps ), but less than the

target for the best decoding for the project (HWGA@30fps). Because the actual decoding

is happening on the DSP in OMAP3430, the CPU load for decoding is quite small –

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 32 64 96 128 160 192 224 255

L out

R out

Input value

Inte

nsi

ty (s

cale

d)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 32 64 96 128 160 192 224 255

L out

R out

Input value

Inte

nsi

ty (s

cale

d)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

16

32

48

64

80

96

11

2

12

8

14

4

16

0

17

6

19

2

20

8

22

4

24

0

25

5L out

R out

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 32 64 96 128 160 192 224 255

L out

R out

MOBILE3DTV D6.2

30

average less than 25%. The re-formatting component is not consuming CPU resources

because the DMA based implementation.

References

[1] L. Onural, T. Sikora, J. Ostermann, A. Smolic, M. R. Civanlar and J. Watson: “An Assessment of 3DTV Technologies,” NAB Broadcast Engineering Conference Proceedings 2006, pp. 456-467, Las Vegas, USA, April 2006.

[2] P. Surman, I. Sexton, R. Bates, W. K. Lee, K. Hopf, and T. Koukoulas: “Latest Developments in a Multi-User 3D Display,” in Proc. SPIE Vol. 6016, Three-Dimensional TV, Video, and Display IV , 2005.

[3] Sharp Laboratories of Europe, website, http://www.sle.sharp.co.uk/research/optical_imaging/3d_research.php

[4] G. J. Woodgate, J. Harrold, “Autostereoscopic display technology for mobile 3DTV applications”, in Proc. SPIE Vol.6490A-19 (Stereoscopic Displays and Applications XVIII), 2007

[5] S.Uehara, T.Hiroya, H. Kusanagi; K. Shigemura, H.Asada, “1-inch diagonal transreflective 2D and 3D LCD with HDDP arrangement”, in Proc. SPIE-IS&T Electronic Imaging 2008, Stereoscopic Displays and Applications XIX, Vol. 6803, San Jose, USA, January 2008

[6] I. Sexton, P. Surman, ”Stereoscopic and autostereoscopic display systems.”, in IEEE Signal Processing Magazine, pp. 85-99, 1999

[7] C. Van Berkel and J. Clarke, “Characterisation and optimisation of 3D-LCD module design”, in Proc. SPIE Vol. 2653, Stereoscopic Displays and Virtual Reality Systems IV, (Fisher, Merritt, Bolas, edts.), p. 179-186, May 1997

[8] W. Izerman et al., “Design of 2d/3d switchable displays,” in Proc of the SID, volume 36, Issue 1, pp. 98-101, May 2005

[9] Boev, A., A. Gotchev and K. Egiazarian, “Crosstalk measurement methodology for auto-stereoscopic screens”, Proc. of 3DTV Con, Kos, Greece, 2007

Mobile 3DTV Content Delivery Optimization over DVB-H System

MOBILE3DTV - Mobile 3DTV Content Delivery Optimization over DVB-H System - is a three-yearproject which started in January 2008. The project is partly funded by the European Union 7th

RTD Framework Programme in the context of the Information & Communication Technology (ICT)Cooperation Theme.

The main objective of MOBILE3DTV is to demonstrate the viability of the new technology ofmobile 3DTV. The project develops a technology demonstration system for the creation andcoding of 3D video content, its delivery over DVB-H and display on a mobile device, equippedwith an auto-stereoscopic display.

The MOBILE3DTV consortium is formed by three universities, a public research institute and twoSMEs from Finland, Germany, Turkey, and Bulgaria. Partners span diverse yet complementaryexpertise in the areas of 3D content creation and coding, error resilient transmission, userstudies, visual quality enhancement and project management.

For further information about the project, please visit www.mobile3dtv.eu.

Tuotekehitys Oy TamlinkProject coordinator

FINLAND

Tampereen Teknillinen Yliopisto

Visual quality enhancement,

Scientific coordinator

FINLAND

Fraunhofer Gesellschaft zur Förderung der angewandten Forschung e.V

Middle East Technical UniversityError resilient transmission

TURKEY

Stereo video content creation and coding

GERMANY

Technische Universität IlmenauDesign and execution of subjective tests

GERMANY

MM Solutions Ltd. Design of prototype terminal device

BULGARIA

MOBILE3DTV project has received funding from the European Community’s ICT programme in the context of theSeventh Framework Programme (FP7/2007-2011) under grant agreement n° 216503. This document reflects onlythe authors’ views and the Community or other project partners are not liable for any use that may be made of theinformation contained therein.