Ms Synchronisation

7/31/2019 Ms Synchronisation

1/38

E0262 MIS - Multimedia Pla back S stems

Prof Pallapa Venkataram,

Electrical Communication Engineering,

Indian Institute of Science,

Bangalore 560012, India

Multimedia StreamsSynchronization


2/38


Synchronization in multimedia systems referes to thetemporal relations between media stream data units in

multimedia system.

Time dependent media data units: Temporal relations

between consecutive units of media stream exist.Time independent media data unit is any kind of

traditional media like text and image.


3/38


Classification of Media Use


4/38


Multimedia Synchronization The process of maintaining the temporal order of one or several media

streams

Relations relevant for audio and video exchange:

Intra-stream synchronization: Voice samples and video frames need

to arrive in time at the receiver before display or playout time to

maintain the continuity of playback.

Inter-stream synchronizationis needed to present voice and video

samples in a certain relation.

Spatial synchronization:Participants in a multimedia conference

should receive audio-visual data at the same time although

geographically distributed.

Life & Synthetic synchronization


5/38


Basic Synchronization Issues

Content Relations: define a dependencies of media data

units on some data

Spatial Relations: usually as layout relationships define the

space used for the presentation of a media data units on

an output device at a certain point of time in a multimedia

presentation.

Temporal Relations: define the temporal dependencies

between the more than one media stream data.


6/38


Sources of Asynchrony Different delays:Assumption of independent network connections

imposes different delays.

Network jitter:asynchronous data transfer destroys synchrony.

End-system jitter:packetizing and depacketizing of media data

units with different size due to encoding introduces jitter as well as

passing media units through the lower protocol layers.

Clock driftbetween the clocks in the servers and in the client is

present because we do not assume global clocks.

Change of the average delay:the synchronization scheme has to

be adaptive with respect to a change of the average delay.

Server drop outs:due to process scheduling are a realistic

assumption when using non-real-time operating systems.


7/38E0262 MIS - Multimedia Pla back S stems

Types of Synchronization Time based synchronization

To synchronize and to disseminate multimedia data using time as a

major parameter

Synchronization is achieved by mean of electronic time signals

generated ad hoc by devices directly connected to a primary time

standard and transmitted by radio or by cable.

Time signals classes: video or sound signals (called right time

signals) for low accuracy manual synchronization, digital signals for

automatic synchronization with medium-high accuracy

Passive synchronization techniques: Do not use a time signalgenerated ad hoc, instead it is used a signal generated for other

purpose



Types of Synchronization Delay based synchronization

Schemes compensate for these delays by computing well-definedstarting times for each stream server of the application.

Allow to initiate the synchronized playback of a media stream that is

composed of several multimedia streams.

Jitter based synchronization

jitter is experienced by media units traveling from the source to the

destination.

To smoothen out jitter, elastic buffers are required. These schemes guarantee a smooth playback of the stream

very low buffer requirements.



Intra-Stream Synchronization

Audio and video streams are isochronous in nature.

processing and network delay jitter (i.e., the variance indelay);

variations in rates of recording and playback; and

unreliable transmission of stream data units. Solutions:

buffer monitoring

feedback technique global clock



Inter-Stream Synchronization

A temporal relationship may exist at the source

between multiple continuous streams

multiplexing of streams

aggregation in one data structure

global clocks synchronization marker

synchronization channel

feedback technique



Spatial Synchronization

all participants in the conference receive the audio

and video data at the same time, to maintain a fairconference

Based on global clocks synchronization channel or

feedback techniques as proposed for inter-stream

synchronization.

When global clocks are available, mechanisms

based on these clocks can achieve the most

accurate spatial synchronization



Synchronization Handlers

roles : to identify the stakeholders involved, as well as the

prohibition and policy concepts.

Roles of these components in handling synchronization

Service provider

Network provider

End-user roles



Service Provider Intra-stream synchronization: SP manipulates incoming

audio/video streams so that outgoing streams are within the

10 ms jitter boundary;

Inter-stream synchronization: SP will manipulate incoming

audio/video streams so that related outgoing audio and

video streams are within the -20 ms and +40 ms range;

Spatial synchronization: SP is responsible for ensuring that

outgoing audio/video streams are played out simultaneously

at the multiple users within the 0.25 s boundary.



Network Provider Compulsory network service:a transportation service with a

deterministic service is provided

Statistical reliable network service:a transportation service

with a certain percentage of QoS violations is provided.

Best effort network service:the request from a client for a

certain transport service is evaluated against the current

network traffic.



End-User With respect to the display of audio and video it is important

that the following requirements to be met:

lip-synchronization is a well known requirement and should

be in the -20 ms to +40 ms range;

audio or video jitter should be within the range of 10 ms;

loss of video frames or audio samples is tolerable when less

than 1% of the total sent;

spatial synchronization should be in the range of -0.25 s to

+0.25 seconds.


16/38


End-User

compulsory end-user service:in this case the

synchronization requirements must be met. statistical reliable end-user service:a certain percentage of

violations of the synchronization requirements is allowed.

best effort end-user service:possibilities to full the

synchronization requirements are based on currentprocessing and storage activities.

User policies are often application dependent.


17/38


Synchronization Methods Startup Synchronization

Constant delay and zero jitter first nmedia units

where tiindicates expected arrival time of the MDU iat the

client-site and t0

is the playout start time of the Io

th stream

Evaluation phase:round trip delays for each of the stream ofthe application are calculated, while

Synchronization phase:the starting time for each server is

calculated and transmitted back to the servers.


18/38


Synchronization Methods Startup Synchronization

Beginning of first phase

Max of the round trip delay

for all nsubstreams

Second phase

Difference between

the arrival timesof arbitrary media units


19/38


Example of Startup Synchronization


20/38


Example of Startup Synchronization


21/38


Intra-Stream Synchronization

Jitter based Intra-Stream Synchronization

Worst case estimate for

error and buffer req.

Interdependencies:


22/38


Inter-Stream Synchronization Inter-stream synchronization example that shows temporal

relations in a multimedia presentation including audio, video,

animation, and picture frames


23/38


Inter-Stream Synchronizations Live Synchronization:Maintains the temporal relations as

they existed during the capturing process

Synthetic Synchronization: Temporal relations are artificiallyspecified.

S h i i R f M d l


24/38


Synchronization Reference Model


25/38



26/38


S h i ti M d l


27/38


Synchronization Models The Temporal Synchronization Model


28/38


Synchronization Models


29/38


Synchronization Models Unix Synchronization Model


30/38



31/38


Object Data Stream Synchronization Technique


32/38


Object Data Stream Synchronization Technique Synchronization point isa point held by all participating media streams

needing to be synchronized

Synchronization points defined on the composite stream specify the

places that synchronous presentation must be checked and maintained.

Let sidenote the maximum time interval that media mican skip and di

denote the current delay time interval with the presentation of media

stream mi. We then have the following situations:

Deadline based Scheduling for Sync


33/38


Deadline based Scheduling for Sync nMDUs in a multimedia document, set of mchannels in

network

Effective bandwidth rate cjper channel and transit delay j,

aggregate capacity is not sufficient to provide inter-stream and

intra-stream synchronization without pre-fetching.

If the ithMDU and playout deadline i

is scheduled for

transmission on thejthchannel at timej, according to some

scheduling policy , then its arrival time, ai, at the user site

becomes

Feedback Techniques for Sync


34/38


Feedback Techniques for Sync

The resynchronization policies are:

Conservative:This scheme reacts only when playout ofdifferent streams is guaranteed to be asynchronous,

Aggressive:In this approach the server reacts as soon as

there is even a slight chance that playback is asynchronous

Probabilistic:The server reacts on the average (assuming

the network delay distributions and playout rate variations

are known).

Virtual Local Time


35/38


Virtual Local Time

Virtual local time

Does not rely on a synchronized network clock, which is not

always feasible to procure on the Internet,

Can be implemented with a minimum overheads, and

Can produce a reasonably good quality even when the

network delay jitter is very high.

Different Playout Strategies


36/38


Different Playout Strategies


37/38


Synchronized multicast media streaming framework


38/38


y g

Date post:	05-Apr-2018
Category:	Documents
Upload:	suklasatapathy
View:	228 times
Download:	0 times

Ms Synchronisation

Documents