Date post: | 16-May-2015 |
Category: |
Technology |
Upload: | yossi-cohen |
View: | 4,942 times |
Download: | 2 times |
Video Delivery Methods
Yossi Cohen
Intro: “Legacy” Methods
Streaming Streaming methods
RTP/RTSP RTMP Real MS WMV/ASF
Proprietary and Fragmented Seek-ability – supported Server side – proprietary technology (FMS) Cache-ability – requires special servers for streaming User experience – very good Cost – high
Progressive DownloadSimple Utilizing existing protocols & servers (HTTP) Media file is prepared: metadata up front Playback - after metadata is received Cache-ability - supported Seek-ability – very limited support Poor user experience - seek, multi-rate Waste of bandwidth when not watched fully Low cost
Pseudo Streaming Media is sent as a regular file like Progressive DW Server must understand how the media is structured Playback: after metadata is received Existing protocols
Non standard server Non standard client component
Cache-ability – Limited ! Seek-ability – supported User experience – better than PD, support seek. Waste of bandwidth when not watched full
HTTP Streaming Intro HTTP Streaming offers the advantages of:
Progressive download in terms of Cost Standard Server Scalability Standard client components (OSMF)
Streaming in terms of User experience Seek-ability of streaming
Delivery Methods Summary
Progressive Download Session Analysis
Flash Video
Progressive Download Uses file download from an HTTP web server. Uses HTTP GET request Flash player enables file playback while the download is
still in progress. The ability to be played while the file is being
downloaded is in the wrapper (container) of the file.
PROGRESSIVE DW FLASH SESSION
Request from YouTube First stage is to connect to YouTube Server with the
HTML page request. Source: 192.168.2.102 Destination: 72.125.39.138 GET /watch?v=lAl28d6tbko&feature=popt00il00
Notice that the request passes throw server 72.125.39.138 witch is identified as a Google server
Request from YouTube The Request itself is pointed to the host name (from the
HTTP message), wich is YouTube HTTP server. The HTTP server provides only the HTML page of the
request Host: www.youtube.com
1) Request HTML Page2) HTML response
Browser192.168.2.102
Web Server72.125.39.138
Session Progress
Requesting the Flash Video Player The Player is then requested automatically by the Flash
client – the information is retrieved from the query string (in bold).
<script> (function() {var isIE = /*@cc_on!@*/false;var swfHTML = (isIE) ? "<object height=\"38" + "5\" width=\"64" + "0\" classid=\"clsid:D27CDB6E-AE6D-11cf-96B8-444553540000\" id=\"movie_player\" ><param name=\"movie\" value=\"http:\/\/s.ytimg.com\/yt\/swf\/watch_as3-vfl157163.swf\"><param
name=\"flashvars ….})();</script>
*note this is only a fragment of the query string (located in var swfHTML )
Session Progress
Browser192.168.2.102
Web Server72.125.39.138
File Server82.80.222.83
1) Request HTML Page2) HTML response3) Request the Flash
Player4) Flash Player response
Requesting of Player Models The swf file that contains TAGS to information that the
player needs. http://s.ytimg.com/yt/swf/watch_as3-vfl157163.swf
The SWF can be analyzed with swfmill. This can help locating the TAG’s inside the file.
For example: the tags containcom.google.youtube.ui.QualityButton_HqOffIcon_dataClass 1 com.google.youtube.ui.WatchEndScreen_replayIcon_dataClass 2 com.google.youtube.ui.QualityButton_HqOffIcon 3
Request from YouTube The Flash Player requests the model information from
the CDN server. YouTube uses many CDN around the world. Source: 192.168.2.102 Destination: 212.179.74.181
GET/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dWRVZSUl9FSkNNNl9OTFZB&fexp=905700%2C902904&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1270674000&key=yt1&signature=B20512CB5B62A0FCEA5D15EFF56AB35B35524614.15CDEF403DDDC9716E51EBB108CBA7D7CEFD7D98&factor=1.25&id=940976f1dead6e4a&
Request from YouTube The host name is different too. Notice that this is a IP
located in Israel (host: bzq-82-80-222-82.red.bezeqint.net) – this is a Proxy.
Request from YouTube Notice that this is a different server: IP: 82.80.222.82 The host name is different too. Notice that this is a IP
located in Israel (host: bzq-82-80-222-82.red.bezeqint.net) – this is a Proxy.
Request from YouTube The original host of the message is Googles CDN
server: Host : video-stats.video.google.com IP: 74.125.79.139
Complete Session
Browser192.168.2.102
Web Server72.125.39.138
File Server82.80.222.83
FLV Server / CDN212.179.74.181
1) Request HTML Page
2) HTML response3) Request Flash
Player4) Flash Player
response5) FLV File Request
Flash Player192.168.2.102
Modern Capture (Byte Ranges)
Changes from Previous
Decision on request type (Standard / byte range( based on Request 204 replay
TCP timing using Tval Tsecr Algorithm definition in request “cburst”, “throttle” Byte range requests
204 Request
Measures latency -> Input for request type mechanism
Algorithm Request
Cburst / throttle depending on content requested Client side instead of server side control
Byte Ranges
HTTP Streaming common
Over HTTP Media Segments:
Complete media units Separate media types (A/V)
Segments alignment GOP Aligned Time Aligned
Description File Must hold segment time frame Must Hold bit rate information Might include Media description like resolution, codecs
HTTP Streaming AppleMicrosoftAdobe
HTTP Live Streamingby Apple
Agenda System Overview Components Session
Apple’s Note
Note: Many existing streaming services require specialized servers to distribute content to end users. It requires specialized skills to set up and maintain these servers, and in a large-scale deployment these servers can be costly. Apple has designed a system that avoids this by using standard HTTP to deliver the streams.
System Overview
Components Review Server
Encoder Segmenter
Distributer Basic HTTP Server
Client
Server Receives Digital / Analog input stream Encodes / Transcode video/audio
H.264 Video AAC audio (HE-AAC or AAC-LC)
Encodes / Transcode audio only: MPEG-2 elementary streams, HE-AAC or AAC-LC
files, or MP3 files Encapsulate in MPEG2
Transport Stream Program Stream
Segmenter All segments should be with the same duration All segments are placed in a separate file Creates Index file with references to segment files For protection, the Segmenter might encrypt each media
segment and create a key file
Distribution Distribution system is a regular HTTP Server Could be Apache or small embedded Server
Files Segments – stored as *.ts files Index files – stored as *.m3u8 Index file format example:
#EXTM3U#EXT-X-TARGETDURATION:10#EXTINF:10,http://media.example.com/segment1.ts#EXTINF:10,http://media.example.com/segment2.ts#EXTINF:10,http://media.example.com/segment3.ts#EXT-X-ENDLIST
Session types Live Stream Broadcast
Index file is continues updated Include a moving window of segments around “live”
part of the session Client should continuously refresh the Index file
VoD Session Index file static Includes ALL the segments of the file Enables “Seek” operation
Multi-bitrate multi–device support Multi-bitrate is enabled via multiple index files Index files are pointed by a global index files Client can select a stream according to:
Device properties Available bit rate
This method is less efficientthan Silverlight Global File
Test yourself1. What are the two Live streaming file types?2. What is the role of the Segmenter?3. On which delivery protocol is the live streaming based?
Silverlight Smooth Streaming
Movi
ng to
DASH
SILVERLIGHT INTRODUCTION
Smooth Streaming
•Microsoft’s implementation of HTTP-based adaptive streaming
•A hybrid media delivery method that acts like streaming but is in fact a series of short progressive downloads
•Leverages existing HTTP caches•Client can seamlessly switch video quality
and bit rate based on perceived network bandwidth and CPU resources
Smooth Streaming Design
•Smooth Streaming File Format based on MP4 (ISO Base Media File Format)
•Video is encoded and stored on disk as one contiguous MP4 file▫Separate file for each bit rate
•Each segment is composed of a 1 or more GOP This allows easy fragmentation at key frames
•Contiguous file is virtually split up into chunks when responding to a client request
Evolution•Previous versions of MS streaming divide
the file into many chunkc 0001.vid 0002.vid etc
•Problematic in caching, CDNs, CMS etc•Today all fragments of a file are contained
in a single bitstream container. Typically 1 fragment = 1 video GOP.
SILVERLIGHT FILES
Containers & Configuration files
MP4 File format MP4 has two format types
Disk Format - for file storage Wire format - for transport
Wire format enables easy CDN support and integration
File extensions Media Files
*.ismv - Audio & Video *.isma – Audio only
Manifest Files *.ism – Server manifest. Describes to the server
Relation between tracks, bitrates & files on disk. Based on SMIL 2.0 XML format specification
*.ismc – Describes to the client the available streams, CODECS used, bitrates encoded, video resolutions, markers, captions. First file delivered to client. It’s the first file delivered to client (“SDP” like).
Directory Structure
Manifest Files
Media file in different bitrates
Manifest files VC-1, WMA, H.264 and AAC codecs Text streams Multi-language audio tracks Alternate video & audio tracks (i.e. multiple camera
angles, director’s commentary, etc.) Multiple hardware profiles (i.e. same bitrates targeted at
different playback devices) Script commands, markers/chapters, captions Client manifest Gzip compression URL obfuscation Live encoding and streaming
ISM file sample<?xml version="1.0" encoding="utf-16" ?> - <!-- Created with Expression Encoder version 2.1.1206.0 --> - <smil xmlns="http://www.w3.org/2001/SMIL20/Language">- <head> <meta name="clientManifestRelativePath" content="NBA.ismc" /> </head>- <body>- <switch>- <video src="NBA_3000000.ismv" systemBitrate="3000000"> <param name="trackID" value="2" valuetype="data" /> </video>- <video src="NBA_2400000.ismv" systemBitrate="2400000"> <param name="trackID" value="2" valuetype="data" /> </video>- <video src="NBA_1800000.ismv" systemBitrate="1800000"> <param name="trackID" value="2" valuetype="data" /> </video>
ISM file sample- <video src="NBA_1300000.ismv" systemBitrate="1300000"> <param name="trackID" value="2" valuetype="data" /> </video>- <video src="NBA_800000.ismv" systemBitrate="800000"> <param name="trackID" value="2" valuetype="data" /> </video>- <video src="NBA_500000.ismv" systemBitrate="500000"> <param name="trackID" value="2" valuetype="data" /> </video>- <audio src="NBA_3000000.ismv" systemBitrate="64000"> <param name="trackID" value="1" valuetype="data" /> </audio> </switch> </body> </smil>
*.ISMC sample<?xml version="1.0" encoding="utf-16" ?> - <!-- Created with Expression Encoder version 2.1.1206.0 --> - <SmoothStreamingMedia MajorVersion="1" MinorVersion="0" Duration="4084405506">- <StreamIndex Type="video" Subtype="WVC1" Chunks="208"
Url="QualityLevels({bitrate})/Fragments(video={start time})"> <QualityLevel Bitrate="3000000" FourCC="WVC1" Width="1280" Height="720"
CodecPrivateData="250000010FD3FE27F1678A27F859E80C9082DB8D44A9C00000010E5A67F840" />
<QualityLevel Bitrate="2400000" FourCC="WVC1" Width="1056" Height="592" CodecPrivateData="250000010FD3FE20F1278A20F849E80C9082493DEDDCC00000010E5A67F840" />
<QualityLevel Bitrate="1800000" FourCC="WVC1" Width="848" Height="480" CodecPrivateData="250000010FCBF81A70EF8A1A783BE80C908236EE5265400000010E5A67F840" />
<QualityLevel Bitrate="1300000" FourCC="WVC1" Width="640" Height="352" CodecPrivateData="250000010FCBE813F0AF8A13F82BE80C9081A7ABF704400000010E5A67F840" />
ISMC File - 2- <SmoothStreamingMedia MajorVersion="1" MinorVersion="0" Duration="5965419999">- <StreamIndex Type="video" Subtype="WVC1" Chunks="299"
Url="QualityLevels({bitrate})/Fragments(video={start time})"> <QualityLevel Bitrate="2750000" FourCC="WVC1" Width="1280" Height="720"
CodecPrivateData="250000010FD3BE27F1678A27F859E804508253EBE8E6C00000010E5AE7F840" /> …..
<c n="0" d="20000000" /> <c n="1" d="20000000" /> ..... <c n="298" d="5000001" /> </StreamIndex>- <StreamIndex Type="audio" Subtype="WmaPro" Chunks="299"
Url="QualityLevels({bitrate})/Fragments(audio={start time})"> <QualityLevel Bitrate="64000"
WaveFormatEx="6201020044AC0000451F0000CF05100012001000030000000000000000000000E00042C0" />
<c n="0" d="20433560" /> .... <c n="297" d="20433560" /> <c n="298" d="4393197" /> </StreamIndex> </SmoothStreamingMedia>
SILVERLIGHT SESSION
Initiation and Flow
Smooth Streaming Protocol Smooth Streaming Protocol uses HTTP [RFC2616] as its
underlying transport . The Server role in the protocol is stateless
Enabling (potentially) different instance of the server to handle client requests
Request can utilize any generic HTTP caches/proxies - > Lowering CDN costs
Messages Smooth Streaming Protocol uses 4 different messages:
Manifest Request Manifest Response Fragment Request Fragment Response
All messages follow the HTTP/1.1 specification
Messages FlowServer
Client
Manifest Request
Manifest Response
Fragment Request
Fragment Response
Fragment Request(s)
Messages Manifest Request and Fragment Request message
MUST use the HTTP "GET" method, generated by the client.
Manifest Request and Fragment Request message use the HTTP Response messages.
Status-Code SHOULD be 200.
Smooth Streaming Transport Protocol Session Details
Manifest RequestManifest Request
Manifest ResponseManifest ResponseVideo Fragment RequestVideo Fragment Request
Fragment ResponseFragment Response
Audio Fragment RequestAudio Fragment Request
Session Details - Manifest Request
In order to initiate a presentation the Client MUST send the server a Manifest Request using the HTTP GET method.
Session Details - Manifest Response
The Response is a ISMC Manifest file describing the session. - <SmoothStreamingMedia MajorVersion="1" MinorVersion="0" Duration="5965419999">
- <StreamIndex Type="video" Subtype="WVC1" Chunks="299" Url="QualityLevels({bitrate})/Fragments(video={start time})">
<QualityLevel Bitrate="2750000" FourCC="WVC1" Width="1280" Height="720" CodecPrivateData="250000010FD3BE27F1678A27F859E804508253EBE8E6C00000010E5AE7F840" />
…..
<c n="0" d="20000000" />
<c n="1" d="20000000" />
.....
<c n="297" d="20000000" />
<c n="298" d="5000001" />
</StreamIndex>
- <StreamIndex Type="audio" Subtype="WmaPro" Chunks="299" Url="QualityLevels({bitrate})/Fragments(audio={start time})">
<QualityLevel Bitrate="64000" WaveFormatEx="6201020044AC0000451F0000CF05100012001000030000000000000000000000E00042C0" />
<c n="0" d="20433560" />
....
<c n="297" d="20433560" />
<c n="298" d="4393197" />
</StreamIndex>
</SmoothStreamingMedia>
Manifest Response reviewed We can see in the ISMC file that the server can support 8 different levels
of quality (bitrate) for the client can chose from between 2.75Mbit to 0.35 Mbit.
- <SmoothStreamingMedia MajorVersion="1" MinorVersion="0" Duration="5965419999">
- <StreamIndex Type="video" Subtype="WVC1" Chunks="299" Url="QualityLevels({bitrate})/Fragments(video={start time})">
<QualityLevel Bitrate="2750000" FourCC="WVC1" Width="1280" Height="720" CodecPrivateData="250000010FD3BE27F1678A27F859E804508253EBE8E6C00000010E5AE7F840" />
<QualityLevel Bitrate="2040000" FourCC="WVC1" Width="1056" Height="592" CodecPrivateData="250000010FD3BE20F1278A20F849E80450823E414DD1400000010E5AE7F840" />
<QualityLevel Bitrate="1520000" FourCC="WVC1" Width="848" Height="480" CodecPrivateData="250000010FCBAE1A70EF8A1A783BE8045081AE62F3F7400000010E5AE7F840" />
<QualityLevel Bitrate="1130000" FourCC="WVC1" Width="704" Height="400" CodecPrivateData="250000010FCBA215F0C78A15F831E8045081A27BD635C00000010E5AE7F840" />
<QualityLevel Bitrate="845000" FourCC="WVC1" Width="576" Height="320" CodecPrivateData="250000010FCB9A11F09F8A11F827E804508199C94077400000010E5AE7F840" />
<QualityLevel Bitrate="630000" FourCC="WVC1" Width="448" Height="256" CodecPrivateData="250000010FCB920DF07F8A0DF81FE804508113396020C00000010E5AE7F840" />
<QualityLevel Bitrate="470000" FourCC="WVC1" Width="368" Height="208" CodecPrivateData="250000010FC38E0B70678A0B7819E80450810E5747B6C00000010E5AE7F840" />
<QualityLevel Bitrate="350000" FourCC="WVC1" Width="320" Height="176" CodecPrivateData="250000010FC38A09F0578A09F815E80450808AADEACF400000010E5AE7F840" />
Manifest Response – reviewed The client also receives the number of chunks for audio and video tracks
and the duration of each chunk so it can request the chunk which fits the desired position in the file
<c n="0" d="20000000" />
<c n="1" d="20000000" />
<c n="2" d="20000000" />
<c n="3" d="20000000" />
....
<c n="297" d="20000000" />
<c n="298" d="5000001" />
</StreamIndex>
- <StreamIndex Type="audio" Subtype="WmaPro" Chunks="299" Url="QualityLevels({bitrate})/Fragments(audio={start time})">
<QualityLevel Bitrate="64000" WaveFormatEx="6201020044AC0000451F0000CF05100012001000030000000000000000000000E00042C0" />
<c n="0" d="20433560" />
<c n="1" d="19969161" />
<c n="2" d="19969161" />
<c n="3" d="20433560" />
<c n="4" d="20433560" />
<c n="297" d="20433560" />
<c n="298" d="4393197" />
</StreamIndex>
</SmoothStreamingMedia>
Session Details – Fragment Request
Client-Server requests are based on RESTFull URLs:GET /mediadl/iisnet/smoothmedia/Experience/BigBuckBunny_720p.ism/QualityLevels(350000)/Fragments(video=0)
The URL includes reference to: Bitrate as QualityLevels which maps to a media file Fragment number
Session Details – Fragment Response
The Server: checks “BigBuckBunny_720p.ism” server manifest file to find the
media file associated with the quality level(350000) Opens and parses the associated media file to get the chunk with
requested time offset (0). Sends the requested media fragment to the client as HTTP
response with status code set to 200
Examplehttp://www.iis.net/media/experiencesmoothstreaming
Adobe HTTP Streaming
Movi
ng to
DASH
Thank you!
• Video Expert
• Lectures on Video / Android / VoIP
• Android Native Developer
More About me:
Yossi CohenYossi [email protected] [email protected]
+972-545-313092+972-545-313092
Backup slides
Adobe Solution main components
Agenda Overview Components Files Session
HTTP Streaming Intro HTTP Streaming offers the advantages of:
Progressive download in terms of Cost Standard Server Scalability Standard client components (OSMF)
Streaming in terms of User experience Seek-ability of streaming
OverviewAdobe HTTP Dynamic Streaming is a solution that allows you to stream live and on-demand content over HTTP to Adobe Flash Player. When content streams over HTTP, clients can seek quickly to any location. HTTP Dynamic Streaming supports adaptive streaming, DVR functionality, and Adobe Flash Access protection (DRM).
COMPONENTS
• Content Ingest• Server• Client
Main components Preparation
File Packager Live Packager for HTTP Dynamic Streaming
Server Apache module (HTTP Origin Module) Flash Access
Client Player with OSMF classes Flash Player version 10.1+ Air 2.0+
Ingest- File Packager•A command-line tool •Used for converting offline content to formats
required for Adobe HTTP streaming •Translates on-demand media files into
fragments and writes the fragments to F4F files.
Ingest - Live Packager• The Live Packager for HTTP Dynamic
Streaming is part of Adobe Flash Media Server.
•The server ingests a live stream over RTMP and translates it into F4F files in real-time.
•The built-in Apache HTTP Server uses the HTTP Origin Module to deliver the live content over HTTP.
Server - Apache module (HTTP Origin)•Extension to Apache HTTP Server version
2.2. •Enables processes of Adobe Files:
▫F4F, F4M,F4X ▫.bootstrap and ▫.drmmeta
•Flash Media Interactive Server 3.8 includes Apache HTTP Server.
Server - Flash Access
• DRM Server • Flash Access delivers protected
media to Flash Player• For content protection, both File
Packager and Flash Media Server are required to package and encrypt the content
Client - OSMF classes• The OSMF Player uses the
ActionScript 3.0 NetStream.appendBytes() API to deliver data to Flash Player.
•OSMF is a robust framework designed to deliver high-quality video.
•Adobe strongly recommends using OSMF to build HTTP Dynamic Streaming players.
ADOBE HTTP STREAMING FILESF4F, F4M,F4X .bootstrap and .drmmeta
Files The files required for HTTP streaming are:
F4F - MPEG4 media format. Holds the media F4M – Media description file(codec, resolution) F4X - Fragments location file .bootstrap – bootstrap information for each
segment .drmmeta – DRM encryption information
*.F4F File
•Standard MP4 format with open file specification
•Each file contains a segment of the source file.
•Each segment contains one or more fragments of content.
•The file formats stores any flash supported codec
•A player can use a URL to address each fragment.
HTTP Streaming file types
•*.F4X File▫Flash Index file. ▫Contains the location of specific
fragments within a stream. •*.F4M File▫Flash Media Manifest file. ▫Contains information about the media
codecs, resolution, and the availability of multi-bitrate files.
F4M File Sample<?xml version="1.0" encoding="utf-8" ?> - <manifest xmlns="http://ns.adobe.com/f4m/1.0"> <id>Dynamic Streaming</id> <duration>253</duration> <mimeType>video/mp4</mimeType> <baseURL>rtmp://cp67126.edgefcs.net/ondemand</baseURL> <media
url="mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_768x428_24.0fps_408kbps" bitrate="408" width="768" height="428" />
<media url="mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_768x428_24.0fps_608kbps" bitrate="608" width="768" height="428" />
<media url="mp4:mediapm/ovp/content/demo/video/elephants_dream/elephants_dream_1024x522_24.0fps_908kbps" bitrate="908" width="1024" height="522" />
</manifest>
Folder example Example for File “foo” Folder will include :
fooSeg#.f4f (many fragments) foo.f4x foo.meta foo.bootstrap foo.drmmeta (if the stream is configured for
encryption)
Session The player fetches foo.f4m manifest. The player request fragment# HTTP Origin Module locates the required f4f files by
looking at F4X index file and send to client. If the client seeks further, the Origin module searches at
the f4m and again locates the f4f file that fits, starts streaming again.
If the player constantly monitor bandwidth and performance and ask the Origin Module for the proper bit rate associated f4f.