Digital Media Technology: Real World Applications By June Bicknese [email protected]Abstract Digital Media Technology combines elements of traditional media production and recent computer technology. My research explains how a major media production company has gradually implemented digital media technology into its production process. My paper will start with the major concepts surrounding video/media servers and explore the current status of digital technology at a major media production company. One practical digital application of this technology is the “kiosk” system architecture within the company’s entertainment facilities. All videos accessible through the kiosks are created through traditional media production and are then edited and stored on a media server machine. The videos can be retrieved from a server and played at a local kiosk via the network. This kiosk system demonstrates the synergy between classic media production and modern computer technology. I plan to discuss in detail video creation, compression, retrieval from the server and storage. Table of Contents I. Video/Media Server Architecture 1. Video Server Architecture 2. Getting an input 1) Analog input 2) Digital input 3. Processing 1) Editing 1
Digital Media Technology combines elements of traditional media production and recent computer technology. My research explains how a major media production company has gradually implemented digital media technology into its production process.
My paper will start with the major concepts surrounding video/media servers and explore the current status of digital technology at a major media production company. One practical digital application of this technology is the “kiosk” system architecture within the company’s entertainment facilities. All videos accessible through the kiosks are created through traditional media production and are then edited and stored on a media server machine. The videos can be retrieved from a server and played at a local kiosk via the network. This kiosk system demonstrates the synergy between classic media production and modern computer technology. I plan to discuss in detail video creation, compression, retrieval from the server and storage.
Table of Contents
I. Video/Media Server Architecture1. Video Server Architecture2. Getting an input
1) Analog input2) Digital input
3. Processing1) Editing2) Compressing3) Encoding
4. Storage1) Ideal disk storage2) RAID as a popular disk storage3) Popular levels of RAID
5. Archiving6. Retrieving
1) Web server vs. Video server2) Retrieving a video with a metafile3) Streaming vs. Downloading4) Playing a video on a client browser
II. A case study of a media production company1. Current status of digital media usage2. Overview of kiosk systems3. Video server
1) Video server architecture2) Process of storing a new input in the media server 3) Uploading a video during a game4) Storage5) Digital archive6) Retrieval7) Network architecture
III. References
I. Video/Media Server Architecture
1. Video Server Architecture
2. Getting an input 1) Digitizing analog input
In order to do anything with this type of input, you have to digitize the input first. This process is called digitizing or capturing.
To capture from a VHS tape, we used to need a capture program. The program digitizes analog signal and save the digitized media as a digital video format. But recent digital technology makes this digitizing process available in the editor or encoder.
Analog input consists of luminance that gives brightness of an image and chrominance that gives colors of an image. These two elements form a frame. A frame is like a picture and a series of these frames create moving images.
For digitizing a VHS video, the analog tape is imported into digitizing software. Then, the digitizing software makes the analog input into the digital format. It means analog movies are converted into streams of bits. Those bits are instructions that can recreate the video or audio in the analog format.
After a digital video is taped, digital videos are directly transferred into the editor of the server.
3. Processing 1) Editing is software-specific. Commonly used editing applications are Avid Xpress,
Avid Composer, Windows Media Encoder, Quicktime Pro, etc. 2) Compressing
Compressing is the method that makes the digital media small enough to fit through the network. Compression should lower the bit rate while maintaining the best possible quality. In details, a compression algorithm analyzes the data and removes or changes bits so that the integrity of the original content is maintained as much as possible, while reducing the file size and bit rate.
(1) Why do we need compression? First, successive video images have redundant and repeated images. So, compression makes the repeated images be encapsulated in such a way that nearly every piece of redundant content should be expressed once, and the system will tell the decoder to repeat what I have given you x times over the next y frames.” Second, since there are some limitations that human can perceive in visual and auditory systems, digital compression can take advantage of this fact. Audio and video data can be compressed up to the human’s visual and auditory systems will never miss the information.
(2) Two types of compression methodThe algorithm used to compress and decompress the media is called codec. Depending on the parameters as the codec is applied, there are two types of compression: lossless and lossy. Lossless compression is that data can be compressed and decompressed without losing any of the original data so that the decompressed data matches the original data exactly. On the other hand, lossy compression means that data is lost during the compression process and cannot be recovered during the decompression. The amount of how much data is lost is depending on what kind of compression scheme is applied to the data. In streaming media, the compression and decompression algorithms are lossy in order to fit the bit rate of the Internet.
3) Encoding After the data is compressed, it then has to be encoded. Encoding is the way to put the
digital media in the correct format so that the media can be downloaded or streamed via network. The encoded file now can be distributed over the network.
4. Storage 1) The ideal disk storage (1) It should be reliable and against corruption. The more bulletproof storage, the more
reliable. It needs total assurance of data protection.
3
=>
Frame
0111011101101110100101010001110011……
< Streams of bits > : Instructions to recreate an analog video
(2) It requires being available for rapid access. Data should be stored and recovered in the most rapid and accurate fashion, and with a minimum of outside activity or interaction. It also needs fault tolerance and backup.
(3) It should be scalable, extensible and easy to upgrade. (4) It should mimic any individual disk drive it replaces. The device, regardless of its
size and complexity, should appear to the primary host system as identical to any other disk in the system. The reason is that when individual disk storage fails, the system should make sure that there is no data lost. When the failed drive is taken out of service and a new one is replaced, it should be done without shutting down or rebooting the system.
2) RAID as popular disk storage (1) The disk storage cost has gone down and the capacity has been improved. (2) A disk array is a group of individual disk drives assembled to appear as one storage
or as a single drive. For the ideal storage system, any drive array, made up of N drives, will mimic the disk system it replaces and the overall storage must appear as one single disk. If a single drive fails in the array of N drives, it must not appear as a failure to the host processor and should continue to respond without any compromise. The array must be capable of repairing itself without seriously degrading ongoing operations.
(3) The user must be able to replace any failed drive when it needs to, without degrading or comprise of operations.
3) Common levels of RAID RAID is an element of the overall server system and a storage array. And RAID is
numbered with an integer as a different level. Each level of RAID has different architecture and the higher RAID number doesn’t mean the better performance. And, the most popular ones are RAID 1, 3, and 5.(1) RAID 1: Since this uses disk mirroring, it is simple, most reliable, and easiest of the RAID architectures to implement. And it guarantees that all data is duplicated and residing on at least two separate drives. However, this requires all the data must have its duplicates. Since video data are much larger than other data types, this results in huge amount of hardware memory. In the production industry, there are so many new videos created, it is not proper for the storage of a video server.
(2) RAID 3: It is an array of N-drives with data that are striped into bits or bytes across all the drives except the parity drive. Some authors prefer this system for media data since large blocks of sequential transfers are required for media files and these large
4
blocks can be striped across the multiple drives. However, this structure gives poor levels of read/write performances, since its I/O requires accessing all the disks, so it takes longer time than the contiguous media block.
(3) RAID 5: Serial data are divided into blocks and striped across all of the disks. Independent data block access is available because it is possible to extract the entire specific data block from any one drive without accessing other drives. And there is no designated parity drive, but parity block is interleaved throughout all the disks. This is commonly used structure. If multiple drives fail and only two drives can function properly, the remaining two drives can work as RAID 5. So, this RAID storage has higher fault tolerance than other RAID levels.
5. Archiving 1) Problem of analog tape library The current problem that a media company is facing is continuously expanding volume of a videotape library. Since this physical storage continues to grow, tracking all the video contents seems problematic. And retrieving a video clip is time consuming. 2) Digital media archive Digital media archive is a system for managing, preserving, and providing access to a wide variety of digital collections, from images to sound and video files. Although the pure volume of material is not diminishing, as a digital compression scheme improves and disk storage has higher capacity, smaller disk space can take the same data blocks
5
that have used large storage space. In addition, with well-setup network, the stored data can be fast accessible. One example of a digital media archive is HSM (Hierarchical Storage Management). It is applied to saving infrequently used data to less-expensive storage devices and mostly retrieved data to more expensive storage, so that seldom-used information frees space for newer and more often-needed information. 6. Retrieving the output 1) Web server vs. Video server
If you need small amount of video or audio clips to be played over the network, your web server doesn’t need to have a separate video server. Your web server can accommodate the basic functions of a video server.
However, if we are considering media production and distribution of the output over the network, we definitely need a video server that has higher quality digital services than the web server.
The following table is comparing media server to web server.Features Media Server Web Server
Stream through most firewalls x xStream without downloading x x
Broadcast (live) xIntelligent streaming x
(The table is adapted from ‘Streaming from a web server’ in msdn library)
2) Windows Media Metafile A metafile is a text file that contains the link from a web page to the actual media
file in a video server or a web server. To play a video via web browser, all you have to do on a web page is to add a
metafile of the actual media file. When the metafile is called, it will reference the media file and the video will be played.
Why do we need a metafile? That is because if a media file is directly linked from a web browser, there is no guarantee that the web browser will know what the file is about and how to use the file. In other words, the browser may download the file, try to open it as a web page, or simply don’t know how to handle the file so return error message to the user. For this reason, you can be assured that metafile will properly link to the media file and the video will be played accordingly.
In addition, if you have both of a web server and a media server, the metafiles can be in the web server and the media files can be in the video server. This is an effective way to combine the web server and the video server.
* Creating a Windows Media Metafile: In a text editor like Notepad, you can create the simple script like the following:
6
Metafile uses XML-based scripting language and instructs Windows Media Player to go to the location and play the media file accordingly. (a) ASX defines the enclosed script as being part of a Windows media metafile.(b) ENTRY specifies the digital media(c) Ref href specifies the URL of the media. MMS is the Windows Media Streaming protocol.After you type the above, save it as MyVideo.wvx. Since you create a metafile corresponding to each media file, if the media file name is MyVideo.wmv, its metafile can be MyVideo.wvx.There are different kinds of metafile extension that correspond to its media file.File extension When to use.wax All digital media is audio-only, with .wma file name extension.wvx Media contains video, with a .wmv file name extension.asx Media contains video, with a .asf file name extension.wmx Media contains video, with a .vm file name extension.wmd Media contains video, with a .wmz file name extension(The table is adapted from ‘Streaming from a Web Server’ in msdn library)
* Adding the metafile on a web page: The easiest way is to put a hyperlink to the metafile as follows:
3) Streaming vs. Downloading(1) Downloading: It is a method of copying a file over a network. It takes time for an entire file to be transmitted and it also requires for the client computer to have enough memory to store the file.(2) Streaming: It is a method of delivering a file over a network and playing the file instantly. It doesn’t require much waiting since as soon as enough data arrives and the rest of the data continuously come, it will play smoothly in the receiving station's buffer. In addition, the streamed file will not be saved at the receiver’s computer so no extra memory is necessary. (3) Intelligent streaming: It dynamically adjusts the bit rate of streaming video as the network bandwidth changes. It is beneficial for a video to be streamed via networks with different bandwidth over the Internet. However, the quality may be downgraded.
(4) Advantages of streaming over non-streaming media: a. It allows a video to be played quickly across a network (real-time playback). b. It reduces the delay caused by waiting for downloading a video file over IP. The compressed video is streamed from the source computer to the destination computer and played instantly while it is streamed.
7
c. This frees the user worry about big memory space for video playing since streaming only requires limited amount of buffer space for the transmission caused by network delay or congestion. d. If a user cancels the playback, the network resources have not been wasted by downloading the entire file. e. A viewer has same control options such as play, stop, pause, fast-forward, and rewind.(5) Disadvantages of streaminga. If a user has a low network bandwidth, streaming videos that have higher bit rate than the available bandwidth will not work well. In order for a video to be streamed, the bit rate of the video has to be smaller than the network’s bandwidth. Because a video is played as it is being received, if the bandwidth is lower than the bit rate, it will not receive consecutive media data and not be played continuously.
4) Playing a video on a client browser The player decompresses the transmitted data and converts the digital instructions into analog video or audio form. The audio or video can be accessed and played by entering a URL using the Windows Media Protocol, such as MMS. There are different ways for a video to be played using the player. First, you can put a simple link on a HTML page. It will open the Media Player outside the browser and initiates the connection between the player and the digital media. And the delivered audio or video will be played outside the browser. This scheme enables the end user to continue to surf with the browser while enjoying the streaming media. Second, if you want the video to be played within the web page, you can embed the player in a web page using OBJECT tags. This method enables you to integrate the design of the player with that of your page.
Widows Media Player Embedded Media Player
The advantage of embedding is that you have a complete control over how the player looks and what the functions are.
Windows Media Player’s original functions The functionalities are changed by embedding
8
In order to embed the player, you must add an OBJECT element for the Windows Media ActiveX control. For embedding a media player, a metafile can be used with PARAM within the OBJECT element like the following:
(More details are in the article “Adding Windows Media to Web Pages” in msdn library.)
7. Network The network architecture for multimedia should be configured to deliver the media of a time sensitive nature. However, the computer platform is not time sensitive and this gives a problem for interactive video media delivery. In packet switching, data can tolerate delays in delivery because nothing has to be done until all the data arrives and is reassembled. Various delays and flow controls regulate data rates. But, audio and video cannot tolerate time delays. This makes packet switching unsuitable for real time traffic applications. One option can be fast packet switching that allows control parameters that specify the importance of in-time delivery in a network header. This option may let some time sensitive data pass through at a higher priority than other types of data for which timing is not the priority. ATM uses cell switching that is a combination of circuit switching and fast packet switching. Each cell size is small enough to pass through most of networks without fragmentation. Once virtual circuit is set up, it is remained until the end of usage and the virtual circuit guarantees fast delivery. So, ATM seems reasonable solution for the multimedia network. However, in reality, Ethernet and ATM are a good match. Ethernet is used inside the building for data traffic among personal workstation or desktop while ATM is used for backbone network. Before considering digital media network, the networks are setup for the Internet. It is told that having Ethernet and ATM lets the data and multimedia effectively share the network.
II. A case study of a media production company 1. Current status of digital media usage
In production area, digital media technology has been used mostly for video editing and commercial playback. A video server is not used for the production. Instead, Avid system is used as a quasi-server.
9
It enables the file specified by the URL value as soon as a media player is called.
Processing• Editing• Compressing • Storage for commercial playback
Analog
Digital Analog
Digital
Windows Media Player7.0 Class ID
The closest of using digital technology is the kiosk systems. Kiosk systems are using essential digital media technology. So, examining the kiosk systems can be a good example of the current applications for digital technology.
2. Kiosk system Kiosks are located at the First Union Center. They are used to help the sports fans and audiences get information about the facilities and events. Kiosk touch screens are especially popular during a sports game intermission. During the intermission, people can see the major highlights that just happened during the game. The kiosk systems are divided into three categories based on the locations, such as Main Concourse, Luxury Suites, and Lexus Club with different user interface design.
Main Concourse Luxury Suites Lexus Club
The major applications1) Macromedia Flash – overall design of the user interface 2) Adobe Photoshop and Adobe Illustrator – graphic and image editing 3) Windows Media Player – streaming video player4) MS SQL server – text contents
10
<Avid System>
Internal Channel within Avid Editor
5) SMS (Systems Management Server) – check the status of all the kiosk systems and allows us to remotely trouble-shoot all the kiosk systems.6) symantec pcAnywhere – remotely access the server and update the contents of the server, and to transfer files from or to the server.
SMS symantec pcAnywhere
3. Video Server 1) Video Server Architecture * Main functions of the server: storage of videos and streaming
2) Process of storing a new input in the media server Video inputs in the server are mostly game highlights or concert footage. They are 30 seconds to 1 minute 30 seconds long.
11
Compressed digital video input
a. Stores the input in order in primary disk array.
b. Retrieve the requested video file in real time.* Stream the video* Transmit the video file
Windows Media Player requests for the media file with
the metafile.
Compressed video is streamed.
Compressed video is downloaded.
< Windows Media Server >
The texts are from MS-SQL.
Icons are edited in Adobe Illustrator .
The pictures are edited in Adobe Photoshop.
Windows Media Player is loading the retrieved video to play.
The overall interface design is done in Macromedia Flash.
* Process of getting a new video file(1) Shoot a video with the analog camera(2) Get composite or component inputs from the analog videotape and import the tape into Avid Xpress
- A component input separates analog video signal into three different elements of red, green, and blue (RGB) and use three different cables, so that they produce better image quality. - A composite input combines all of three elements and use one cable so that it is easier to import the input but has less image quality.
(3) When a video is shooted by the analog camera, it is uncompressed and flawless state. So, we need to digitize the analog tape into digital in Avid Xpress.(4) Compress the video with 2-to-1 compression algorithm and save it as motion-JPEG.
In Avid Xpress, there are various compression schemes like the following: 2 to 1 3 to 1 ......... ....
20 to 1 An algorithm for Avid Xpress is Motion jpeg. JPEG is a standard for storing
and compressing digital images. Motion-JPEG extends this standard by supporting videos. In motion-JPEG, each frame in the video is stored with the JPEG format, and compresses each video field separately, returning the resulting JPEG bit streams consecutively in a single frame.
(5) The motion-JPEC file is changed into an omfi file. Motion-JPEG has no universal decoding algorithm standard. The decoding algorithm can vary from one manufacture to another. This means that a motion-JPEG file may produce different results on different applications. For this reason, there is an omfi compressed file format, used as the interchange of digital media data between different platforms or applications. OMFI means Open Media Framework Interchange, a registered trademark of Avid Technology, Inc. It is a standard format of the interchange of digital media data among heterogeneous platforms. The format is designed to encapsulate all the information required to interchange a variety of digital media, such as audio, video, graphics, and still images as well as rules of combining and presenting the media. And the format includes rules for identifying the original source of the digital media, and it can encapsulate both compressed and uncompressed digital media data.
12
Higher compression: Less space More artifacts
Lower compression: More space Better looking
(6) The omfi file is exported from Avid Xpress and imported into Windows Media Cleaner.(7) Encoding: In the Windows Media Cleaner, the omfi file is encoded as a Windows Media file.
(8) Save the digital video clip as a media file (.wmv) and create its metafile (.wvx) in a notepad.
<ASX version = “3.0”> <Entry>
<Ref Href = “Path” /></Entry>
</ASX>(9) Transfer the wvx and wmv files via symantec pcAnywhere into the two different
server machines.
3) Process of uploading a new video during a game During the game, if there is a memorable moment, the highlight is uploaded into the video server so that audience can see it again during the intermission. This uploading is almost like real-time happening, but it takes about 5 minutes after the actual event. (1) All games are recorded into Avid Xpress.
(2) Once there is a good highlight, it is edited and exported into the Windows Media Cleaner.
(3) The Cleaner encodes it as a Windows Media file (4) In order to upload the clip into the video server, you need to save it as a
Windows Media file and Windows metafile. (5) Transfer the wvx and wmv files via symantec pcAnywhere into the two
different server machines. 4) Storage
(1) RAID level 5 – RAID 5 allows three or more drives to stripe the data across the multiple drives. Even two drives can function well.
13
(2) How videos are stored in the server disk storage* Videos are named as team name / show name + purpose (history, highlight, award etc) + number in orderFor example, the first Flyers history video is named as flyhis01.wvx in database and flyhis01.wmv in the video server.* There are subvideos. These are smaller video files than the main video files. It is used when somebody comments on a particular game or play that happened. It is played with the main video clip. The processes of uploading and retrieving a subvideo file are same as the video ones.
One example from the two tables is the following:<Video>VideoID Date Description Team ... Filename ...
1005 5/19/1974 Flyers Win 1st Stanley Cup Flyers ... flyhis05 ...
(2) The user interface application, macromedia flash has the code in action script that calls video.asp. The action script passes the video id as an input parameter and asks for the metafile.
(3) Video.asp requests the web server for the metafile by passing the video id. (4) The web server looks at the video table in the database and retrieves the metafile.
(5) The metafile is sent back to the asp middleware and then to the action script. (6) In the macromedia flash, Windows Media Player gets the metafile and accesses
the video server to find the actual media file. (7) In the video server, the metafile references its media file.
* The metafile: Windows Media Protocol + Windows Media Server + video folder + media file. a. A file retrieval
<ASX version = “3.0”> <Entry><ref href = mms://ctc_wms/video/sixgre03.wmv /> </Entry></ASX>
(8) The video is played by streaming method. In details, as soon as the enough data
is arriving, Windows Media Player decompresses the digital file, transforms it into analog format, and plays it. The Player must receive the file continuously, because streaming media is a real-time play and it needs to receive continuous media data.
* The overall process of retrieving a video from the video server:
* If there are multiple requests from multiple clients, the server use multithreading for multiple streaming. The server can usually multi-stream videos more than 200 streams. Since the total number of the kiosks is around 150, the max number of streams is enough.
* Intelligent streaming: although the server has the capacity, it doesn’t use intelligent streaming because the quality of the video that is buffered by intelligent streaming is not as high as original file. Besides, since there is a dedicated network for kiosk, there is no need for that.
For doing simple media streaming over the Internet, check out the web site, http://catalyst.washington.edu/catalyst/how-to/digitalvideoThere are specific instructions depending on which application you are using such as RealMedia, QuickTime, and Media Cleaner Pro. For Windows Media Encoder, check out msdn library.
In each building, the network architecture is Ethernet. And between the buildings, it is set up as frame relay using a permanent and dedicated virtual circuit. The bandwidth of Ethernet is 100MB, and the bandwidth of Frame relay is 1Gb.(1) Frame Relay Gateway: Frame Relay is a high-speed and packet switching protocol that moves packets over virtual circuits to connected network devices. A Frame Relay connection establishes a cost-effective Virtual Private Network (VPN) between networks. Frame Relay provides speedy LAN-to-LAN or WAN-to-LAN internetworking data transfer, and connects multiple networks. The Frame Relay gateway moves packets, through the PVC (Permanent Virtual Circuits), which is a logical path through a Frame Relay network and delivers them on to its destination. Since Frame Relay is a link layer protocol, it looks at the IP address in the body of the packet and in its routing tables to route packets to and from the Frame Relay network. Frame relay doesn’t do any error correction. Each switch inside a frame relay network just relays the data to the next destination. The today’s networks are sufficiently error free to move the burden of error correction to the end points. Because frame relay passes blocks of data from switch to switch without error correction, the packet delivery through the network is very fast.
(2) RTSP (Real Time Streaming Protocol) : RTSP is an application-level protocol for control over the real-time data delivery. It is associated with the port 554. Sources of data can be audio, video, or live data feeds. This protocol is designed to cope with bandwidth-demanding and real-time streams. It can be used with TCP or UDP. In real application, since the corporate network should prevent any network congestion, it is used with TCP. Since TCP has congestion control mechanism, if there is any network congestion, TCP packets will back off and wait for available time. This is important since there may be more important data traveling over the network and this data should be able to reach its destination without the network congestion problem.
17
III. References* Karl Paulsen, Video & Media Servers: Technology & Applications Second edition 2001 Focal Press* http://www.ecsl.cs.sunysb.edu/~andrew/VideoServer/videoserver/index/book1.html* http://multimedia.asti.dost.gov.ph/videoserver/*http://www.thedigitalhub.com/thedigitalhub/digitalmedia/whatisdigitalmediainIreland.html* http://www.thedigitalhub.com/thedigitalhub/digitalmedia/streamingmedia.html* http://www.avid.com/products/xpress/* http://www.symantec.com/pcanywhere/Consumer/* http://www.microsoft.com/smserver/default.asp* http://www.microsoft.com/windows2000/technologies/web/default.asp* http://www.microsoft.com/windows/windowsmedia/default.asp* http://msdn.microsoft.com/library/default.asp?URL=/library/en-us/dnwmt/html/asx.asp* http://catalyst.washington.edu/catalyst/how-to/digitalvideo* http://www.columbia.edu/itc/itc/webdev/design/av-intro.html* http://www.macromedia.com/support/flash/ts/documents/flash_database.htm* http://www.webopedia.com/TERM/M/motion_JPEG.html* http://www.wynit.com/video/glossary.htm* http://www.sirsi.com/glossary.html* http://www.rad.com/networks/1994/fram_rel/frame.htm * http://www.intel.com/support/si/library/bi0801.htm* http://www.dcbnet.com/notes/framerly.html* http://www.tcm.hut.fi/Studies/Tik-110.300/1998/Essays/rtsp.html* http://www.ericsson.com/about/telecom/part-f2/f2-2-3.shtml* http://www.opnet.com/products/library/fr.html
