Broadcast Technology No.43, Winter 2011 ● C NHK STRL 6 Hybridcast TM System and Technology Overview Hybridcast is a platform for enhancing broadcasts through the use of communications networks, and it will extend broadcast services in various ways. In particular, it will make possible new services, such as customized program viewing, social networking, and video on demand, and it will have functions for linking mobile terminals and PCs to television receivers. Here, we present an overview of Hybridcast and the technologies that make it possible. 1. Introduction The digitization of broadcasting and the spread of broadband environments have led to an information infrastructure that can provide high-quality digital broadcasts and a variety of Internet services. A new era will soon dawn where broadcasting and communications networks work together seamlessly to provide new services. NHK is focusing to provide broadcast services that make the most out of this fusion of networks. In this article, we introduce Hybridcast, the system linking broadcasting and communications that NHK STRL is developing for this new era. After describing the Hybridcast concept and objectives, we give examples of services that can be implemented with it, describe a system prototype, and introduce R&D on some of its underlying technologies. 2. Concept The Hybridcast concept is illustrated in Figure 1. Hybridcast is a platform for building systems that make the most of the characteristics of broadcasting (simultaneous delivery, high quality, and high reliability) and those of communications (the ability to respond to the individual needs and requests of users). It is a hybrid system that uses communications to enhance broadcasting services. The following elemental technologies form the basis of Hybridcast. (1) Technology to synchronize and combine broadcast programming sent over the airwaves with content arriving over a communications network for presentation. (2) Content management and distribution technology utlizing a network cloud* 1 . Here, content means not only content from broadcasters but also one from social network services (SNS) and consumer generated media (CGM) from viewers, etc. (3) Technology to link mobile terminals with home televisions. This technology enables users to use mobile devices to select content for viewing on their home televisions. (4) Security technology. Content copyrights and users' personal information must be protected to enable content to be viewed safely anywhere. Information provided through communications channels should maintain the same high level of security as broadcasting. CGM, SNS Program Broadcaster Additional program-related information Content management/delivery technology Network cloud Broadcast (simultaneous delivery, high-quality, reliable) Communications (able to respond to individual viewer’s request) Content rights protection technology Viewer private information protection technology Enriched viewing experience Receiver TM Household Synchronized playback technology Mobile terminal linking technology Figure 1: Hybridcast system concept Hybridcast is a trademark of NHK Engineering Service Inc.
Transcript
Broadcast Technology No.43, Winter 2011 ● C NHK STRL6
HybridcastTM System and Technology Overview
Hybridcast is a platform for enhancing broadcasts through the use of communications networks, and it will extend broadcast services in various ways. In particular, it will make possible new services, such as customized program viewing, social networking, and video on demand, and it will have functions for linking mobile terminals and PCs to television receivers. Here, we present an overview of Hybridcast and the technologies that make it possible.
1. IntroductionThe digitization of broadcasting and the spread of
broadband environments have led to an information infrastructure that can provide high-quality digital broadcasts and a variety of Internet services. A new era will soon dawn where broadcasting and communications networks work together seamlessly to provide new services. NHK is focusing to provide broadcast services that make the most out of this fusion of networks. In this article, we introduce Hybridcast, the system linking broadcasting and communications that NHK STRL is developing for this new era. After describing the Hybridcast concept and objectives, we give examples of services that can be implemented with it, describe a system prototype, and introduce R&D on some of its underlying technologies.
2. ConceptThe Hybridcast concept is illustrated in Figure 1.
Hybridcast is a platform for building systems that make the most of the characteristics of broadcasting (simultaneous delivery, high quality, and high reliability) and those of communications (the ability to respond to the individual needs and requests of users). It is a hybrid system that uses communications to enhance broadcasting services.
The following elemental technologies form the basis of Hybridcast.
(1) Technology to synchronize and combine broadcast programming sent over the airwaves with content arriving over a communications network for presentation.
(2) Content management and distribution technology utlizing a network cloud*1. Here, content means not only content from broadcasters but also one from social network services (SNS) and consumer generated media (CGM) from viewers, etc.
(3) Technology to link mobile terminals with home televisions. This technology enables users to use mobile devices to select content for viewing on their home televisions.
(4) Security technology. Content copyrights and users' personal information must be protected to enable content to be viewed safely anywhere. Information provided through communications channels should maintain the same high level of security as broadcasting.
Communications(able to respond to individual viewer’s request)
Content rights protection technologyViewer private information protection technology
Enriched viewing experience
Receiver
TM
Household
Synchronized playback technology
Mobile terminal linking technology
Figure 1: Hybridcast system concept
Hybridcast is a trademark of NHK Engineering Service Inc.
Broadcast Technology No.43, Winter 2011 ● C NHK STRL
Feature
7
Opinions and impressions from viewers regarding the program
Opinions and impressions from viewers are categorized into groups, and the numbers of submissions for each group are displayed in different colors.
Figure 2: Program customization service screen
The whole reason for developing Hybridcast is the various services that can be supported by broadcasting and communications networks working together. Besides the broadcasting system, Hybricast will have servers in a network cloud that provide various kinds of content and functionality enhancements and receivers that coordinate and synchronize broadcasts with the functionalities and content offered through the communications network. Moreover, the effect on current broadcast equipment and receivers should be kept to a minimum when introducing Hybridcast; the specifications for the existing broadcast system must be changed as little as possible. Our tasks are discussing the user requirements, studying the system architecture and defining the technical specifications, such as specifying application programming interfaces (APIs).
3. New Services Made Possible with Hybridcast3.1 Service Examples
Each of the services described below makes use of the functions and information in the cloud to enhance an existing broadcast service. Various extensions to services can be realized depending on how the communications network is used.
(1) Program customization This service provides supplementary
information regarding the program currently being broadcast through a communications network such as the Internet and presents it in synchronization with the broadcast program. Individuals will be able to view programs in ways tailored to their own needs and interests. For example, foreigners with limited Japanese ability will be able to select from subtitles in multiple languages through the communications channel (Figure 2), whereas elderly persons who may have trouble hearing sound in a program will be able to slow down the audio in a way that makes it easier to hear (while fitting within the same overall duration). The receiver receives supplemental data from the network server as it receives the broadcast program, and it synchronizes these different streams before it displays them. By presenting one program using a combination of broadcasting
and communications, high-quality broadcast services can be customized for individuals.
(2) Social television This is a service uniting social networking services on
the Internet with broadcasting (Figure 3). With a social television service, viewers can express their opinions and impressions regarding a program while the program is being broadcast, and viewers not actively participating in the SNS can also share their opinions and impressions. The server in the network cloud analyzes opinions and impressions from viewers, divides posts into groups that are displayed in different colors on the screen, and graphs the numbers of submissions for each group to clearly show the distribution of opinions.
(3) Program recommendations This service provides the viewer with recommendations
from a large library of video-on-demand (VOD) programs available on the Internet. Recommendations from different perspectives are displayed in different columns (Figure 4). Selected programs can be viewed immediately
*1 Computer resources on the Net. Vari-ous types of processing are done using computer resources on the Net.
Figure 3: Social television service screen
Broadcast Technology No.43, Winter 2011 ● C NHK STRL8
Related program
Highly evaluated program
Program you may be interested in
“Hot” topic program
Bookmarked program list On air program
Program thumbnail
Program title
Program description
See reviews
Delete bookmark
Watch program
Usage Help
Recommended programs
OverviewPlayback
Taro logged in.
Play them on the television
Search for recommended programs...Read the QR code
Figure 4: Program recommendation service screen
Figure 5: Bookmark service screen
Figure 6: Mobile terminal linking service concept
by using the remote control, or they can be bookmarked for later viewing on a bookmark server on the network (Figure 5).
(4) Mobile terminal link Various personalized services, such as the program
recommendation service, are provided with Hybridcast.
The viewer’s mobile terminal can be used to simplify the login process while viewing television. Technologies such as Bluetooth*2 can be used to link mobile terminals to televisions. Moreover, we have developed our own method using QR codes (see Section 4.4).
Figure 6 illustrates how the mobile-terminal-linkage service works. Using the mobile terminal, viewers can access the personalized services described above such as recommendations and bookmarking, even with televisions in hotels and other locations away from home. The mobile terminal can also be used as a remote control to operate the television set.
3.2 Prototyping of Hybridcast ServicesThe prototype implements the
four services described in Section 3.1. We first set out the basic functions required of the Hybridcast system as a platform. We built servers for each service and implemented the receiver functions in a receiver. The overall architecture is shown in Figure 7. The application execution environment is capable of running the various services on the receiver, and the user can switch among services by selecting applications from a menu on the screen. The receiver runs the applications for each service while displaying the broadcast program. The applications provide the services by obtaining the required information from the servers. For this prototyping, the connections between the receiver and each of the servers were created on a LAN.
Currently we are defining the specifications for the Hybridcast system and it includes ones for APIs for communications from one server to another or between servers and the applications running on the receivers and for control APIs to use the various receiver functions. The prototype implements the communications
APIs using the Representational State Transfer (REST) format*3 . In the future, we will consolidate these results
*2 An IEEE short-range wireless standard for digital devices.*3 A method of communication between server and client over
the Internet which uses XML and the HTTP protocol.
Broadcast Technology No.43, Winter 2011 ● C NHK STRL
Feature
9
and define and publish these specifications for the Hybridcast system.
4. Technology Supporting Hybridcast4.1 Synchronization of Broadcasting and Communications
For the program customization service discussed in Section 3.1, the receiver must combine the broadcast content with content obtained through the communications channels (various media such as video, audio, subtitles, or metadata) on the receiver. The broadcast program is received in real time, and the supplementary data received through the communications channels must be synchronized with it in real time.
Although broadcasts have a stable and accurate timing, packets sent over IP networks can vary in their arrival times, which can cause a delay of IP-delivered
content and disturb synchronization between broadcast and IP content. One way to compensate these delays is to transmit the communications content earlier than the broadcast content. Unfortunately, this cannot be done with live programs because the supplementary data to be transmitted over the communications channel cannot be prepared ahead of time. Instead, the broadcast content can be buffered in the receiver for a period of time that covers the delays of the communications content to which it is to be synchronzed1).
Figure 8 shows the architecture of the transmission and reception systems for implementing this service. On the transmission side, the network servers generate additional content in response to users’ requests and stream that content to the users as the broadcast is being transmitted. The Presentation Time Stamp (PTS), which references the Program Clock Reference (PCR) of the broadcast stream, is attached to the content sent over the
100
Prototype receiver (PC-based receiver)
Simulated broadcast station
Multi-lingual subtitle server
Speech-rate conversion audio server
Social TV server
Related program recommendation server
Program review server (Teleda)
VOD Program
server
Login management and device linking
control
SNS Live comments
Recom
mend
ations
Recom
mend
ations
Recom
mend
ations
Prog
ram
revie
ws
Bookmarks
Bookmarks
Sim
ulat
ed b
road
cast
si
gnal
(TS
)
Control API
Communication API (REST)
Program customization Social Television Recommendations Mobile link
Supplementary content(Presentation-time information
(PTS) based on broadcast system clock)
Tuner/Decoder
Receiver
System clock value
IP Receiver/decoder
Screen renderingVideo
Graphic Texts
Audio
Presentation composition
Synchronization buffer (delay)
Sync
hron
ize
Synchronization buffer
Request generation
Broadcast
Communications networkReceiver
Broadcaster
Figure 7: Hybridcast system architecture
Figure 8: Transmitter/Receiver system architecture for synchronized broadcasting
Broadcast Technology No.43, Winter 2011 ● C NHK STRL10
communications channel so that it can be synchronized with the broadcast content by the receiver*4.
The broadcast content is delayed long enough at the receiver to absorb delay fluctuations in the communications channel, so that the content from the two channels can be synchronized and displayed. Synchronization is accurate to within one video frame2).
4.2 Comment-Clustering TechnologyWhen displaying comments related to a program on
the television receiver, filtering technology is needed to prevent inappropriate comments from being displayed. We have developed a method for analyzing viewer comments regarding a program and grouping users with similar comments in order to find out what the overall impressions of the program are and how many people hold them3). The comment analysis method looks for a set of the object and emotional expressions about the object of each comment. The object could be, for example, one of the characters appearing in a drama. Emotional expressions are categorized as affirming or denying expressions, expressions of surprise or sadness, etc. The scene to which the comment applies might also be identified from the time the comment was posted, correcting for input delay using subtitle timing information. The similarity among users can be quantified from the data sets of the object and emotional expressions’ relations to the objects of the comments. The television interface that displays the results of the analysis is in the REST format4).
4.3 Program Recommendation TechnologyVarious methods for generating recommendations
have been developed for Internet services, and we at NHK STRL have developed several methods as well.
For example, the EnVision technology recommends programs related to the one being viewed by using content metadata to calculate the similarity between programs5).
We have developed a method to analyze viewer feedback in order to recommend content that seems likely to interest the viewer6). This method analyzes comments related to a television program and recommends related programs that have garnered favorable comments about their characters or actors.
4.4 Device Linking TechnologyTo provide bookmarking or social television services,
the individual watching the television must be identified by the receiver displaying the service. The user’s mobile terminal would be a good means of accomplishing this.
A user-management server in the cloud of Hybridcast manages user IDs, mobile terminal Ids, and receiver device IDs. The mobile terminal accesses the user management server, and the server registers the user ID and mobile terminal ID. A possible login operation for linking the receiver and the user could be done as follows (Figure 9). When "Use Mobile" is selected on the receiver menu, the receiver displays a QR code on the screen. The QR code encodes the receiver ID and a program ID for the program currently being watched. The user captures the QR code using the mobile device (1), and the mobile device sends the receiver ID obtained from the code and the mobile terminal ID to the user management server (2). This links the user ID, mobile terminal ID, and receiver ID on the user management server, completing the user login process. This is just one possible login means, and login procedures remain as a topic for development.
5. ConclusionThe Hybridcast system will make possible new services
by linking broadcasting and communications. We will clarify the specifications for the servers and receivers supporting Hybridcast, define various APIs, and conduct tests to verify the design. After that, we will make Hybrid
User management server User ID, Mobile terminal ID, Receiver ID
Television receiver (display QR code) Mobile terminal
(1) Receiver ID
(2) Receiver ID, mobile terminal ID(3) Receiver ID
(4) User ID
Figure 9: Mobile terminal linking service
*4 To synchronize the broadcast station system clock and the receiver system clock, the Program Clock Reference (PCR) of MPEG2-TS is multiplexed into the stream at fixed intervals. The multiplexed audio and video streams have a Presenta-tion Time Stamp (PTS) set to this system clock; thus, the presentation times of the audio and video will be the same.
Broadcast Technology No.43, Winter 2011 ● C NHK STRL
Feature
11
services available to the public.(Kinji Matsumura and Yasuaki Kanatsugu)
References1) K. Matsumura, M. J. Evans, Y. Shishikui and A.
McParland: "Personalization of Broadcast Programs using Synchronized Internet Content," Int. Conf. on Consumer Electronics 2010 (ICCE), 4-1.5 (2010).
2) K. Matsumura, Y. Kanatsugu, and H. Hamada, "Prototyping a program extension service with broadcast-synchronized IP streaming," ITE Annual Symposium, 10-5 (2010) (Japanese).
3) K. Ariyasu, Y. Kanatsugu, and H. Hamada: "Considering the results of program comment clustering analysis," IPSJ 50th Anniversary National Convention, 1H6, pp. 4-63-4-64 (2010) (Japanese).
4) K. Ariyasu, Y. Kanatsugu, and H. Hamada: "A Social
Television Service for sharing impressions of television programs," ITE Annual Symposium, 10-4 (2010) (Japanese).
5) H. Sumiyoshi, M. Sano, J. Goto, T. Mochizuki, M. Miyazaki, M. Fujii, M. Shibata and N. Yagi: "Curio View: TV Recommendations Related to Content Being Viewed," IEEE Int. Symp. on Broadband Multimedia System and Broadcasting 2010, pp. 1-6 (2010).
6) K. Ariyasu, Y. Kanatsugu, and H. Hamada: "A consideration of social television: Program Comment recommendation based on Comment Analysis," IEICE 2010 HCG Symposium, HCG2009-A8-4 (2009) (Japanese).
