NHK STRL is researching 8K Super Hi-Vision (SHV), a television system that delivers immer-sive video and sound. We aim to conduct test broadcasting of SHV in 2016 and begin full ser-vice broadcasting in 2020.

In our research on video formats, we developed a wide-color-gamut SHV camera and a wide-col-or-gamut SHV projector with a laser light source. The combination of these devices enables accu-rate reproduction of highly saturated object col-ors that could not be handled with Hi-Vision. We studied technologies to convert the SHV video signal format used in current program produc-tion to the full-resolution format in order to sup-port the broadcasting signal format to be used in 2016 and devised a method suitable for real-time processing to interpolate color signals precisely.

We developed a single-chip color camera sys-tem with a wavelength division multiplexing unit and a camera control unit for practical use. We also prototyped a cube-shaped compact camera head weighing only 2 kg. We improved the sensi-tivity of the image sensor to be used in full-speci-fication SHV cameras (33 megapixels, 120 Hz) by using a nanofabrication semiconductor process. We also had the opportunity to use the silent, high-sensitivity “theater camera” prototyped in FY 2012 to shoot Teatro alla Scala’s performance of “Rigoletto” for a live viewing event. The SHV video of this performance was favorably received by the audience.

We studied light-emitting displays for full-specification SHV video and demonstrated the feasibility of 120-Hz organic light-emitting diode (OLED) displays by identifying the design param-eters and circuit structure of the oxide TFT.

In our effort to give a high degree of mobil-ity to SHV program productions, we developed a compression recording technology for the cam-era and a high-speed recording technology using multi-parallel NAND flash memory. We also pro-totyped a compact video recorder.

Regarding video coding, we upgraded the SHV video encoder to embody the MPEG-H High Effi-ciency Video Coding (HEVC)/H.265 scheme that we developed in FY 2012. We updated the bit-stream generation function to be compliant with the international standard for HEVC and added a multiplexing function for the connection to the transmitter. We also compared its performance with that of a conventional MPEG-4 AVC /H.264 encoder and confirmed that its image quality is superior.

In our work on audio, we added functions to

save, read, and adjust frequently used production patterns to a 22.2 multichannel sound mixing system in order to shorten the production time. We also implemented 22-directional reverbera-tion waveforms measured in various places in a 3D reverberator and developed a display-inte-grated 12-loudspeaker system capable of repro-ducing 22.2 multichannel sound.

Regarding our standardization activities, we led the effort to establish an ITU-R Recommenda-tion on sound systems combining channel-based and object-based methods, including loudspeak-er layouts for 22.2-multichannel sound. The Rec-ommendation was published in February 2014. We also proposed to the Association of Radio In-dustries and Businesses (ARIB) an interface for connecting UHDTV video devices in studio. The interface transmits various UHDTV signals, in-cluding full-specification SHV signals, over mul-tiple 10-Gbps links. ARIB established a standard for it in March 2014.

In our work on satellite broadcasting in the 12-GHz band, we studied the 16 Amplitude and Phase Shift Keying (APSK) modulation scheme with high frequency usage efficiency and pro-posed to the Information and Communications Council a transmission system for SHV that is ca-pable of transmitting 100-Mbps signals through a single channel of the satellite transponder. We also worked on international standardization of multiplexing technologies; our proposal was adopted by MPEG as an international standard for its new media transport scheme called MPEG Media Transport (MMT).

We are also developing large-capacity trans-mission technologies for the next generation of digital terrestrial broadcasting. This year, we in-stalled an experimental station for testing dual-polarized multiple-input multiple-output (MIMO) transmissions in Hitoyoshi, Kumamoto Prefec-ture, and began measurements of the channel characteristics. The experiments showed that SHV signals compressed to 91 Mbps could be transmitted over a long distance (27 km).

In our work on optical transmission of uncom-pressed SHV signals, we developed a method to transmit 72-Gbps full-resolution SHV signals on eight wavelengths (10 Gbps per wavelength). We also proposed to the Japan Cable Television En-gineering Association (JCTEA) a method to parti-tion compressed SHV signals and transmit them through multiple channels of cable television networks.

4　｜　NHK STRL ANNUAL REPORT 2013　

1 Television conveying a strong sensation of reality

1.1 Super Hi-Vision

1.1.1 Super Hi-Vision format

We have made progress with our R&D and standardization activities related to the 8K Super Hi-Vision (SHV) video system.

■ Wide color gamutThe SHV video format employs wide-gamut system colo-

rimetry that covers a wider range of colors than for Hi-Vision and can reproduce most real object colors. We designed the spectral sensitivity characteristics that would be needed for a camera supporting the wide-color-gamut and used the results to develop prisms for full-resolution SHV cameras with three 33-megapixel image sensors (1). We built a wide-color-gamut SHV system consisting of this wide-color-gamut camera and a wide-color-gamut SHV projector (see 1.1.3). At the NHK STRL Open House, we used the system to capture and display im-ages of objects with highly saturated colors and demonstrated the improvement in accuracy of the reproduced colors includ-ing highly saturated ones.

In order to convert the colors of the SHV wide-gamut sys-tem colorimetry into those of the Hi-Vision color space, we are studying a method that decreases only chroma without altering lightness and hue to maintain image quality. Since sufficient data had not been reported on the characteristics of perceptual hues in the color space outside the Hi-Vision color-gamut, we performed subjective evaluation tests using the wide-color-gamut SHV projector. The results showed that some hues have different characteristics from the ones pre-dicted by the conventional uniform color space.

■ High resolutionWe are studying technologies to convert SHV video captured

by 8-megapixel four-sensor SHV cameras or 33-megapixel single-sensor SHV cameras, which has a Bayer pattern pixel structure, into full-resolution SHV video with high quality. In the Bayer pattern, two green, one red and one blue pixels are arrayed in a square grid. We devised a method suitable for re-al-time processing to interpolate color signals precisely (2) and confirmed that it can generate color images that look natural and have better resolution and minimal artificial colors.

To better understand the resolution characteristics of SHV cameras, we developed a new method to measure not only horizontal or vertical but also multi-dimensional resolution characteristics precisely by analyzing an image of a slightly-slanted knife-edge target captured by the cameras (3).

To clarify the relation between spatial video resolution and preferred viewing distance during TV viewing, we measured the preferred viewing distance when displaying a program consisting of still images at different levels of spatial resolu-tion. The results showed that a shorter viewing distance tends to be preferred for higher spatial resolution (4).

■ InterfacesWe are standardizing an interface for connecting video de-

vices in the studio to transmit various UHDTV signals over 10-Gbps multilinks at the Association of Radio Industries and Businesses (ARIB). The UHDTV signals include full-specifica-tion SHV signals (7680 × 4320 pixels, 120-Hz frame frequency, 12 bit), 4K signals (3840×2160 pixels), 60-Hz frame frequency, and the 4:4:4, 4:2:2 and 4:2:0 formats. The ARIB Standard STD-B58 (5) was established in March 2014, and it includes specifica-tions of the data mapping and serialization, transmitter and receiver characteristics, and optical connector characteristics.

■ Subjective evaluation of image qualityWe studied methods for subjectively evaluating the image

quality that would be suitable for SHV video, which has high resolution and a wide field of view. Through subjective evalu-ation tests of coded video, we found that subjective evaluation scores depend on the viewing distance and horizontal view-ing position. Based on this result, we proposed that for a sub-jective evaluation of SHV video using a display of about 80 inches, three evaluators should observe horizontally trisect-ed parts of an image at a viewing distance of 0.75 times the screen height. This research was conducted as part of a project commissioned by the Ministry of Internal Affairs and Commu-nications, titled “The research and development project for the expansion of radio spectrum resources”.

■ Broadcasting format standardizationWe contributed to the standardization of ultra-high defini-

tion television (UHDTV) broadcasting systems at the Informa-tion and Communications Council and ARIB. In March 2014, the Council issued specifications for video formats including SHV with a 120-Hz frame frequency and High Efficiency Video Coding (HEVC).

[References](1) T. Soeno, K. Masaoka, T. Yamashita, T. Nakamura, R. Funatsu, Y.

Nishida, M. Sugawara, A. Saita: “Wide-color-gamut Super Hi-Vision Camera,” ITE Annual Convention 2013, 10-5 (2013) (in Japanese)

(2) S. Tajima, R. Funatsu, Y. Nishida: “Chromatic interpolation based on anisotropy-scale-mixture statistics,” Signal Processing, Vol. 97, pp. 262–268 (2014)

(3) K. Masaoka, T. Yamashita, Y. Nishida, M. Sugawara: “Modified slant-ed-edge method and multidirectional modulation transfer function estimation,” Optics Express, Vol. 22, No. 5, pp. 6040–6046 (2014)

(4) M. Emoto, M. Sugawara: “Still Images of High Spatial Resolution Enable Short TV Viewing Distances,” ITE Transactions on Media Technology and Applications, Vol. 2, No. 2, pp. 185-191 (2014)

(5) ARIB Standard STD-B58 (ver. 1.0), “Interface for UHDTV Production Systems” (2014) (in Japanese)

1.1.2 Cameras

We are researching cameras, including single-chip color compact cameras, video production technology, and wide col-or gamut devices for 8K Super Hi-Vision (SHV). We are also proceeding with R&D on image sensors for full-specification SHV.

■ SHV imaging systemToward the development of a compact SHV camera for prac-

tical use, we fabricated a camera system with more operability by adding a wavelength division multiplexing unit and a cam-era control unit to the compact single-chip color camera head prototyped in FY 2012. We exhibited this system at the NHK STRL Open House, NAB Show 2013, and IBC 2013.

We studied the characteristics of the on-chip color filter ar-

NHK STRL ANNUAL REPORT 2013　｜　5

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision

rays for single-chip color imaging compliant with the color gamut specified in Rec. ITU-R BT.2020. In FY 2013, we devised a method to reproduce the wide color gamut with higher preci-sion by combining signal processing and optical filter charac-teristics (1).

We built an experimental device that can process signals from the 133-megapixel single-chip color image sensor devel-oped in FY 2012. We optimized the operating conditions of this image sensor and confirmed that its signals are output cor-rectly.

We shot Teatro alla Scala’s performance of the opera “Rigo-letto” at NHK Hall and showed the video at a live viewing in September. The videos were made with a silent-operating high-sensitivity SHV camera, named the “theater camera” (2), prototyped in FY 2012. This camera captured the performance without generating any noise that would disturb the audience. We also used the full-resolution camera with the wide color gamut (also prototyped in FY 2012) to shoot SHV evaluation footage.

■ Increasing the sensitivity of full-specification SHV imaging devicesWe have developed image sensors for full-specification SHV

in accordance with Rec. ITU-R BT.2020. Since FY 2012, we have been studying ways to increase the sensitivity of SHV image sensors to the level of current Hi-Vision cameras. We evaluat-ed the sensitivity of the 33-megapixel, 120-Hz frame frequency SHV image sensor we fabricated in FY 2012 with a semicon-ductor process having a 0.11μm design rule (3). We improved the sensitivity by 1.4 times over earlier devices by nanofabri-cating the charge-voltage converter in the pixels and reduced noise from the circuitry. To gain even higher sensitivity, we designed and built a full-specification SHV image sensor by making more extensive use of the nanofabrication semicon-ductor process.

We worked to reduce the power consumption of the analog-to-digital (A/D) converter embedded in the image sensor. We developed a new structure for the converter and proved that it could lower power consumption through simulations and measurements of a prototype.

We also tried to improve the A/D converter characteristics by incorporating a digital error correction function and we ex-perimentally verified the effectiveness of a prototype circuit.

One of our papers from FY 2012 on full-specification SHV image sensors won the internationally prestigious Walter Ko-sonocky Award.

■ Full-specification single-chip color SHV imaging devices We are researching technologies for a single-chip image

sensor that will be part of a full-specification SHV camera (133 megapixels, 120 Hz).

To minimize the effect of the lower sensitivity of a single-chip image sensor with many pixels, we conducted simula-tions on a backside-illuminated structure enabling an optical aperture ratio of 100% and found that it would have 1.65 times the sensitivity of a frontside-illuminated structure.

To evaluate the characteristics of single-chip color imaging devices and enrich the variety of SHV content, we prototyped a single-chip color image sensor with 33 megapixels and mount-ed it on a cube-shaped compact camera head weighing only 2 kg. We used the camera to shoot SHV video footage of an opera for live viewings and a New Year’s Eve music show shot in front of a live audience.

Research on the full-specification SHV imaging devices and the full-specification single-chip color SHV imaging device was conducted in cooperation with Shizuoka University.

[References](1) T. Hayashida, T. Soeno, T. Nakamura, R. Funatsu, T. Yamashita, T.

Yasue, H. Shimamoto, K. Masaoka: “A Study of Spectral Sensitivity Correction of Single-chip Color Camera for Wide Gamut Imaging,” ITE Winter Conference 2013, 11-10 (2013)

(2) T. Nakamura, T. Hayashida, R. Funatsu, T. Soeno, T. Yamashita, T. Yanagi, T. Yoshida: “Development of a Super Hi-Vision camera to-ward shooting theater contents,” ITE Winter Conference 2013, 11-12 (2013)

(3) T. Yasue, T. Hayashida, J. Yonai, K. Kitamura, T. Watabe, T. Kosugi, T. Watanabe, H. Ootake, H. Shimamoto, S. Kawahito: “Sensitivity of 120-Hz Super Hi-Vision Image Sensor using 0.11μm CIS Process,” ITE Annual Conference 2013, 10-4 (2013)

1.1.3 Displays

We are continuing our research on 8K Super Hi-Vision (SHV) direct-view displays and projectors.

■ High frame rate technologiesWith the goal of displaying full-specification SHV video with

light-emitting displays, we continued with our research and development on liquid crystal displays (LCDs) and organic light-emitting diode displays (OLEDs) that can support a 120-Hz frame frequency. In FY 2013, we prototyped an SHV LCD that can reproduce SHV video at 120 Hz. We also estimated the display performance of moving pictures at 120 Hz, on the basis

6　｜　NHK STRL ANNUAL REPORT 2013　

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision

Figure 1. Shooting test of the theater camera (right)

Figure 2. Cube-shaped compact SHV camera head

of an evaluation of the moving picture resolution of LCDs and plasma display panels (PDPs) (1). In particular, we found that a light emitting aperture time of 25 – 50% per frame is required for the hold-type SHV displays. We began designing an active matrix pixel circuit for a 100-inch diagonal OLED. We conduct-ed circuit simulations to analyze the drive performance of a panel equipped with oxide thin-film transistors (TFTs), which are advantageous to large panels. We determined the design parameters (size of electrodes and mobility of TFT) and circuit structure of the oxide TFT required for the 120-Hz simultane-ous dual-line driving method and confirmed the feasibility of 120-Hz OLEDs (2).

■ Wide-color-gamut SHV projectorWe are developing a projector capable of displaying full-

specification SHV video.To build displays supporting the wide-gamut system colo-

rimetry, we developed a projector with a laser light source, in-stead of an ultra high performance (UHP) lamp, that was based on the projector for displaying 120-Hz frame-frequency SHV video that we developed in FY 2012 (3). We used laser diodes with central wavelengths of 639 nm for red, 532 nm for green, and 462 nm for blue. While leaving the optical system of the projector head as it was, we installed an external light source unit, from which laser light is injected through optical fiber to the projector head. We exhibited a wide-color-gamut SHV system consisting of the wide-color-gamut SHV camera (see 1.1.1) and this projector at the NHK STRL Open House (Figure 1) and demonstrated that it can accurately reproduce highly saturated object colors that Hi-Vision fails to handle.

[References](1) T. Usui, H. Satoh, K. Ishii, Y. Takano, T. Yamamoto: “Evaluation of

Moving Picture Resolution for a Super Hi-Vision Plasma Display Panel,” ITE Annual Conv., No. 5, 16-7, (2013) (in Japanese)

(2) H. Satoh, K. Ishii, T. Usui, Y. Takano, H. Tsuji, T. Yamamoto: “Simu-lation of 120Hz driving performance for SHV OLED Display,” IEICE General Conf., (2014) (in Japanese)

(3) Y. Kusakabe, Y. Iwasaki, Y. Nishida: “Wide-color-gamut Super Hi-Vision Projector,” ITE Annual Convention 2013, 16-1 (2013) (in Japa-nese)

1.1.4 Recording systems　

■ Super Hi-Vision recorderToward our goal of developing video recorders for 8K Super

Hi-Vision (SHV) single-chip cameras, we created compression recording technology using solid state memory and prototyped a compact video recorder (1).

Regarding the compression recording technology, we em-ployed a method in which two green (G) signals of the dual-green format from a single-chip SHV camera are upconverted to full resolution images by using data interpolation and then, these full-resolution images are compressed. This method re-duced the degradation of image quality by about 1 to 2 dB in terms of the peak signal-to-noise ratio (PSNR), as compared with the conventional method which compresses 4K image units for each color. We also developed a high-speed record-ing interface to connect a memory controller and solid state memory, which are separated in our design. The solid state memory is detachable at the interface. By recording of up to 16 channels into the solid state memory in parallel and using large-sized block recording (suitable for recording video), we achieved a recording speed in excess of 12 Gbps per memory package in the solid state memory. With a 25% compression rate, we were able to record a 45-minute SHV video in the re-cording capacity of 2 TB.

Using this technology, we prototyped an SHV studio re-corder (Figure 1) and a camera-integrated recorder. The studio recorder has separate signal processings for encoding and de-coding; thus, it can perform recording and reproduction simul-taneously. This is convenient in that a user can view and check the video while recording. The studio recorder also supports

back-up on a PC and connection to an editing machine, as it employs a format in which compressed SHV video is recorded in frame units. We also equipped the camera-integrated re-corder with a simpler reproduction function than the studio recorder to make it more compact and power-saving. Using the prototype recorders, we demonstrated real-time encoding and decoding of SHV video as well as recording and reproduction of compressed video.

[References](1) E. Miyashita, T. Kajiyama: “Compact Camera Recorder for Super

Hi-Vision,” IEEE International Conference on Consumer Electronics (ICCE), 2014 Digest, pp. 165-166 (2014)

NHK STRL ANNUAL REPORT 2013　｜　7

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision

Figure 1. Wide-color-gamut SHV system (left: objects, middle: camera, right: screen)

Figure 1. SHV studio recorder prototype and its memory package

1.1.5 Video Coding

We are researching video compression coding methods for 8K Super Hi-Vision (SHV) broadcasting.

■ HEVC high efficiency video encoder We improved the SHV video encoder using the MPEG-H High

Efficiency Video Coding (HEVC)/H.265 scheme developed in FY2012 and conducted performance evaluations. We made the encoder compliant with the HEVC international standard and added a bit-stream generation function using MPEG-2 Trans-port Stream (TS) multiplexing. We also compared the perfor-mance of the HEVC encoder with that of a conventional MPEG-4 Advanced Video Coding (AVC)/H.264 encoder in objective and subjective evaluations. The objective evaluations showed the advantage of HEVC over AVC on the basis of the peak sig-nal to noise ratio (PSNR) criterion. In contrast, the results of subjective evaluations demonstrated that HEVC scored lower than AVC under some conditions, though it scored higher than AVC in most cases. While the HEVC scheme improves coding efficiency by using large blocks, coarse quantization in these blocks degrades the texture of the original image, causing block noise, at low bit rates. This would be the reason HEVC could not improve the image quality in certain cases. We plan to improve the image quality of HEVC by taking account of the analysis. This research was conducted under the government-commissioned project titled “The research and development project for the expansion of radio spectrum resources” of the Ministry of Internal Affairs and Communications, Japan.

We have begun development of an SHV video decoder using the HEVC scheme and completed the basic design in FY2013. We studied the logic design and circuit size for a function to analyze bit streams and extract image data as well as the in-verse transform and motion compensation functions to gener-ate a decoded image from the bit stream. This research was conducted in cooperation with Mitsubishi Electric Co. Ltd.

We also studied ways to reduce the huge amount of compu-tations entailed by HEVC coding. We found that the choice of prediction modes shows a bias that depends on the reference frame structure of encoded frames. Considering this, we have developed a software encoder that skips the prediction mode selection to achieve approximately a 10% reduction in compu-tations while keeping image degradation at a minimum.

In parallel with these R&D activities, we studied domestic television systems for SHV broadcasting. We devised a cod-ing method to enable 120-Hz frame frequency broadcasting in the future while maintaining backward compatibility with 60-Hz receivers to be used at the start of SHV broadcasting. We also conducted subjective evaluations of multi-format coding (8K, 4K, and 2K) to clarify the bit rates needed for broadcasts (Table 1). We submitted our results to the working group on video coding systems of the Association of Radio Industries and Businesses (ARIB).

■ IP transmission for SHVIn preparation for SHV live viewings of 2014 FIFA World

CupTM in Brazil, we conducted IP transmission experiments in which AVC bit streams were sent between Brazil and Japan in August 2013 and February 2014, in cooperation with NTT and Rede Nacional de Ensino e Pesquisa (RNP) — a Brazilian re-search institution on academic networks. We also performed three live viewings in Japan by IP transmission: “Rigoletto” performed by Teatro alla Scala on September 15 at NHK Sendai station; the ISU Grand Prix of Figure Skating 2013, NHK Trophy on November 8 – 10 at NHK Nagoya station; and the 64th New Year’s Eve Music Show (Kouhaku) on December 31, at Grand Front Osaka.

■ Reconstructive video coding systemWe are developing a video coding system using super-reso-

lution and image reconstruction technologies.To support 120-Hz video, we developed a spatio-temporal

video format converter capable of reducing and reconstructing the spatial resolution and frame frequency simultaneously in real-time. This device, which can handle up to 4K 120-Hz vid-eo, is implemented with a spatio-temporal hybrid super-reso-lution technique that recovers the spatial resolution and frame

8　｜　NHK STRL ANNUAL REPORT 2013　

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision

7320×4320/60Hz 80〜 100 Mbps 3840×2160/60Hz 30〜 40 Mbps 1920×1080/60Hz 10〜 15 Mbps

Table 1. Estimated bit rate needed for each video format

補助情報符号化

Transmitted video Transmission Spatio-temporal video format converter Received video

4K120Hz

4K60Hz

2K120Hz

4K120Hz

4K60Hz

2K120Hz

2K60Hz

Spat

io-te

mpo

ral r

esol

utio

n re

duct

ion

and

optim

izat

ion

Spat

io-te

mpo

ral h

ybrid

imag

e re

cons

truct

ionLow-resolution video Video stream

Side stream

Local decoded video

Side data

Video encoding

Side data encoding

Side data decoding

Video decoding

Approx. 1 to 10% of video stream

Optimal selection dataSynchronization data

Synchronization of received video and side dataOptimal reconstruction based on selection data

Figure 1. Reconstructive video coding by spatio-temporal data hybrid super-resolution

rate (Figure 1). We also studied a coding method for 120-Hz video combining a transmission-side frame sub-sampling to 60 Hz and a receiver-side frame interpolation. Since the frame interpolation from the interleaved frames may cause image degradation due to mismatches in the motion vector detection, we devised a method for finding the optimal motion vector that gives the best result from among multiple motion vector candi-dates. We confirmed that this method can attain a high-quality frame interpolation.

We also developed a new video transmitting algorithm combining the spatial resolution conversion process with the bit-depth reduction and reconstruction method we had previ-ously developed. The algorithm is optimized for prospective hardware implementations by minimizing the complexity of the constituent processes. With the help of side information, which is optimized by testing the reconstruction process on the transmission side, the algorithm is capable of robust and

precise super-resolution reconstruction and bit-depth restora-tion and can be used to make a high-quality video coding sys-tem that has an extremely high compression rate.

[References](1) Y. Sugito, K. Iguchi, A. Ichigaya, K. Chida, S. Sakaida, Y.Shishikui,

H. Sakate, T. Itsui, N. Motoyama, S. Sekiguchi: “Development of the Super Hi-Vision HEVC/H.265 Real-time Encoder,” SMPTE 2013 An-nual Technical Conference & Exhibition (2013)

(2) K. Iguchi, A. Ichigaya, Y. Sugito, S. Sakaida, Y. Shishikui, N. Hiwa-sa, H. Sakate, N. Motoyama: “HEVC Encoder for Super Hi-Vision,” IEEE International Conference on Consumer Electronics 2014 (ICCE 2014), p.61-62 (2014)

(3) Y. Matsuo, T. Misu, S. Iwamura, S. Sakaida: “Ultra High-definition Video Coding using Bit-depth Reduction with Image Noise Reduc-tion and Pseudo-contour Prevention,” IEEE Visual Communications and Image Processing (VCIP) 2013, FP 489 (2013)

1.1.6 Sound systems providing a strong sense of presence

We are researching three-dimensional 22.2 multichannel sound system for 8K Super Hi-Vision (SHV).

■ SHV sound production systemWe are progressing with our R&D on sound production

systems that can efficiently produce SHV sound. In FY 2013, we added a function to save, read, and adjust frequently used production patterns (templates) to a 22.2 multichannel sound mixing system in order to shorten the sound production time. The new function improved the operational efficiency of mov-ing multiple sound sources from side to side or up and down simultaneously. We enhanced the 3D reverberator function-ality by incorporating 22-directional impulse responses (IRs) measured in concert halls, studios, field and various other places. We also explored ways to obtain all reverberation IRs of 22 directions without actually measuring them and devel-oped a method to generate multiple reverberation IRs by shift-ing the reflected sounds in a single reverberation IR randomly on the time axis (1).

We studied the relation between the distribution of spa-tial and temporal reverberation and the spatial impression of sound in order to make it easy to control the spatial impression SHV sound during production.

We continued with our study on the shape and signal pro-cessing of a compact spherical microphone that can be in-stalled in a camera. In FY 2013, we examined a signal process-ing method to compensate for the deterioration in directivity of a smaller microphone (2) and designed and prototyped a device using this method.

■ FPD-integrated reproduction system for home useWe are researching the reproduction of 22.2 multichannel

sound so that its immersive sensation can be experienced at home. We developed a loudspeaker system in which 12 loud-speakers capable of reproducing 22.2 multichannel sound are mounted the front of the frame surrounding the flat panel dis-play (FPD), not on the side or back of the display. By applying the technology of binaural reproduction, we hope to complete this FPD-integrated home reproduction system by 2016.

We conducted subjective evaluations of the sound images reproduced by the FPD-integrated home reproduction system and found that loudspeakers arranged above and below the display are the most appropriate.

For sound image reproduction at the side and back of the display, we developed a method to expand the listening area by improving the binaural reproduction algorithm used by the system. We also developed a technology to help in front-back discrimination of the sound image by controlling the frequen-

cy amplitude response of sound transmitted to both ears. We analyzed the stability of binaural sound reproduction

over the loudspeakers. The results demonstrated the stability increases as the number of loudspeakers increases and that the 12-loudspeaker frame system is more effective than a two-loudspeaker system.

■ Loudspeaker unit for binaural reproductionWe developed a new loudspeaker unit to improve the perfor-

mance of the FPD-integrated home reproduction system. We changed the structure of full-band unit and enhanced the re-production quality. To make up for the lack of bass sound from the small-bore unit, we developed a compact subwoofer with a wider range of lower frequency sound by increasing the weight of the diaphragm. This research was conducted in cooperation with Foster Electric Company, Ltd.

■ Loudness meter for 22.2 multichannel soundWe prototyped a 22.2-multichannel sound loudness meter to

measure the loudness levels that listeners perceive. We tested the device in an SHV public viewing of the popular New Year’s Eve music show, Kouhaku Uttagassen, and confirmed that it could suppress loudness differences between programs.

■ StandardizationAt the April meeting of ITU-R, we proposed common loud-

speaker layouts for channel-based and object-based sound systems and a framework of Recommendations on sound sys-tems, in cooperation with foreign broadcasters and research institutions. At the November meeting, we proposed a draft new Recommendation on an advanced hybrid sound system combining channel-based and object-based methods includ-ing a specification on the 22.2-multichannel sound loudspeak-er arrangement. The draft was approved as an ITU-R Recom-mendation in February 2014 (3).

At the Association of Radio Industries and Businesses (ARIB), we led the effort toward a studio standard for 22.2 mul-tichannel sound. The standard was established in March 2014.

At SMPTE, our efforts led to re-approval of the ST2036-2 standard for 22.2 multichannel sound.

At MPEG, we proposed adding a 22.2 multichannel configu-ration to the MPEG-4 Advanced Audio Coding (AAC) standard to enable 22.2 multichannel sound broadcasting using the MPEG-4 AAC LC profile. Our proposal was approved as a re-vised International Standard (4).

In accordance with the standardization of SHV broadcast-ing standards, we began development of a 22.2 multichannel

NHK STRL ANNUAL REPORT 2013　｜　9

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision

sound encoder using MPEG-4 AAC.

■ “Ultra-reality” meterWe continued with our study on objectively evaluating oth-

erwise subjective factors such as the sense of presence and emotional effect. In FY 2013, we enhanced the model for pre-dicting acoustic impressions from acoustic feature values into a model using time-series data from the start of listening. We found that the prediction accuracy for each instant of time im-proved when we used the top 5% values of time-series data arranged in descending order of acoustic feature values, rather than using their instantaneous values or average values. This research was conducted under contract by the National Insti-tute of Information and Communications Technology (NICT) for its project, “Research and development of ultra-realistic communication technology through innovative three-dimen-sional image technology.”

[References](1) C. Mori, T. Nishiguchi, K. Ono, K. Hamasaki: “Study on the directiv-

ity of microphone for measuring impulse responses used in 3D re-verberator,” Preprint of the Acoustical Society of Japan, 1-P-22, pp. 867-868 (2013 Autumn) (in Japanese)

(2) K. Ono, T. Nishiguchi, K. Matsui, K. Hamasaki: “Spherical micro-phone for Super Hi-Vision 22.2 multichannel sound,” AES 135th Convention, Convention paper 8922 (2013)

(3) ITU-R Rec. BS.2051 “Advanced sound system for programme pro-duction” (2014)

(4) ISO/IEC 14496-3:2009/Amd.4:2013(E) New levels for AAC profiles (2013)

1.1.7 Satellite broadcasting technology

We are researching satellite transmission technologies and multiplexing technologies for 8K Super Hi-Vision (SHV) satel-lite broadcasting.

■ Transmission systems for satellite broadcastingFor ultra-high-definition television broadcasting in the 12-

GHz band, we increased the symbol rate of the satellite trans-mission scheme by reducing the roll-off factor and using 16 Amplitude and Phase Shift Keying (APSK). These improve-ments are part of the ARIB standard, “Transmission system for advanced wide band digital satellite broadcasting” that is under examination by the Information and Communications Council.

We prototyped a transmitter and receiver with a roll-off fac-tor as small as 0.01 and helped to perform the verification tests of their transmission scheme at ARIB. The tests led to ARIB adopting a roll-off factor of 0.03 and symbol rate of 33.7561 Mbaud. We also submitted to ARIB a scheme for 100-Mbps transmissions over a single channel of the satellite transpon-der that uses 16 APSK and a coding rate of 7/9 (an error-cor-recting code parameter).

We conducted transmission tests using the N-SAT-110 and BSAT-3a broadcasting satellites and verified the feasibility of this scheme (Figure 1)(1). The test results were incorporated in a report issued by the broadcasting system committee of the Information and Communications Council. As part of our ARIB related activities, we also studied on-premises cable distribu-tion of ultra-high resolution television satellite broadcasting.

Part of this research was conducted as part of a govern-ment-commissioned project of the Ministry of Internal Affairs and Communications titled, “The research and development project for the expansion of radio spectrum resources”.

We also worked on improving the multi-level coded modu-lation for 8 Phase Shift Keying (PSK). We devised a hybrid set partitioning method, which adjusts the error-correcting capa-bility of each bit by interchanging bits allotted to 8PSK symbols and showed that it could outperform the normal 8PSK system in advanced wide-band digital satellite broadcasting (with a coding rate of 3/4).

Figure 2. FPD-integrated 12-loudspeaker system

Figure 1. Functional verification of production system

10　｜　NHK STRL ANNUAL REPORT 2013　

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision

Markers indicate code rates of: 9/10, 7/8, 5/6, 4/5, 7/9, 3/4, 2/3, 3/5,

1/2, 2/5, 1/3 (from the top)

Required C/N (dB)

QPSK

π/2 shift BPSK

8PSK

16APSK

32APSK

Info

rmat

ion

bit r

ate

(Mbp

s)

0-5 0 5 10 15 20 25

20

40

60

80

100

120

140

160CS satellite return

Figure 1. Required C/N vs. information bit rate in the ARIB satellite trans-mission test

■ Media transport scheme for multiple delivery channelsWe are researching a media transport scheme using mul-

tiple delivery channels without needing to pay attention to the sort of network (broadcasting and broadband) the channels belong to. This scheme is also intended to accommodate the diversifying terminal environment and channels of viewers.

Synchronized presentation of program components trans-mitted through multiple channels has been difficult for con-ventional media transport schemes such as MPEG-2 Transport Stream (TS), which assumes a single channel, and Real-time Transport Protocol (RTP), which does not support multiplexed transmission of program components and signaling informa-tion. In contrast, a multiplexing technology called Moving Pic-ture Experts Group Media Transport (MMT) can easily com-bine high-resolution video transmitted by broadcasting with other program components transmitted over broadband net-works.

We presented the results of our study on a next-generation broadcasting system using MMT at international conferences(2) and demonstrated the effectiveness of MMT as a multiplexing technology for SHV satellite broadcasting.

We conducted a study on migrating the signaling informa-tion used for current broadcasting systems to MMT. The re-sults were incorporated in a report issued by the broadcasting system committee of the Information and Communications Council.

We also worked on international standardization of our re-search results. Part of our proposal was incorporated in the ISO/IEC 23008-1 international standard on MMT.

■ Advanced satellite broadcasting systemWe studied a micro-strip antenna for feeding power to right-

and left-hand circularly polarized receiving antennas for 12-GHz satellite broadcasting. Computer simulations showed that a cross polarization discrimination (XPD) level of over 25 dB could be obtained by using sequentially rotated four-element

arrays. We are developing engineering models of the transponder

and antenna for a 21-GHz-band broadcasting satellite. We prototyped a wideband band pass filter (BPF) and high-power traveling wave tube (TWT) for the 21-GHz band and conducted thermal tests on them(3). We also fabricated a partial prototype of the beam forming network consisting of a 21-GHz array-fed imaging reflector antenna and a prototype of a 32-element horn array and evaluated its electrical characteristics. We also analyzed the surface distortion caused by temperature fluc-tuations of the reflecting mirror. This research was conducted as part of a government-sponsored project of the Ministry of Internal Affairs and Communications, titled “Research & de-velopment of efficient use of frequency resource for next-gen-eration satellite broadcasting system”.

To reduce waveguide loss in the array feeding unit, we stud-ied a thermal transportation method that could deal with the case in which the TWT and output filter are close to each other. We conducted a thermal vacuum experiment on the TWT and filter using a thermal dummy and found that heat transfer to the satellite bus could be improved by using solid-state materi-als for the thermal filter inserted between the heat-generating part and the onboard equipment panels. This research was conducted in cooperation with the Japan Aerospace Explora-tion Agency.

[References](1) Y. Suzuki, K. Tsuchida, Y. Matsusaki, A. Hashimoto, S. Tanaka, T.

Ikeda, N. Okumura: “ARIB Evaluation Tests of Transmission Sys-tem for Ultra High Definition Television Satellite Broadcasting,” ITE Technical Report, Vol. 38, No. 14, pp.33-38, (2014) (in Japanese)

(2) S. Aoki, K. Otsuki, H. Hamada: “Effective Usage of MMT in Broad-casting Systems,” IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, mm13-11(2013)

(3) M. Kamei, Y. Matsusaki, M. Nagasaka, S. Nakazawa, S. Tanaka, T. Ikeda: “Developments of Wideband BPF for Output Filter of 21GHz-band Broadcasting Satellite,” IEICE General Conference, B-3-15 (2014)

1.1.8 Terrestrial transmission technology

We are researching transmission technologies for the next generation of terrestrial broadcasting that will broadcast 8K Super Hi-Vision (SHV) for fixed reception and Hi-Vision for mo-bile reception over a single channel.

■ Transmission technology for fixed receptionWe are researching large-capacity transmission technolo-

gies for terrestrial broadcasting of SHV. We had previously de-veloped a transmission technology combining ultra-multilevel orthogonal frequency division multiplexing (OFDM) technolo-gy, which has a carrier symbol modulation order of up to 4096, with dual-polarized multiple-input multiple-output (MIMO) technology, which simultaneously uses horizontal and vertical polarizations to transmit twice as much information as sin-gle-input single-output (SISO) and measured its transmission characteristics in field experiments.

In FY 2013, we applied transmit diversity technology using space time coding (STC) to single frequency network (SFN) in which two transmitting stations cover a certain area with sig-nals of the same frequency. This method reduced the degrada-tion caused by overlapping reception of waves from two trans-mitting stations. We demonstrated over-the-air transmission of SHV signals using the STC-SFN method at the NHK STRL Open House. To improve the bit error ratio (BER) performance, we designed a circuit that performs a log likelihood ratio (LLR) iterative computation at every sum-product decoding. The cir-cuit performed better than conventional LDPC decoding meth-ods. We studied a spatially coupled low density parity check

(LDPC) code featuring a parity-check matrix with diagonally arranged nonzero elements, devised codes with various code rates, and verified their performance through computer simu-lations. Regarding the carrier modulation scheme, we studied a non-uniform constellation (NUC), in which signal points are distributed non-uniformly and confirmed through computer simulations that the NUC is superior to a conventional uniform constellation. To increase the transmission capacity, we de-signed a circuit to extend the OFDM bandwidth by about 214 kHz (1/28×6 MHz).

■ Long-distance SHV transmission test at Hitoyoshi experimental station

We installed an experimental transmission station for du-al-polarized MIMO transmission (Table 1) at NHK’s TV relay station in Hitoyoshi City, Kumamoto Prefecture and began measurements of the MIMO channel characteristics for long-distance transmissions.

We measured the received electric field strength, delay pro-file, condition number, etc., at 52 sites in Hitoyoshi with the reception antenna set at different heights. At 30 of these sites, we also measured the error rates and the required carrier-to-noise ratio (C/N). We also performed the world’s first long-distance SHV terrestrial transmission(1). The SHV signal was encoded at 91 Mbps by using the MPEG-4 AVC/H.264 scheme and broadcast from the experimental transmission station to a receiver 27 km away in Yunomae Town, Kumamoto Prefec-ture. The video decoded and displayed at the reception point

NHK STRL ANNUAL REPORT 2013　｜　11

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision

showed that SHV content could be transmitted stably over the long distances covered by current digital broadcasting (Figure 1). This research and development is being performed under the auspices of “Research and Development of Basic Technol-ogy Encouraging Effective Utilization of Frequency for the Next Generation Broadcasting System” program and it funded by the Ministry of Internal Affairs and Communications, Japan.

■ Transmission technology for mobile receptionWe are studying 2×2 MIMO-OFDM, which uses two anten-

nas each for transmission and reception, for the purposes of mobile reception.

In FY 2013, we conducted computer simulations to compare the mobile reception characteristics of STC-MIMO, which en-codes and transmits same data from the transmission anten-nas, and Space Division Multiplexing (SDM)-MIMO, which transmits different data from the transmission antennas, at the same transmitting efficiency (number of bits per carrier × error correction coding rate) (2). The results showed that superiority reverses between STC-MIMO and SDM-MIMO at a transmit-ting efficiency of 3 bit/Hz in a mobile reception environment with a 20-Hz maximum Doppler frequency. We also devised a new decoding method that shuffles the decoding order of the maximum-likelihood decoding with QR decomposition. The method improved the characteristics of SDM-MIMO by 0.9 dB, compared with conventional methods (Figure 2).

The transmission characteristics of MIMO transmission

method deteriorate if the two channels from each transmis-sion antenna to each reception antenna are similar. In FY 2013, we measured the channel characteristics for a case in which two pairs of transmission antennas or reception antennas are set at some distance from each other and for a case in which they are installed at the same location and the vehicle drives on a road under non-line-of-sight conditions and analyzed the

Experimental station

Current terrestrial digital broadcasting・Modulation scheme: 64QAM・Use single polarization

Experimental station (UHF46ch)・Ultra-multilevel modulation (4096QAM)・Use dual polarization (Dual-polarized MIMO)

Dual-polarized receiving antenna

4096QAM

64QAMLong distance (27km)

Twice of terrestrial digital broadcastingTwice of terrestrial digital broadcasting

⇒

Dual-polarized transmitting antenna

Large-capacity transmission (approx. 4 times of current digital broadcasting)

Figure 1. Comparison of the experimental station and current terrestrial digital broadcasting

Item

Modulation schemeOccupied bandwidthTransmission frequencyTransmission powerCarrier modulationFFT sizeGuard interval ratio

Error correction code

Transmission station

OFDM5.57 MHz671.142857 MHz (UHF 46ch)Horizontal polarization: 10W Vertical polarization: 10W BPSK, QPSK, 16QAM, 64QAM, 256QAM,1024QAM, 4096QAM8k, 16k, 32k, 64k 1/8, 1/16, 1/32 Inner code: LDPCCoding rate r=2/3, 3/4, 5/6Outer code: BCHEstablished at NHK Hitoyoshi TV relay station

Specifications

Table 1. Experimental station specifications

Transmitting efficiency (bits/Hz)

Req

uire

d C

/N (d

B)

STC−MIMOSDM−MIMO (Conventional method)SDM−MIMO (Proposed method)

30

25

20

15

10

5

00 1 2 3 4 5 6 7 8 9 10

Figure 2. Required C/N of STC-MIMO and SDM-MIMO at the same trans-mitting efficiency

12　｜　NHK STRL ANNUAL REPORT 2013　

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision

space correlation indicating the similarity of channels. The re-sults showed the median value of the spatial correlation is 0.2 or less regardless of the distance between the antennas, which means the channel characteristics are different, and this dem-onstrated the effectiveness of MIMO transmission.

We are studying a method to multiplex transmission signals for fixed reception and mobile reception using frequency divi-sion multiplexing. With this method, however, it is necessary to compensate for the degradation caused by inter-carrier-inter-ference (ICI) from Doppler effects during mobile reception. In FY 2013, we used a basis expansion model for the channel es-

timation and conducted computer simulations to evaluate the mobile reception characteristics when belief propagation is used for ICI compensation. The results showed that is scheme improved the required C/N by 3 dB at 100 km/h compared with the conventional channel estimation without ICI compensa-tion and that it is an effective compensation method (Figure 3).

■ Advanced Television Systems Committee (ATSC) 3.0 standards

We submitted a proposal for ATSC 3.0, the U.S. standards for next-generation terrestrial digital broadcasting (3), and con-tributed to its standardization activities. Our proposal includes an OFDM scheme with a segmented structure capable of hier-archical transmission, which is the feature of Integrated Ser-vices Digital Broadcasting-Terrestrial (ISDB-T), dual-polarized MIMO and MISO, ultra-multilevel modulation up to 4096QAM, spatially coupled low-density parity check (LDPC) code, NUC mapping, inter-polarization interleaving and dispersion, and the Space Time Coding (STC) - Single Frequency Network (SFN) method.

[References](1) NHK INFORMATION: “8K (Super Hi-Vision) Long-distance transmis-

sion test is successfully achieved,” http://www.nhk.or.jp/strl/eng-lish/data/20140203.pdf

(2) Y. Narikiyo, H. Miyasaka, M. Nakamura, H. Sanei, M. Takada: “Com-parison of SFBC and SDM MIMO reception performance under mo-bile reception environment by computer simulations” ITE Technical Report, Vol. 38, No. 5, pp. 113-116 (2014) (in Japanese)

(3) ATSC: “Summaries of Responses to ATSC 3.0 Physical Layer Call for Proposals,” http://www.atsc.org/cms/index.php/the-news/327-sum-maries-of-responses-to-atsc-30-physical-layer-call-for-proposals

1.1.9 Wired transmission technology■ Optical transmission of uncompressed SHV signals

We are researching a system to transmit uncompressed Super Hi-Vision (SHV) signals over optical fibers to avoid de-lays or image degradation resulting from image compression. The system is for transmitting program contributions within and between broadcast stations or from relay destinations to broadcast stations.

In FY 2013, we developed a method to transmit a 72-Gbps full-resolution SHV signal over eight wavelengths (10 Gbps per wavelength). Long-distance transmissions over optical fiber are prone to errors that degrade signal quality. To enable lon-ger distance transmissions, we developed a technology to add error-correcting code to the signals. With this technology, the maximum transmission distance can be extended from 80 km to approximately 100 km.

For our study on SHV transmission over Ethernet, we de-veloped a control method to reduce the jitter of video clock

reproduced by the receiver and demonstrated its effectiveness through simulations and experiments using 10-gigabit Ether-net (10 GE).

We also contributed to a study on interfaces for UHDTV production systems that is being conducted by a Ultra-high-definition TV studio facilities working group at the Association of Radio Industries and Businesses (ARIB).

■ Cable TV transmission of SHV signalsWe are researching a method to transmit partitioned SHV

signals over multiple channels (1) so that SHV programs can be distributed through existing coaxial cable television networks. In FY 2013, we designed the service information structure of transmission and programs to receive the signals correctly and gave the results of our study to the Japan Cable Television Engineering Association (JCTEA).

We also developed a method to time-multiplex multiple digi-tal broadcasts in a baseband signal and transmit it over an op-tical fiber, as a way to distribute digital broadcasting to homes over FTTH (Fiber To The Home). In FY 2013, we prototyped test equipment to transmit multiple programs including SHV sig-nals over an optical network using 10 GE technology.

[References](1) Y. Hakamada, N. Nakamura, T. Kurakake, T. Kusakabe, K. Oyamada:

“UHDTV (8K) Distribution Technology and Field Trial on Cable Tele-vision Networks,” ITE Trans. on MTA, Vol. 2, No. 1, pp. 2-7 (2014)

Moving speed (km/h)

Req

uire

d C

/N fo

r BER

=10-7

af

ter L

DPC

dec

odin

g (d

B)

FFT size: 32kCarrier modulation method: QPSKError correction: LDPC codeCoding rate: 1/2Transmission frequency: 600MHz

with plain channel estimationwith ICI compensation

15

10

5

00 50 100 150

Figure 3. Characteristic improvement due to ICI compensation

Receiver

Transmitter

Reproduced video signals

Regenerated clock frequency for video signals

Video signal transmission system using Ethernet

Figure 1. Clock regeneration test in Ethernet transmission

NHK STRL ANNUAL REPORT 2013　｜　13

1　Television conveying a strong sensation of reality　｜　1.1 Super Hi-Vision