1
Agenda
Quality ChangeResolution / fps / Color DepthCodecs
New MediaCoding MachinesCPU->GPU / Cloud
2
3
Improving Video Quality
4
Increase temporal resolution
More frame: 24 ~ 100 Fps
Increase view points, viewing angle
5
More View (3D): Stereo, Multiple view, free view
Increase views – Virtual & 360
1 view 2 views M views -> 360
…
Improving Video Resolution
More pixels: SD – HD –UHD (4k, 8K)
Increase spatial resolution
HDTVUHDTV
SDTV
Moving to HDR
HDR quality is noticeable from any distance and screen sizeResolution is only apparent when you are close
HDR is more important for video quality than 4K (Netflix)
Improving Video Quality
More pixel’s info (Better pixel)
Higher contrast: SDR vs. HDR
Standard(SDR)
Enhanced(EDR)
High(HDR)
10 16 f-stops
n f-stops = a difference of 2n : 1 contrast ratio
Quantization bit: 8 – 10 – 12
strong contrast between the bright parts and the dark parts of an image
9
SDR
HDR
Problem of Static Range
10
10-6 106
10-6 106
Picture
High dynamic rangeReal world
0 to 255
10-6 106
10-6 106Picture
High dynamic rangeReal world
0 to 255
Wider Color Gamut
Current color gamut is limited Current HDTV (BT.709) – 1990 Created for CRT, and HDTV Around 100 nits. 8 bpp
11
More Colors: REC. 2020 for UHDTV 10, 12 bpp.
12
12
Media Coding trends
CPU Coding -> GPU Coding PC->Cloud Private API from Intel & Nvidia Pulic/Standardized API
OpenVX OpenCL FFMPEG
Moore’s Law
Serial scaling performance has reached its peak.
Processors are not getting faster, but wider
GPUs
CPU vs GPU
GPU devotes more transistors to data processing
GPU Video coding options
Intel Intel Media SDK API – High level APIFEI API – Lower level API which enables improved coding
controlNvidia
NVEncode SDK – High level APIFFMPEG
H264_QSV Encoder (and decoder) filter for Intel GPUNVEnc filter for Nvidia GPU
Comparison Quality vs Performance
Source: http://on-demand.gputechconf.com/gtc/2016/presentation/s6226-abhijit-patait-high-performance-video.pdf
Application
Increased usage of live videoTwitch, YT GamingFacebook live
Music as a monetization featureYoutube is turning to music
AR and VR
19
Audio Trends
We see regular streaming going toward higher bitrates with the following formats:
FLAC / ALAC – usually 1411Kb (already exists in some sites). Lossless compression of PCM
MP4 AAC SLS – MP4 AAC Scalable Lossless codec. A relatively new format from 2014. The Codec is made of a base layer encoded as basic AAC with one or more enhancement layers
MQA - Master Quality Authenticated – A new codec for lossless audio
Hi-Res Music – A new “Quality certification” by RIAA Requires streaming with MQA or MP4 AAC SLS coding About 1.5Mb Bitrate Peeling
Oclus RIFT
Audio is usually 4 separate channels but streamed together in a single Ogg file unlike 5.1 for example which are jointly coded channels
No possibility for Audio throttling
22
22
HTTPS, DRM, SPDY, QUIC
Increased use of Encryption and DRMExpect 80% of video traffic to be encrypted/DRMedMove toward ABR
SPDY -> HTTP2.0 (not much of a difference) QUIC traffic might increase to 10% Of total traffic
23
24
Thank you!More About me:
Yossi CohenYossi [email protected]@gmail.com+972-545-313092+972-545-313092
Video compression, computer vision and Machine Learning instructor
