Previous CEO/Founder NGCodec Advisor to Xilinx Data Center Group
RECENT TRENDS IN LIVE CLOUD VIDEO TRANSCODING
Agenda
NGCodec History
Cloud Transcoding Market
Benefits of Hardware encoding
Conclusions
Q&A
2
NGCodec History
NGCodec History
● 2012 - Founded on the back of HEVC/H.265, Silicon IP licensing business● 2013 - Failed to close funding with IP business model● 2014 - Pivot to FPGA Cloud encoding with Saas, Seed $3M from Belmore cap● 2015 - License of FPGA encoder to HaiVision● 2016 - Xilinx 1st round of investment, First company to demo FPGA on AWS ● 2017 - Starts VP9 development, commits to all CODEC and ASIC● 2018 - Xilinx 2nd round of investment, Twitch customer deal signed● 2019 - Demos 4K AV1 Intra live encoder, Acquired by Xilinx
AV1
A V C
V P 9
C h e sV P 9 2
NGCodec History (Founded 2012)
2015 2016 2017 2018
C a sh e w (H E V C )
P in e (H E V C )
C h estn u t 1(H E V C )
C h estn u t 2(H E V C )
2019
C h estn u t 3(H E V C )
● I&P fra m es● 1 ref fra m e● 10 8 0 p60
+ B fra m es+ Ad a ptive AQ
+ 2 ref fra m es+ U ltra low la ten c y+ 4 k p30 + TM V P
+ L ook a h ea d+ M u ltip le g op s izes+ AB R
+ V a ria ble g op+ F ra m e level
F W c on trol (L eg o)
+ C on ten t a d a ptive
+ 4 K p60
7 ye a rs of d e ve lop m e n tM u ltip le ite ra tion sM u ltip le sta n d a rd sM u ltip le c u stom e rs
M u ltip le g ra n te d p a te n ts~4 0 P e op le
Encoder Simplified Block Diagram mm2c
Inter Cost
Intra Search
Select ReconMC
Recon-struct
SAO Filter CABAC
Deblock Filter c2mm
NalHu es2mm
PredCache 1*
Inter IRefine
1Inter IRefine
0
Inter SRefine
1Inter
SRefine 0
PredCache 0*
Master AXI DDR interfaces
Bitstream outReference
frame outRaw pixels in
Reference frame in
Reference frame in
All blue blocks are HLS modules
Cloud Transcoding Market
Codec Adoption Forecast by Number of bits/s
● AVC/H.264 was standardized in 2003
● Other things that were new in 2003
○ Sony Trinitron Wega KV-20FS100
○ Blackberry Quark 6210
○ Thinkpad X31
H.264 Surprising Result?
9
● It works everywhere!
● Open source ecosystem
○ x264 is a very good H.264 encoder
● Mature workflows for content creation
● Momentum in the industry
● Predictable licensing model
Why is so much content encoded in H.264?
10
Growth in Video Processing / Costs
50 % C AG R in S tream in g C ost, 10 x in 20 24 !
Streaming Cost Growing 50% CAGR
TC O for s p e c ific vid e o e n c od in gD rive rs : B itra te for s p e c ific V Q
B a n d w id th to C D N c os ts($ 0 .0 4 p e r G B yte , $ M m on th )
S tora g e c os ts($ 0 .0 2 p e r G B yte , $ M m on th )
C om p u tin g TC O e n c od in g /M L c os ts($ 1.0 0 p e r H ou r , $ M M on th )
B a n d w id th a n d s tora g e c os ts(S e rv ic e p rovid e r C D N & c on s u m e r d a ta p la n )
Q u a lity of e xp e rie n c e (S ta rtu p tim e , v is u a l q u a lity , s ta lls )
W h y V is u a l Q u a lity (V Q ) M a tte rs
HUYA 6-K 11/13/2018 (Twitch of China)
“Bandwidth costs increased by 66.8% to US$25.3 million 3Q2018 from same period of 2017, primarily due to an
increase in bandwidth usage as a result of increased user base on Huya’s
platform and enhanced live streaming video quality improvement”
Benefits of Hardware encoding
Encoding Computing Landscape
Performance & Power Efficiency
Flex
ibili
ty &
Eas
e of
Use
LibVPx
CPU (90%)
FPGA
Fixed Instruction setRigid memory hierarchy
Adjust algorithm to architecture
Flexible fully customizable architectureAdjust architecture to algorithm
ACAP
CPU are too slow for live encoding
SVT-HEVC (Intel)
Why software is too slow for live video encoding● Brute force does not work
○ Full RDO mode decision is required
○ Smart algorithms needed for uniform & balanced VQ
○ One imperfection can destroy the VQ results
○ Same performance at 4K as 1080p
Google/Twitch/NGCodec technical blog
https://blog.twitch.tv/how-does-vp9-deliver-value-for-twitchs-esports-live-streaming-35db26f6322f
http://subjectify.us
M6
Cloud Transcoding with FPGA’s
TCO & VQ Comparison Total Cost of Ownership (TCO) per 1080p60 channel per year,3 Year Amortization Servers, 10 Years Amortization Space, 24c per KWh
R e fe re n c e +4 5% -20 % -30 % -4 0 %B IT R A T E D IF F E R E N C E
F O R S A M E V Q
Conclusions● Cloud Video costs are grow at 50% CAGR
○ Majority done by Internet Hyperscalers
○ Bandwidth and compute costs
● CPU’s are too slow for live encoding
○ Have to use ‘faster’ presets
○ ‘Slower’ presets not possible for live & expensive
● Programmable hardware (FPGA) delivers best VQ
○ Available today in many public/private Clouds
○ Roadmap to ASIC
Q u e s t io n s ?