From Zero to Zero-CopyAccelerating Your Embedded Media Player

Lukas RusakSeptember 2018

whoamiLukas Rusak

Bachelors of Applied Science in Mechanical Engineering

Team Kodi Member

Team LibreELEC Member

LibreELEC Board Member

Okanagan Region of B.C.

lrusak@kodi.tv

Mission Hill Vineyard

Kelowna B.C and Okanagan

Lake

Agenda

● Introducing zero-copy● Current proprietary methods● Open source solutions● Video decoding pipeline● Rendering methods● Board specifics

Zero-copy

"Zero-copy" describes computer operations in which the CPU does not perform the task of copying data from one memory area to another. [1]

● Embedded devices with minimal CPU power trying to run 4K HEVC video

● Critical for devices with limited memory bandwidth

● Available in proprietary and open source solutions

Current Proprietary Methods

● Raspberry Pi (broadcom) dispmanx○ libbcm_host

● i.MX6 Vivante○ Blob that does everything for windowing and rendering○ libvpuwrap for decoding

● Amlogic libamcodec○ Userspace library dependent of vendor BSP Kernel (3.14)

● Impossible to fix bugs● Locked to vendor kernels● No chance to update● Code maintenance for vendor specific methods

Open Source Solutions

● Video 4 Linux 2 (V4L2) Driver○ Access to onboard decoding capabilities○ Memory-to-Memory devices○ Codecs are hardware and driver dependent

● Direct Rendering Manager (DRM) Driver○ Only required if utilizing direct to plane rendering

● OpenGLES 2.0 and EGL○ Only required if utilizing EGL image import to OpenGLES○ EXT_image_dma_buf_import [2]○ OES_EGL_image_external [3]○ NV12 or YUYV or YU12 or similar YUV format support

Video Decoding Pipeline

● V4L2 interface directly using IOCTL’s● Gstreamer

○ dma-buf ready● FFmpeg

○ dma-buf patches under review

● Using Gstreamer or FFmpeg shifts video decoding out of your application

Decoded Frame

AVDRMFrameDescriptor

Compressed Frame

Decoder

fd

Direct to Plane RenderingPositives

● Embedded devices may support YUV formats directly

● Most performant (no empirical evidence)● No OpenGLES/EGL context required

○ Skips the GPU entirely● Video overlays are possible using DRM Planes● Very straight forward when using atomic DRM

Negatives

● Requires* in driver scaling capabilities● No OpenGLES capabilities (shaders)● More complex if using multiple planes

Video + GUI

GUI Only

Primary Overlay Cursor (DRM Planes)

OpenGLES/EGL Image Import

Positives

● Shader support● Only one DRM Plane required

Negatives

● More expensive (requires EGL context and GLES operations)● Requires extensions that are less common

Mali Proprietary Blob

● Open source kernel module paired with closed source binary blob for OpenGLES

● Licensed from ARM● Bundled library containing either:

○ x11○ Wayland/GBM○ GBM

● Impossible to fix bugs● Lima and Panfrost open source efforts underway

Allwinner

● DRM Driver○ “sun4i”○ Supports scaling○ Supports overlays○ Supports NV12

● V4L2 Driver○ Patches pending (requires V4L2 request API)○ Supports NV12

● OpenGLES Driver○ “mali”○ Supports EGL import○ Supports NV12

Orange Pi PC

Amlogic

● DRM Driver○ “meson”○ Supports Scaling (WIP)○ Supports Overlays (WIP)○ Supports NV12○ Neil Armstrong <narmstrong@baylibre.com>

● V4L2 Driver○ “Meson” (WIP)○ Supports NV12○ Supports HEVC, H264○ Maxime Jourdan <mjourdan@baylibre.com>

● OpenGLES Driver○ “mali”○ Supports EGL import○ Supports NV12

Libre Computer Le Potato (S905)

Broadcom

● DRM Driver○ “vc4”○ Supports scaling○ Supports overlays○ Supports NV12 and YUV420

● V4L2 Driver○ “Bcm2835-v4l2-codec” (WIP)○ Supports NV12 and YUV420

● OpenGLES Driver○ “vc4” ○ No YUV import

Raspberry Pi 3B+

NXP

● DRM Driver○ “imx” and “etnaviv”○ No scaling (available via a separate ip)○ Supports overlays○ Supports NV12

● V4L2 Driver○ “coda”○ Supports NV12

● OpenGLES Driver○ “etnaviv”○ Supports EGL import○ Supports NV12 (WIP)

■ Target mesa 18.3■ gc3000 good performance■ gc2000 ok performance

Solidrun Cubox i4-Pro (gc2000)

Qualcomm

● DRM Driver○ “msm”○ Supports scaling○ Supports overlays○ Supports NV12

● V4L2 Driver○ “venus”○ Supports NV12

● OpenGLES Driver○ “freedreno”○ Supports EGL import○ Supports NV12

Dragonboard 410c

Others

HiSilicon

MediaTek

Renesas

Rockchip

Asus Tinkerboard S (RK3288)

Renesas R-Car

Index

1. https://en.wikipedia.org/wiki/Zero-copy

2. https://www.khronos.org/registry/EGL/extensions/EXT/EGL_EXT_image_dma_buf_import.txt

3. https://www.khronos.org/registry/OpenGL/extensions/OES/OES_EGL_image_external.txt

Kodi Implementation Links:

https://github.com/xbmc/xbmc/tree/master/xbmc/windowing/gbm

https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererDRMPRIME.cpp

https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/VideoRenderers/HwDecRender/RendererDRMPRIMEGLES.cpp

https://github.com/xbmc/xbmc/blob/master/xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecDRMPRIME.cpp

Q/ATheme By: Samfisher Design