Remote Android Rendering

Joel Isaacson

Ascender Technologies Ltd.

Copyright 2014 Joel Isaacson

The Problem

There are just too many pixels to simply transmit over a long-haul network.

There are a number of techniques that have been attempted.

They all entail some compromises: Resolution Accuracy Frame Rate Latency

Remote RenderingAscender Technologies Ltd

The Problem:Pixel Count 2008-2011

Copyright Romain Guy, Chet Haas, Google I/O 2011Ascender Technologies

LtdRemote Rendering

The Problem:Pixel Count 2008-2012

Ascender Technologies Ltd Remote Rendering

The Problem:Pixel Count 2008-2014

Ascender Technologies Ltd Remote Rendering

Pixel Count 2008-2014

Ascender Technologies Ltd Remote Rendering

DisplayCommercialAvailability

Rows Columns Pixels

DVD 1997 720 480 345,600

720p 1998 1289 720 921,600

1080p 2006 1920 1080 2,073,600

Nexus 10 2012 2560 1600 4,096,000

4K Ultra HDTablet

2014 3840 2160 8,294,400

Android Graphics Stack

Ascender Technologies Ltd Remote Rendering

Choosing How ToExport Graphics

Graphics can be exported from any of the four levels of the graphics stack Application level Toolkit level Rendering level Pixel level

We choose to export the rendering level.

Remote RenderingAscender Technologies Ltd

Exporting The Toolkitand App

It is technically very complex. Android, to date, has 17 different toolkit API variants.

Every application can extend the toolkit with custom widgets (subclasses of android.view.View).

Clearly impossible.

Remote RenderingAscender Technologies Ltd

Exporting The Toolkitand App

It is technically very complex. Android, to date, has 17 different toolkit API variants.

Every application can extend the toolkit with custom widgets (subclasses of android.view.View).

Clearly impossible. Effectively exporting graphics at the toolkit

level and even the application level is in fact possible via the rendering API.

Remote RenderingAscender Technologies Ltd

The GUI Rendering Layer Expanded

Ascender Technologies Ltd Remote Rendering

NDK App With OpenGL

Ascender Technologies Ltd Remote Rendering

NDK App With Skia Rendering

Ascender Technologies Ltd Remote Rendering

Android Remote Rendering

Ascender Technologies Ltd Remote Rendering

Android Remote RenderingNo GPU on Server Needed

Ascender Technologies Ltd Remote Rendering

ICS Rendering Results

Even with simple techniques the compression ratio is over four orders of magnitude (x10,000 reduction).

The number of bytes per frame, for the GUI rendering, is typically 300 bytes, as opposed to 4-16 Mbytes for uncompressed frames.

The compression encodes 2-4 rendering operations per byte (2-4 bits per rendering operation).

Remote RenderingAscender Technologies Ltd

The Google Play UniverseAPI Coverage

Remote RenderingAscender Technologies Ltd

Cloud Gaming

Currently cloud gaming is done with pixel rendering performed on the remote server. The frames are H264 encoded and sent over the network to the remote client.

Our remote rendering technology does not need special hardware on the server side. The computational load on the server and network are minimized.

Playing latency (lag) is minimal.

Remote RenderingAscender Technologies Ltd

A Perfect Storm

It seems that a technological cosmic alignment has happened: Fast, low-power 64 bit ARM multi-processors

(Cortex A50) with virtualization extensions. Adoption of Android apps in a broad gamut of use

cases, including the enterprise. Ever increasing adoption of cloud based solutions. Possibility of efficiently transporting Android

graphics via a long haul network.

Remote RenderingAscender Technologies Ltd