Adobe Session on Flash Online Conference #12

© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

Video and Camera Support for Stage3D using Video TextureNimisha/Chandra Prakash | AIR Engineering


Overview

2

How Stage3D fits within the Flash display model? Problem at hand: using video as a content in 3D scene Current method to fix the problem at hand New approach to solve the problem How to use the new approach? Use cases


Stage3D: The stage behind the Stage

3

Set of special Stages that are specifically dedicated to 3D


Demo

4


How to fix : Bitmap representation of video

5

Problems Requires multiple copying of

bitmap Slow and Resource Intensive

Video rendered in RGB on a Video Object

Video rendered in RGB on a Video Object

Bitmap Representation of Video Bitmap Representation of Video

Create a TextureCreate a Texture

Update the texture with the bitmapUpdate the texture with the bitmap


New Approach - Video Texture

6

Directly access texture from netstream or camera

Directly access texture from netstream or camera

Advantages Source texture in stage 3D

pipeline Fast and Hardware accelerated


Video Texture – How to use?

Confirm if VideoTexture is supported by Context3D class

7

if( stage.stage3Ds.length > 0 ){ var stage3D:Stage3D = stage.stage3Ds[0]; stage3D.addEventListener( Event.CONTEXT3D_CREATE, Context3DHandler ); stage3D.requestContext3D( ); } function Context3DHandler ( event : Event ) : void{ if (Context3D.supportsVideoTexture) { // We can create VideoTexture Object with Context3D object. }}



Create VideoTexture object and attach NetStream/Camera object to created VideoTexture object

8

var ns:NetStream;

var context3D:Context3D;

var texture:VideoTexture;

texture = context3D.createVideoTexture();

texture.attachNetstream(ns);

texture.addEventListener(Event.TEXTURE_READY, renderFrame);

ns.play("http://www.adobe.com/AIR/videoTexture/demo.mp4");


Demo

9



Video frame corresponding to current audio playback Callback for an event Event.TEXTURE_READY is fired Use an updated VideoTexture object with currently available video frame

10

var context3D:Context3D;

function renderFrame(e:Event):void

{

// Render on stage3D with VideoTexture

}



VideoTextureEvent object that works same with the StageVideoEvent event

11

texture.addEventListener(VideoTextureEvent.RENDER_STATE, onRenderState);

function onRenderState (event:VideoTextureEvent)

{

if ( event.status == VideoStatus.SOFTWARE) {

// Indicates software video decoding works.

}

if ( event.status == VideoStatus.ACCELERATED ) {

// Indicates hardware-accelerated (GPU) video decoding works.

}

if ( event.status == VideoStatus.UNAVAILABLE ) {

// Indicates Video decoding is not supported.

}

}


Video Texture – Use Cases

12

Camera Input

In-game video call with shader effects

Efficient embedding of camera streams

Video Ads

In place Video Ads

In game video streams containing streamed content or live content or commercials

Credit: Waste Invaders


Roadmap

Implementation of video texture on iOS We have already signed off the feature for iOS and the developer community is using it

Implementation of video texture on Android Feature is ready for release and is available in labs for developers to try out video texture on Android The feature would most probably be usable in a full fledged way in our next release

13


Questions

14


ETC2 Texture Compression (Ericsson Texture Compression)Ridam Batra | AIR Engineering


GPU TEXTURE COMPRESSION

16

CPU GPU

Upload of textures from CPU to GPU is a very slow operation

GPU can understand block compressed texture formats like etc1,pvrtc,etc2,dxt etc.

Compressed Textures

Texture Compression is the ANSWER.


ADOBE TEXTURE FORMAT

17

• Container format for compressed images. • AIR or Flash Player automatically extracts the appropriate texture depending on platform.


Problem

18

ETC1

(android) Size on CPU – 170KB Size after GPU upload - 170KB

PVRTC (ios)Size on CPU – 170KBSize after GPU upload - 170 KB

Original png image Size on CPU – 256KB Size after GPU upload - 1048KB

DIFFERENT RENDERING RESULT ON IOS AND ANDROID !!!!


SOLUTION

OpenGL ES 3.0 introduced high quality ETC2 compression as a standard feature. It works on ios and android both.

19

Original png image

1048 KB

ATF with ETC1 + PVRTC

340 KB

ATF with ETC2170 KB

SIMILAR RENDERING RESULT ON IOS AND ANDROID SIMILAR RENDERING RESULT ON IOS AND ANDROID

SIZE OF TEXTURES FURTHER REDUCED

SIZE OF TEXTURES FURTHER REDUCED

AND AND


Demo

20


Comparing The Two Applications

21

Size of APK with PNG files - 37.7 MB Size of APK with ATF Files - 21.2 MB

App With PNG Images GPU Memory for textures– 73,738 KB

App With Atf Files GPU Memory for textures– 9,216 KB

GPU MEMORY


Questions

22


64-bit Support & Parallel Compilation for iOSAbhinav Dhandh | AIR Engineering


64-bit support for iOS applications

Work on supporting 64-bit binary feature for iOS applications Support for newer A7+ processor devices:

Additionally, Apple issued new guidelines for app submission

iPhone 5s iPhone 6 iPhone 6+ iPad Air Mini iPad Air iPad Air 2


Universal binary was the way..

SWFs ABC Files

32-bit Compilation and Linking

Executable File

IPA Combine

ADT

Additionally, stability of New compiler has been improved Major bug fixes for real world applications

64-bit Compilation and Linking


But..

Average Packaging Time Increase : 93%


Parallel Compilation for packaging time reduction:

27

SWFs

ABC Files

ABC Files

32-bit Compilation and

Linking

Executable File

IPA

Combine

ADT

Multi-threaded Compilation

Executable File

Combine


Linking32-bit

Compilation and Linking


Linking64-bit

Compilation and Linking64-bit

Compilation and Linking


Packaging time comparison

Average reduction in packaging time : 44%


Questions

29


Standard Extended ProfileMayank Bhagya | AIR Engineering


Features

Enhanced register set

Shader size

31

AGAL v2 AGAL v3

Vertex Attributes

8 16

Fragment Constants

64 200

AGAL v2 AGAL v3

Tokens 1024 2048


A sample shader

32


Usecases

33

Complex shaders

© Jojo Nicado

Water© Dom Crossley

Fog


Demo

34


Adobe Graphics Assembly Language - v3

Standard Extended Profile Mobile – AIR v17 Desktop – FP & AIR v18

Mini Assembler @ GitHub https://github.com/adobe-flash/graphicscorelib

35


Questions

36


Date post:	05-Aug-2015
Category:	Technology
Upload:	flash-conference
View:	539 times
Download:	5 times

Adobe Session on Flash Online Conference #12

Technology