+ All Categories
Home > Documents > Special considerations for Standards in Mobile AR

Special considerations for Standards in Mobile AR

Date post: 15-Feb-2016
Category:
Upload: dani
View: 36 times
Download: 0 times
Share this document with a friend
Description:
Special considerations for Standards in Mobile AR. International Workshop on AR Standards – 11. + 12. October 2010, Seoul [email protected] - Institute for Graphics Research IGD. Fraunhofer IGD Institute for Computer Graphics Research. - PowerPoint PPT Presentation
Popular Tags:
27
© Fraunhofer IGD Special considerations for Standards in Mobile AR International Workshop on AR Standards – 11. + 12. October 2010, Seoul [email protected] - Institute for Graphics Research IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea
Transcript
Page 1: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Special considerations for Standards in Mobile ARInternational Workshop on AR Standards – 11. + 12. October 2010, [email protected] - Institute for Graphics Research IGD

Page 2: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Fraunhofer IGDInstitute for Computer Graphics Research

Mission: Technologies and Applications of Visualization, Interaction and Communication in the field of ICT

Director: Prof. Dr. techn. Dieter W. Fellner 4 Locations: Darmstadt, Rostock (Germany), Singapore and Graz

(Austria) 180 Researchers (FTE) 14,7 Mio. € Turnover -> Department Virtual and Augmented Reality

Page 3: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Dep. Virtual & Augmented RealityResearch Topics

Computer Vision

Computer Graphics

Interaction Technologies

Page 4: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Overview

The aim of the presentation is triggering consciousness about the idea of AR and presenting an open approach practically ..

Where are the problems in standardization of AR? What are the common structures and where are variations? What is the role of those structures in the context of AR? Technological hardware issues Why and what kind of standards are there and do we need? Our approach to overcome the mentioned issues and help

developing standards

Cannot cover the whole horizon of problems .. But can try …

Page 5: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

ProblemsGeneral goal of having standard(s) which apply to general needs of AR.

What is the problem? There are tracking technologies for AR, but they are in an early

stage. Every application type needs its custom strategy, technology and specialized content.

What do we want to cover with a standard? Due to the complexity of many parties involved in the field of AR and

the high potential of possibilities. The terminology AR is not clearly differentiated generally and can

lead to confusion. Which sense is augmented by what and through what? 2D,3D, training, leading, informing, disinforming, helping, covering

etc.. Diverging hardware and software technologies Bringing the paradigm to content masses

Page 6: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

CommonTechnologies

CommonTechnologies

Arising potentials of technological developmentin contrast to commonly used technologies in apps Mass -

Solution

Customized Mass-Solution

CommonTechno-logies Cutting

Edge-Solution

Scientific SpecificLaboratory-Solutions

Time

Development

Provide tools for everyoneLayar, Wikitude etc..

Branded…i.e. iLiving

Complexity

Page 7: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Augmented RealityAbstract Building Blocks

ContextEngine

Content

AnnotationEngine

ContextInformation

Interaction Engine

ApplicationModel

Reference Model Annotation Model

Sensors

Output

InteractionDevice

Page 8: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Computer Vision Based Augmented Reality

Camera

VisionEngine

Content

3D AnnotationEngine

Extrinsics (+ Intrinsics,Occlusions,Lights,Models etc..)

Interaction Engine

Image+Intrinsics

Display ImageScreen

ApplicationModel

Reference Model Annotation Model

Page 9: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Role of Reference Model

Retrieval of the camera Pose (6Dof) Find the absolute coordinates of user and objects

GPS, Compass, Acceleration Data Use prototype model for registration (i.e. Poster, Marker, Lines…)

Representation Absolute in latitude, longitude, altitude, roll, pitch, yaw Relative in Cartesian coordinates and angles (or axis angle, quaternion..) In relation to some object or object group – real or virtual?

Interaction Find differences of formulated and real world (POI,AOI,VOI)

This creates potential content for interaction Track movable and deformable objects Detect user interaction/behavior using sensors (Hands, gestures etc..)

Allow feedback of generated data

Page 10: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Role of Visual Annotations in ARBuilding elements in GUIsVisual Aids Direct Visual Aids (DVA): Superimposed 3D-models/animations

(direct association with object) Indirect Visual Aids (IVA): “Annotations” related to tracked object

(reminder)

Mental Model Building User’s internal representation of the task Can be improved by providing context information

Passive Parts Information about the task in general Higher-level instructions global picture

Collaborative Annotations (expert trainer)

DVA

IVA

Page 11: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Role of the interaction model

Customizing the Annotation Model Manipulation Scene Graph Interactive Content

Customizing the Reference Model Dynamically change tracking behavior and point of focus (POI) Allow interchange in between Annotation and Registration Model Use of declarative formulation as far as possible. Allow scripting for specialization Tracking and Events: 2D-Projected Positions with distance hint Detection Events (i,e. Object enters/leaves display) Interaction of user/objects with the scene

Content must be delivered free or encrypted.. Who will do this?

Page 12: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Hardware technology issues on mobile devices

Low computational resources in relation to modern PCs Limited memory OpenGL ES Different camera modules (Intrinsics) Sensor support and calibration Partly supported hardware acceleration Software Interfaces and Languages Symbian (C++, Microkernel, Qt) Android (Java, C++ through NDK) iPhone (Obj-C, C++ support)

Algorithms need to be optimized for HW Costly and hard to maintain large set of handsets in one common

Code-Base

Page 13: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Open StandardsSustainability for AR-ContentApplications and generation tool chains should be based on.. X3D (ISO/IEC 19775:2004) HTML5 ECMAScript (ISO/IEC 16262:2002) JavaScript CSS STEP-Format KMLBenefits Sustainability Interoperability Porting Rapid development

Page 14: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

X3DOM (x3dom.org)HTML5/X3D Integration Model

Simple car configurator to interactively change color and rims

N900 with miniB

Page 15: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea15

X3DOM (x3dom.org)HTML5/X3D Integration ModelX3DOM: experimental open-source framework and runtime One way for integration of HTML5 and declarative 3D Ongoing discussion in the Web3D and W3C communities

Declarative content design Allows embedding 3D data directly into the HTML-DOM First approaches of CSS integration Application concepts map well to generic scene-graph

Runtime independent (supports native implementation, plugin, WebGL)Suitable for desktop and mobile systems alike

Page 16: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

instantReality – mobileAR - BrowserExtendable generic AR-Web-Browser for mobile devices based on HTML5 and Javascript

Page 17: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

mobileAR – Open Architecture Proposal for open realization of declarative approaches

Generic Application (arBrowser)OS

WEB-View (HTML5 +JS)

DOM-Declarative

JavaScript

IMA

X3Dom

webGL

SVG A2ML

Native Interface

VisionRender Sound

Open Annotation Open Context

Content

AR-KML

??

Sensors

Abst

ract

ion

Individualization of App

Reac

habi

lity

for m

asse

s of

deve

lope

rs

Page 18: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

mobileAR – AR – Browser for mobile devicesAdditionally to Web-Browser a renderer and vision engine is integrated. Extends the internal Web-Browser with a simple but powerful API Interaction and visualization can be done declarative in HTML and/or

via JS Can render X3D via native renderer or even X3Dom Generic Vision Interface Integrates Vision Processor (VisionLib) for

Fiducials (Markers, Image-Markers) Poster-Tracking Object recognition Panorama and 3D-Model Based scene tracking Server based Tracking (bidirectional streaming)

Additional Features (about to come…) Remote Video & Audio Support Scenarios Encryption and packaging

Camera

Live/Pause

dImage

3D-View

X3D

optional

DebugView

WEB-View

HTML5+JS

Page 19: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Next steps…Identification and unification of common technology data Annotation Data Reference Data

Create open framework for declarative approaches Open Annotation Engine Open Context Engine Unify Vision and Sensor based approaches KML in the context of industrial maintenance applications Evaluate using show-cases

Extend native interfaces Incooperate Sound A2ML Better content management (Offline Usage, Encryption)

Make it free for masses for developing

Page 20: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Happy Workshopping

THANK YOU !

Page 21: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Building Block Hierarchy – What data to processContext EngineBased on available sensors in device

ContextEngine

LocalizationEngine

VisionEngine

AuditiveEngine

Other sensors..

GeolocationEngine

Page 22: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Building Block Hierarchy - DerivativesContext EngineBased on capabilities of device

AnnotationEngine

2D-AnnotationEngine

VideoEngine

AuditiveEngine

3D-AnnotationEngine

GeolocationEngine

Sound-Annotation

Engine

Page 23: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Parties involved in the game of AR

Physical Player The user of the app Network Provider Producer of Cellphones Application Developer Content

developer/provider Technology developer AR Standardization

…some of them want to

Learn something Network Save costs Save resources Save time Earn money Have fun Sustainability

Page 24: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Identification of needs...

App Devel

ContentProvide

r

ContentCreator

User(s)

CommunicationProvide

r

DeviceCreator

Techno-logy

Devel

The user of the app Network Provider Producer of Cellphones Application Developer Content developer/provider Technology developer AR Standardization

Page 25: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Applications (Fallback Model in Action)

IE8 Plugin (OpenGL and Raytracing) and native iOS app with x3dom layer Runtime independent – supports (native implementation,) plugin, and WebGL

Page 26: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Geo Located - Augmented RealityAbstract Building Blocks

LocalizationEngine

Content

AnnotationEngine

Position

InteractionEngine

ApplicationModel

Reference Model Annotation Model

Localization Sensors

Output

InteractionDevice

Page 27: Special considerations for Standards in Mobile AR

© Fraunhofer IGD Timo Engelke, A4, October 11th,2010, International Workshops on AR Standards, Seoul, Korea

Geo located and Vision Based Augmented RealityTechnically Abstract Building BlocksCamera

LocalizationEngine

Content

2D AnnotationEngine

Extrinsics

Interaction Engine

Display ImageScreen

ApplicationModel

Reference Model Annotation Model

Localization Sensors Intrinsics

Source: Layar


Recommended