The Reality of Augmented Reality

Mark Billinghurst

mark.billinghurst@hitlabnz.org

The HIT Lab NZ, University of Canterbury

December 2011

Falling in Love

1989…

Virtual Reality Was COOL!

Joining the HIT Lab in Seattle

Only $250K for 1500 polygons/sec!

My VR Predictions I knew everyone would use VR when:

HMDs were cheap (<$300) Computers generate millions of

polys/sec Tracking was inexpensive Good 3D input devices

Cheap HMDs

Gartner Hype Cycle

1990-95

1995-2000

April 2007 Computer World VR Voted 7th on list of 21 biggest technology

flops - MS Bob Winner

Back to Reality 1999 Fred Brooks – “What’s Real about

Virtual Reality” In 1994 VR barely works In 1999 VR is now really real

3 stages of application maturity: Demonstration Pilot Production

There are successful VR application domains

VR Business Today

> $3-5 Billion VR business (+ > $150 Billion Graphics Industry) Visualization, simulation, gaming, CAD/CAE, multimedia, graphics arts

> $3-5 Billion

Lessons learned Don’t believe the hype Many factors determine technology

acceptance- Design for users

Need to move from Demo to Production- Profitable niche markets

Follow the money

What’s Real About Augmented Reality?

Key Questions Where is AR technology today? What are the key opportunities? What are research obstacles? Where will the technology be in 5-10

years?

AR History1960’s – 80’s: Early

Experiments1980’s – 90’s: Basic

Research Tracking, displays

1995 – 2005: Tools/Applications Interaction, usability, theory

2005 - : Commercial Applications Games, Medical, Industry

2007 - AR Reaches Mainstream

MIT Technology Review March 2007 list of the 10 most

exciting technologies Economist

Dec 6th 2007 Reality, only better

Google Searches for AR

2009 Key crossover point – More people interested in AR than VR

Gartner’s top 10 disruptive technologies 2008-2012: Multicore and hybrid processors Virtualisation and fabric computing Social networks and social software Cloud computing and cloud/Web platforms Web mashups User Interface Ubiquitous computing Contextual computing Augmented reality Semantics

AR Technology Today

1977 – Star Wars

Key Features Classic Augmented Reality (Azuma 97)

Combines Real and Virtual Images- Display technology

Interactive in Real-Time- Real time graphics

Content Registered in 3D- Viewpoint tracking

Other Features Shared 3D viewing

- Individual views

HMD, HMD, HMD...

Tracking, Tracking, TrackingContoursFeature Points

Surfaces

1999 - HIT Lab US Shared Space

AR Conferencing

Moves conferencing from the desktop to the workspace

Virtual Viewpoint Generation

3D Live System

2008 - CNN

The Red Planet (2000)

Outdoor AR Flexible Lens Surface

Bimanual interaction Digital paper analogy

AR FlexiLens

Real handles/controllers with flexible AR lens

2008: Location Aware Phones

Nokia NavigatorMotorola Droid

HIT Lab NZ Outdoor AR Platform

Cross platform Android, iPhone

3D onsite visualization Intuitive user interface

Positions content in space Camera, GPS, compass

Client/Server software architecture Targeting museum guide/outdoor site

applications

Earthquake Reconstruction

See past, present and future building designs Earthquake survivor stories shown on map view Collect user comments Android platform

IronMan2

Natural Hand Interaction (2011)

Using bare hands to interact with AR content MS Kinect depth sensing Real time hand tracking Physics based simulation model

AR Today Key Technologies Available

- Robust tracking (Computer Vision, GPS/sensors)

- Display (Handheld HMDs)- Input Devices (Kinect, etc)- Developer tools (Qualcomm, Metaio, ARTW)

Commercial Business Growing- Gaming, GPS/Mobile, Online Advertisement

• >$5 Billion USD by 2016 (MarketsandMarkets)• >$1.5 Billion USD in Mobile AR by 2014 (Juniper Research)

AR Business Today Marketing

Web-based, mobile Mobile AR

Geo-located information and service Driving demand for high end phones

Gaming Mobile, Physical input (Kinect, PS Move)

Upcoming areas Manufacturing, Medical, Military

Research Directions

Important Research Directions

Fundamental Technologies Unobtrusive displays Ubiquitous tracking

Improving the user experience Interaction, Information management Social networking + AR

Standards (Content, interaction) Integrating with related technologies

AR User Experience

Future Displays

Always on, unobtrusive

Contact Lens Display Babak Parviz

University Washington MEMS components

Transparent elements Micro-sensors

Challenges Miniaturization Assembly Eye-safe

Contact Lens Prototype

Information Presentation

Public and private annotations Aid recognition, “extended memory”

Wikitude – www.mobilizy.com

Blah

Blah

BlahBlah

Blah

BlahBlah

Blah

BlahBlah

Blah

BlahBlah

Blah

BlahBlah

Blah

BlahBlah

Blah

BlahBlah

Information Filtering

Information Filtering Information Filtering (Julier et al. ’00)

• Remove clutter by goal- and distance based filtering • User’s task is route finding: Sniper and relevant buildings are displayed; objects, which are determined to be unnecessary, removed

Metaverse Neal Stephenson’s “SnowCrash” The Metaverse is the convergence of:

1) virtually enhanced physical reality 2) physically persistent virtual space

Metaverse Roadmap http://metaverseroadmap.org/

Metaverse Dimensions• Augmentation technologies that layer

information onto our perception of the physical environment.

• Simulation refers to technologies that model reality

• Intimate technologies are focused inwardly, on the identity and actions of the individual or object;

• External technologies are focused outwardly, towards the world at large;

Mirror Worlds Mirror worlds are informationally-

enhanced virtual models or “reflections” of the physical world. Google Earth, MS Street View, Google Maps

LifeLogging Technologies record and report the

intimate states and life histories of objects and users Nokia LifeBlog, Nike+

Ubiquitous AR (GIST, Korea)

How does your AR device work with other devices?

How is content delivered?

ubiHome @ GIST

ubiHome

What/When/How

Where/When

Media services

Who/What/When/How

ubiKey

Couch SensorPDA

Tag-it

Door Sensor

ubiTrack

When/HowWhen/HowWho/What/When/How

Light service MR window

CAMAR GIST(Context-Aware Mobile Augmented

Reality)

Conclusions

Conclusions AR is becoming a real industry Key areas of everyday use include

Location Based, Gaming Web-based, Mobile AR

Important research for the future Tracking, Interaction, Displays, User

experience

More Information

• Mark Billinghurst– mark.billinghurst@hitlabnz.org

• Website– http://www.hitlabnz.org/