Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

1

Technological Trends in Wearable Computing and Augmented Reality

Steven FeinerComputer Graphics & User Interfaces LabDepartment of Computer ScienceColumbia University New York, NY 10027

Startup Village, Skolkovo Innovation City, Moscow, June 1–3, 2014

2

Wearable Computing

Clock watchhttp://en.wikipedia.org/wiki/File:Henlein_Taschenuhr.jpg

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

3

Wearable Computing

Wristwatch www.watchtime.com/wristwatch-industry-news/watches/iwc-portuguese-grande-complication/

4

Wearable Computing

Halden Calculex Slide Rulehttps://sites.google.com/site/bobscalculatorsandsliderules/home/slide-rules/halden

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

5

Wearable Computing

Cavalry Sketch Board www.gemmary.com

6

Wearable Computing

Plus Four Wristlet Route Indicator© Phillip Tosscano

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

7

Wearable Computing

Pocket Diarywww.walesonline.co.uk/news/wales-news/captain-scotts-diary-prepared-ill-fated-1812944

8

Edward Thorp & Claude Shannon

Wearable Computing

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

9

Wearable Computing

Steve Mann

10

Wearable Computing

MIT Cyborgs

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

11

AR Restaurant Guide, 2001

Where: Back

Columbia Touring Machine monet.cs.columbia.edu/projects/mars

12

Where: Wrist

Pebble Smart Watch

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

13

Where: Eye

Google Glass www.independent.co.uk/incoming/article8526907.ece/BINARY/original/IsabelleOlsson_GoogleGlass.jpg

14

Where: Clothes

B. Stern, L. Buechleyhttp://web.media.mit.edu/~leah/LilyPad/share.html

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

15

Trends

16

Augmented Reality

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

17

AR vs. VR google.com/trends

18

AR vs. VR google.com/trends

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

19

Virtual Reality Back for Real

Oculus Rift

20

Eyewear

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

21

Google Glass prototype, 2012

Eyewear

MicroOptical Corp. clip-on, late 20th C. IBM Visionpad, late 20th C.

22

Lumus

Konica Minolta

Sony

Warmer Still…

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

23

2014: See-Through Head-Worn Displays++

Integrate ≥ 1 ofOrientation trackerInput deviceRGB cameraDepth cameraComputerRadio

GlassUp

Meta

Google

Epson

Laster Optinvent

Vuzix

24

Courtesy of a time traveler The Time Machine

2XXX eyewear in 1999

AppearanceComfortFOVPer-pixel opacityVariable depth of fieldIntegrated sensors,…Production plans

Computer? Networking? Content?The milieu is missing!

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

25

Personal Area Network

Devices wirelessly connected to a smartphone hub

E.g., Bluetooth LE

26

Recolonizing the Wrist

SmartwatchFitness tracker

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

27

Natural Feature Tracking

Qualcomm Vuforia

28

Body Input

Other input: Myo

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

29

Cameras: RGB and Depth

Softkinetic DS325

30

Cameras: RGB and Depth

Leap Motion Controller www.eyalo.com/wp-content/uploads/2013/07/leaprev7.jpg

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

31

Ideas Refined by 3D Printing

MakerBot Thing-O-Maticwww.domusweb.it/en/news/2012/02/28/print-3d-an-exhibition.html

32

Application Domains

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

33

Helmets and Masks

Helmets http://www.fusar-tech.com/http://www.skullysystems.com/

34

ARMAR: Workpiece TaskAssistance

S. Henderson & S. Feiner, ISMAR 2011

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

35

ARMAR: Workpiece Task Assistance Study

Is AR better than conventional (computer)documentation?

Assemble aircraft engine combustion chamberSelect bottom and top, align correctly, pin

+ =

36

ARMAR: Workpiece Task Assistance Study: LCD Condition

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

37

ARMAR: Workpiece Task Assistance Study: AR Condition

38

ARMAR: Workpiece Task Assistance Study: AR Condition

OpticalSee-Through HWD

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

39

ARMAR: Workpiece Task Assistance (Optical See-Through)

40

O. Oda, M. Sukan, S. Feiner, and B. Tversky, 3DUI 2013

Remote Localization and Referencing in Collaborative AR

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

41

Collaborative MainRemote Localization and Referencing in Collaborative AR

O. Oda, M. Sukan, S. Feiner, and B. Tversky, 3DUI 2013

Remote SME View

Local Technician View

42

Remote Localization and Referencing in Collaborative AR

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

43

Multiple Displays

44

Environment Management

AR as computational æther embedding other tracked displays and the environment

How do we control/manage/coordinate what information goes where?

Head space?Body space?Device space?World space?

A. Butz et al., IWAR 1999

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

45

Multiple Displays: Hybrid UIs

Combining a shared tabletop display with personal head-tracked see-through displays and phones

N. Dedual, C. Elvezio, and S. Feiner

46

Multiple Displays: Hybrid UIs

Data about POIs from Yelp, Foursquare, Instagram, CityMaps, NYC 311

Displayed in summary grids of pictograms

N. Dedual, C. Elvezio, and S. Feiner

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

47

Multiple Displays: Hybrid UIs

N. Dedual, C. Elvezio, and S. Feiner

48

Multiple Displays: Hybrid UIs

N. Dedual, C. Elvezio, and S. Feiner

Steven Feiner, Technological Trends in Wearable Computing and Augmented RealityStartup Village 2014, Skolkovo Innovation City, Moscow, Russian Federation, June 1–3, 2014

49

Acknowledgments

Nick DedualCarmine ElvezioSteve HendersonOhan OdaMengu SukanBarbara TverskyLauren Wilcox

USMC personnel at Marine Corps Logistics Base: Mike Shellem, Curtis Williams, Andrew Mitchell, Alan ButterworthUSMC cadre and students at Aberdeen Proving GroundSupported in part by NSF, ONR, Raytheon, USMC, and gifts from, Google, Microsoft, Nokia, VTT, Vuzix, Canon

Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation