Date post: | 12-Apr-2017 |
Category: |
Data & Analytics |
Upload: | joerg-osarek |
View: | 127 times |
Download: | 2 times |
Virtual and AugmentedVirtual and Augmented
Reality AnalyticsReality Analytics
Virtual and AugmentedVirtual and Augmented
Reality AnalyticsReality Analytics
Frog #84027132 © julien tromeur – fotolia.com - extended licenseFrog #84027132 © julien tromeur – fotolia.com - extended licensetransformed into VR Frog by Joerg Osarektransformed into VR Frog by Joerg Osarek
Joerg OsarekJoerg OsarekIT-avant-gardist, Business-IT-ArchitectIT-avant-gardist, Business-IT-Architect
Skilltower InstituteSkilltower Institute
Partner, its-peoplePartner, its-people
DigilityDigilitySeptember 2016September 2016
Cologne, GermanyCologne, Germany
© Skilltower Institute, 20162
Intro: Joerg Osarek – vr.skilltower.com
ManagementManagement
IT-avant-gardeIT-avant-garde
Virtualization (VMware)Virtualization (VMware)
Oracle, LinuxOracle, Linux
Virtual Reality AnalyticsVirtual Reality Analytics
Cinematic VRCinematic VR
VR BusinessVR Business
Co-founder:Co-founder:www.its-people.dewww.its-people.de
© Skilltower Institute, 20163
Reports from the future – my passion ever since
Always some years ahead of Mainstream (Years Ahead of Mainstream = YAM)
1989: HP: Computer Viruses: 10 YAM
1998: DOAG: VRML – The future of the Internet: 9 YAM (2ndLife 2007)
2007: DOAG: How does SOA influence Business Intelligence?: 3 YAM
2010: DOAG: Cloud Computing and the Technology-Monster: 2 YAM
2011: DOAG: The Organic Computing Age: 9 YAM?
2012: DOAG: SoMoLo (Social, Mobile, Local) / Big Data: 3 YAM
2015: DOAG: Virtual Reality Analytics: 3-5 YAM?
© Skilltower Institute, 20164
Reports from the future – my passion ever since
1998: DOAG: VRML – The futureof the internet
Many thanks to Wolfgang Taschnerfor his archived copy
© Skilltower Institute, 20165
VR/AR Analytics
How VR and AR change business intelligence and how you can integrate VR/AR Analytics in your BI-systems and data warehouses
I. Introduction into VR/AR today and tomorrow
II. VR/AR Analytics: Your new business intelligence gold mine
II.a Integrate VR/AR Sources into your Big Data Systems
II.b Chances and Challenges with VR/AR Analytics
III. The 3 waves – from VR to AR to USEMIR: What future awaits us?
© Skilltower Institute, 20166
I. Introduction into VR/AR today and tomorrow
© Skilltower Institute, 20167
Virtual Reality: The power of Immersion
VR puts tough guys under a spell unlike any other medium.Reactions are ranging
from delight to pure horror
Try it yourselfor visit our VR Meetup in Frankfurt.
© Skilltower Institute, 20168
Display Technologie
2015: Samsung Gear, Oculus DK2, 2015: Samsung Gear, Oculus DK2, Mobile VR, Google Cardboard, +++Mobile VR, Google Cardboard, +++
2016: Oculus Rift Consumer Version2016: Oculus Rift Consumer VersionPlaystation VR (Morpheus)Playstation VR (Morpheus)HTC Vive, FOVE (Eye Tracking)HTC Vive, FOVE (Eye Tracking)Microsoft Hololens für EntwicklerMicrosoft Hololens für Entwickler
What is happening at VR / MR? - VR is not just headsets!
© Skilltower Institute, 20169
Display Technology
2015: Samsung Gear, Oculus DK2, 2015: Samsung Gear, Oculus DK2, Mobile VR, Google Cardboard, +++Mobile VR, Google Cardboard, +++
2016: Oculus Rift Consumer Version2016: Oculus Rift Consumer VersionPlaystation VR (Morpheus)Playstation VR (Morpheus)HTC Vive, FOVE (Eye Tracking)HTC Vive, FOVE (Eye Tracking)Microsoft Hololens für EntwicklerMicrosoft Hololens für Entwickler
FulldomesFulldomes
Leia HolodisplayLeia Holodisplay
Magic LeapMagic Leap
What is happening at VR / MR? - VR is not just headsets!
© Skilltower Institute, 201610
Display Technology Recording
2015: Samsung Gear, Oculus DK2, 2015: Samsung Gear, Oculus DK2, Mobile VR, Google Cardboard, +++Mobile VR, Google Cardboard, +++
2016: Oculus Rift Consumer Version2016: Oculus Rift Consumer VersionPlaystation VR (Morpheus)Playstation VR (Morpheus)HTC Vive, FOVE (Eye Tracking)HTC Vive, FOVE (Eye Tracking)Microsoft Hololens für EntwicklerMicrosoft Hololens für Entwickler
FulldomesFulldomes
Leia HolodisplayLeia Holodisplay
Magic LeapMagic Leap
Google Jump Google Jump + Youtube VR+ Youtube VR
JauntVR NeoJauntVR NeoLightfield CamLightfield Cam
Lytro ImmergeLytro Immerge
What is happening at VR / MR? - VR is not just headsets!
© Skilltower Institute, 201611
Display Technology Recording
Interaction 2015: Samsung Gear, Oculus DK2, 2015: Samsung Gear, Oculus DK2,
Mobile VR, Google Cardboard, +++Mobile VR, Google Cardboard, +++
2016: Oculus Rift Consumer Version2016: Oculus Rift Consumer VersionPlaystation VR (Morpheus)Playstation VR (Morpheus)HTC Vive, FOVE (Eye Tracking)HTC Vive, FOVE (Eye Tracking)Microsoft Hololens für EntwicklerMicrosoft Hololens für Entwickler
FulldomesFulldomes
Leia HolodisplayLeia Holodisplay
Magic LeapMagic Leap
Google Jump Google Jump + Youtube VR+ Youtube VR
JauntVR NeoJauntVR NeoLightfield CamLightfield Cam
LeapMotionLeapMotion
Fairy LightsFairy LightsHologrammHologrammzum Anfassenzum Anfassen
VR SculptingVR Sculpting
Google SoliGoogle SoliRadar ChipRadar Chip
Disney DISHDisney DISH
ILM X-LabsILM X-Labs
Lytro ImmergeLytro Immerge
What is happening at VR / MR? - VR is not just headsets!
© Skilltower Institute, 201612
What is happening next?
Voice Mood Analysis (e.g. cogito App)
Brain Wave interfaces (e.g. NeuroSky Neuro Headset)
Nerve Induction? (e.g. direct projection into retina or nerves)
Mark Zuckerberg: „we'll be able to send emotions to each other soon“*
* Article at theverge.com from Rich McCormick on June 30, 2015.* http://www.theverge.com/2015/6/30/8873819/mark-zuckerberg-facebook-sending-emotions-soon
© Skilltower Institute, 201613
What is happening next?
Quelle: Woman and pan: combined of two public domain images from www.pixabay.com
Mark Zuckerberg: „we'll be able to send emotions to each other soon“… could look like this:
© Skilltower Institute, 201614
What is happening next?
Quelle: Woman and pan: combined of two public domain images from www.pixabay.com , Quelle: Angry driver: Vladimir Mucibabic/123rf.com #4554701_l
Mark Zuckerberg: „we'll be able to send emotions to each other soon“… could look like this:
LIKE ?LIKE ?
© Skilltower Institute, 201615
II. VR/AR Analytics: Your new business intelligence gold mine
© Skilltower Institute, 201616
VR Analytics comes in two flavours
a) Visualization of analytics data in VR
© Skilltower Institute, 201617
VR Analytics comes in two flavours
a) Visualization of analytics data in VR
We don't cover this today.
b) Analyze VR experiences in your DWH
© Skilltower Institute, 201618
Let's Play
© Skilltower Institute, 201619
II.a Integrate VR/AR Sources into your Big Data Systems
© Skilltower Institute, 201620
Level 1: Your Mission
BUILD ME THE PERFECTVR ANALYTICS PLATFORM.
MUHAHAHA...
© Skilltower Institute, 201621
Level 2: Make or buy? - Status VR Analytics vendorsLevel 2: Make or buy? - Status VR Analytics vendors cognitiveVRcognitiveVR: : http://cognitivevr.co/http://cognitivevr.co/ Analytics platform and plugins for Unity and Unreal Game Engines Analytics platform and plugins for Unity and Unreal Game Engines
retinadVR.comretinadVR.com (Unity Plugin) → joined (Unity Plugin) → joined rothenbergventures.comrothenbergventures.com (investing in frontier tech startups)(investing in frontier tech startups)
http://ghostline.xyz/http://ghostline.xyz/ VR Analytics tool PSVR, Vive, Oculus (pc based) VR Analytics tool PSVR, Vive, Oculus (pc based)
http://www.viarbox.com/2016/05/02/virtual_reality_analytics/http://www.viarbox.com/2016/05/02/virtual_reality_analytics/ - end to end Platform - end to end Platform
http://www.finwe.mobi/main/liveheat/http://www.finwe.mobi/main/liveheat/ - 360 video heat maps - 360 video heat maps
Election.tvElection.tv Engages Kosher Virtual Reality Analytics For Exclusive 2016 US Election Tech Polling Engages Kosher Virtual Reality Analytics For Exclusive 2016 US Election Tech Polling Partnership (Partnership (Kosher.tvKosher.tv, a media ventures company), a media ventures company)
Some BI / DWH experts start to broaden their portfolio Some BI / DWH experts start to broaden their portfolio (like (like its-peopleits-people and and skilltower instituteskilltower institute))
It is quite early for the large BI / DWH software vendors but we should watchIt is quite early for the large BI / DWH software vendors but we should watch trendsetterstrendsetters like: like: IBM, SAP, Qlik, Oracle, SAS, Information Builders, IBM, SAP, Qlik, Oracle, SAS, Information Builders, MicrostrategyMicrostrategy and and challengerschallengers like: like: Microsoft, Tableau, Tibco, Infor, BoardMicrosoft, Tableau, Tibco, Infor, Board and some more.and some more.
© Skilltower Institute, 201622
2USEMIR:VR/AR Analytics
links oben:www.retinad.io
Mitte: cognitivevr.co
Demo Demo timetime
© Skilltower Institute, 201623
CognitiveVR scene explorer in ordinary web browser
© Skilltower Institute, 201624
CognitiveVR scene explorer in ordinary web browser
© Skilltower Institute, 201625
CognitiveVR scene explorer in ordinary web browser
© Skilltower Institute, 201626
CognitiveVR scene explorer in ordinary web browser
Robert Merki, CognitiveVR:Robert Merki, CognitiveVR:
We also provide users with a We also provide users with a multitude of pre-built analytics calls. multitude of pre-built analytics calls.
These include algorithms for These include algorithms for detecting comfort, detecting comfort,
controller/HMD occlsion, controller/HMD occlsion, VR performance, VR performance,
and even arm length of users.and even arm length of users.
Robert Merki from cognitiveVR.co (Canada) holding my book – Selfie Image from Robert Sept.19th.2016
© Skilltower Institute, 201627
CognitiveVR scene explorer in ordinary web browser
Robert Merki, CognitiveVR:Robert Merki, CognitiveVR:
The user who bought the game The user who bought the game could not get passed a certain level.could not get passed a certain level.
Upon analysis of the session, Upon analysis of the session, it was clear that it was clear that this particular this particular
user was too short to see the button user was too short to see the button that needed to be press[ed] that needed to be press[ed]
to open the next area!to open the next area!
Robert Merki from cognitiveVR.co (Canada) holding my book – Selfie Image from Robert Sept.19th.2016
© Skilltower Institute, 201628
Level 3: Approaches for displaying VR in headsets
Hybrid Rendering
Nozon.com Presenz
some light field tech
Live 3D in Game engine
(Unity3D, CryengineUnreal Engine, ...).
Pre-rendered
YouTube VR or 360°
VR Player apps, ...
© Skilltower Institute, 201629
measurement data, e.g.:measurement data, e.g.:Position 5,3,1Position 5,3,1Rotation 23°left, 18°upRotation 23°left, 18°upSpeed of movementSpeed of movement(eye position, parallax)(eye position, parallax)......
Mapping:Mapping:Position in VRPosition in VRObjects watchedObjects watchedInteractionsInteractions……
Merging with otherMerging with other
Data WarehouseData WarehouseContentContent
Enriched AnalysisEnriched Analysis
determinedeterminemeaningmeaning
MeasureMeasureAnalyzeAnalyze
← ← (Interactive live return path) ←(Interactive live return path) ←
Level 4: How to build a semanticLevel 4: How to build a semantic Meta Layer for VR Analytics? Meta Layer for VR Analytics?
© Skilltower Institute, 201630
depends on the particular VR/MR experience. depends on the particular VR/MR experience.
time matters as objects move over time!time matters as objects move over time!
meaning must be mapped manually or... meaning must be mapped manually or...
(soon) can be mapped automatically by autonomous analysis (soon) can be mapped automatically by autonomous analysis algorithms (Google/Facebook recognize objects, text and algorithms (Google/Facebook recognize objects, text and speech in images, videos, audio) e.g. Google Translate speech in images, videos, audio) e.g. Google Translate Augmented Reality or Youtube TranscriptionAugmented Reality or Youtube Transcription
Level 4: How to build a semanticLevel 4: How to build a semantic Meta Layer for VR Analytics? Meta Layer for VR Analytics?
© Skilltower Institute, 201631
2USEMIR: Neural Talk 2 – free image analysis with artificial intelligence (AI)
https://github.com/karpathy/neuraltalk2https://github.com/karpathy/neuraltalk2http://cs.stanford.edu/people/karpathy/neuraltalk2/demo.htmlhttp://cs.stanford.edu/people/karpathy/neuraltalk2/demo.html
© Skilltower Institute, 201632
Level 5: data interchange between VR/AR and our reality
SmartWatch:SmartWatch:heart rate, ...heart rate, ...
voice- and voice- and mood analysismood analysis
facial expression facial expression analysisanalysis
NLP* NLP* eye movementseye movements
*NLP = Neuro Linguistic Programming:*NLP = Neuro Linguistic Programming:an approach to communication, personal development, and psychotherapyan approach to communication, personal development, and psychotherapy
created by Richard Bandler and John Grinder in California, United States in the 1970s.created by Richard Bandler and John Grinder in California, United States in the 1970s.
© Skilltower Institute, 201633
II.b Chances and Challenges with VR/AR Analytics
© Skilltower Institute, 201634
Challenges of the emerging VR market: Mistakes to avoid with VR/AR Analytics
The even Bigger Data challenge
technical fragmentation challenge
The challenge of Standards and content vs. technology improvements
The challenge of Data privacy and Security
The challenge of patent wars
© Skilltower Institute, 201635
The even Bigger Data challenge
BigBigDataData
© Skilltower Institute, 201636
The even Bigger Data challenge
BigBigDataData
GiganDataGiganData
VR/ARVR/ARturnturn
intointo
© Skilltower Institute, 201637
solution approaches: interpolate Data reduction – example from Blender – simplify F Curves addon.
For a VR Analytics system this has to be done by an automated algorithm.
© Skilltower Institute, 201638
Herausforderungen im VR Markt und Lösungsansätze
Challenges:
Big Data → GiganData
Technical Fragmentation
Standards vs. Innovation
Data Privacy and Security
Patent Wars
Approaches for solutions
interpolate, translate to meaning
Include intermediate results layer and abstraction tiers
Include intermediate results layer
open dialogue / co creation,security concept for VR/AR similar to BYOD
...
© Skilltower Institute, 201639
Level 5: facial expression analysis
Science Fiction? Only for Headsets?
predicted in 2012 at DOAG for 2017, predicted in 2012 at DOAG for 2017, already reality in 2015already reality in 2015
© Skilltower Institute, 201640
Level 5: facial expression analysis
Emotion Analyticswith PC or smartphone www.affectiva.comwww.affectiva.com
The video, that watches you.The video, that watches you.analysis of facial expressions and your moodanalysis of facial expressions and your moodwith a simple webcam → Emotion Database with APIwith a simple webcam → Emotion Database with API
© Skilltower Institute, 201641
Level 5: facial expression analysis
Yesterday:Yesterday:What did the user do while interacting?What did the user do while interacting?
Today:Today:What is the user doing while interacting?What is the user doing while interacting?
Tomorrow:Tomorrow:What is the user going to do What is the user going to do while interacting?while interacting?(interactive and responsive, blending worlds)(interactive and responsive, blending worlds)
© Skilltower Institute, 201642
Mission accomplishedMission accomplished
measurement data → semantic layer
combination withadditional data(heart rate, facial expressions,eye movement)
combination withclassic data warehousedata streams
enables (NLP)conclusionsclose to thought reading andprediction of behaviour
YES. YES.MUHAHAHA...
© Skilltower Institute, 201643
III. The 3 waves – from VR to AR to USEMIR: What future awaits us?*
*which has a huge impact onour business intelligence systems
and our culture
Level up!Level up!which will lead us to:which will lead us to:
USEMIRUSEMIR UUbiquitious biquitious SESEnsory nsory MIMIxed xed RRealityeality
*Quantum*Quantum TechTech
GeneticsGenetics RoboticsRobotics ITIT Nano-TechNano-Tech Space-TechSpace-Tech
accellerated by Q-GRINS*accellerated by Q-GRINS*
© Skilltower Institute, 201645
2USEMIR: Next generation ofmobile 3D devices
Google builds VR Unit+ Google Cardboard+ Google Jump / Daydream+ YouTube VR / Spotlight Stories+ ca.1.4 Billion Investment in MagicLeap 2014, 2015 Google + others+ self driving cars
Apple seems to build VR Unit+ acquired primesens+ acquired faceshift + hired Doug Bowman+ plans for self driving cars
Bildquelle: Project Tango, Google.com
First Google Tango Phone: First Google Tango Phone: Lenovo PHAB2 Pro announced Lenovo PHAB2 Pro announced for September 2016 for 499,-€for September 2016 for 499,-€
© Skilltower Institute, 201646
2USEMIR: MSQRD (simple smartphone camera – real time)
erhältlich im iStore und Playstore: https://play.google.com/store/apps/details?id=me.msqrd.android&hl=de
© Skilltower Institute, 201647
2USEMIR: Real-Eyes lightfield display without headset
www.real-eyes.eu – own photos by Joerg Osarek.
© Skilltower Institute, 201648
11 Stereo-photoswith photogrammetryturned back into3D model.
2USEMIR: Real-Eyes lightfield display without headset
www.real-eyes.eu – own photos by Joerg Osarek.
© Skilltower Institute, 201649
USEMIRUSEMIR tomorrow – or in the far future?tomorrow – or in the far future?
We will be surrounded byU.SE.MIRU.SE.MIR
UUbiquitious biquitious SESEnsory nsory MIMIxed xed RRealityeality
==bidirectional intelligent bidirectional intelligent
interactioninteraction
only linear progress only linear progress necessary!necessary!
Images: presskit from Microsoft.com hololens, affectiva.com , presskit fromhttp://digitalnature.slis.tsukuba.ac.jp/2015/06/fairy-lights-in-femtoseconds/
© Skilltower Institute, 201650
I have an idea.
Let's provide themwith technology
to read our thoughts.
Quelle: Wikimedia.org – Erstellt 1590 – Public Domain
Who ought to know what about us? - a comparable encounterWho ought to know what about us? - a comparable encounter
© Skilltower Institute, 201651
What choices do we have?
back to the caves?back to the caves?
prepare to be terminated?prepare to be terminated?
© Skilltower Institute, 201652
What choices do we have?
Wrong question!Wrong question!Why do we fear a Terminator-Scenario?Why do we fear a Terminator-Scenario?
Why don't we consider ourselves asWhy don't we consider ourselves asself-confident honourable partners?self-confident honourable partners?
Is there a third choice?Is there a third choice?
© Skilltower Institute, 201653
The third choice:We can qualify for the 22nd century by cultivating chivalry and wisdom.
Q22=C+W
humbleness, honour, kindness, confidence,
politeness, courage and bravery, clemency, moderate behaviour ,
continuity, loyalty, dignity, discipline and decency
© Skilltower Institute, 201654
The third choice:We can qualify for the 22nd century by cultivating chivalry and wisdom.
Q22=C+W
self confidence by acting
honourably
working togetheras civilizationto boost our development
EARNING BONUS POINTS:EARNING BONUS POINTS:
appearing as self confident
and honourable examples and
partners
© Skilltower Institute, 201655
LEVEL UP!LEVEL UP! INSERT COINSINSERT COINSTO CONTINUETO CONTINUE
The rest is up to us now.
We shape our future together: with our systems and with our actions.
Remember:"with great powercomes greatresponsibility" (Stan Lee)
© Skilltower Institute, 201656
additional informationon VR/AR Analytics
read additional read additional information information in the book:in the book:
Robert Merki from cognitiveVR.co (Canada) holding my book – Selfie Image from Robert Sept.19th.2016
© Skilltower Institute, 201657
additional informationon VR/AR Analytics
3 video 3 video TrainingsTrainingsavailable soon atavailable soon attraining.vr.skilltower.comtraining.vr.skilltower.com
Robert Merki from cognitiveVR.co (Canada) holding my book – Selfie Image from Robert Sept.19th.2016
© Skilltower Institute, 201658
Contact and further information on Business VR / AR
Skilltower Institute
Joerg OsarekFounder and CEO
Triftstr. 30D-61350 Bad HomburgGermany
mailto:[email protected].: +49(0)151-23 0 24 333
vr.skilltower.com