Virtual Sports for Real!
Elmar Eisemann1 and Stephan Lukosch2
1Computer Graphics and Visualization, Faculty of Electrical Engineering, Mathematics and Computer Science 2Systems Engineering Section, Faculty of Technology, Policy and Management
Outline
• Virtual Sports
• Creating virtual worlds for sports
• Virtual reality equipment
• Motion data analysis and visualization
• Outlook
VIRTUAL SPORTS
Virtual Sports https://www.youtube.com/watch?v=FdyeWExPZYg
Virtual Sports in the Future
• Physically train and exercise
• Create controlled environments
• Perform precise measurements
• Provide personalized and adaptive real-time coaching
• Foster engagement and motivation for sports
http://cdn.theatlantic.com/assets/media/img/sponsored/2014/09/article_4/resp-feature.jpg?1427821271
http://cdn.theatlantic.com/assets/media/img/sponsored/2014/09/article_2/resp-feature.jpg?1427821241
Virtual Sports in the Future
• Run with virtual trainer
• Run with other runners on the same track sharing performance details
• Run against ghosts, i.e. runners that have run here before
• Run with runners somewhere else on the world
• Run on a treadmill for precise gait evaluation and training
http://numrush.com/2013/11/07/glassfit-puts-fun-excitement-back-running-using-augmented-reality/
http://www.teach42.com/2014/01/15/20-things-you-cant-do-with-glass-but-you-really-can/
https://www.youtube.com/watch?v=NKhh3Q2zYDA
Superhuman sports
CREATING VIRTUAL WORLDS FOR TRAINING
Overview
• Several Different Creation Scenarios
• Real-world Environments
• Reproduction of Particular Scenarios
• Free-Form Design of Virtual Environments
automatic
user interaction
Creating Virtual Worlds with Real-world Data
• AHN 2 Data Set – 8 TB [Kehl, Tutenel, de Hahn, Eisemann – Geos. World F.2013]
Geometry Recognition
here 128 M points in 2 min
[Sunkel, Jansen, Wand, Eisemann, Seidel - EG 2012][Silva, Eisemann, Bidarra, Coelho - PCG 2015]
[Wu, Wand, Hildebrandt, Jansen, Seidel - 3DV 2014]
Tools for Creating Virtual Environments[Smelik, Tutenel, Kraker, Bidarra – C&G 2010]
Making Environments Look Realistic[Klehm, Seidel, Eisemann – TVCG 2016]
[Holländer, Ritschel, Eisemann, Boubekeur - EGSR 2011][Ritschel, Eisemann, Ha, Kim, Seidel - CGF 2011]
VIRTUAL REALITY EQUIPMENT
Virtual Reality Display Options
Samsung Gear VR Oculus Rift
HTC Vive
A Large Field-of-View Solution for Eye-tracking HMDs
• Main difficulties: • Maintain Large Field of View• Solve Calibration due to Fabrication Imperfections
[Stengel, Grogorick, Eisemannx2, Magnor – ACM MM 2015]
A Large Field-of-View Solution for Eye-tracking HMDs
• Gaze evaluation useful for…• Gaze Evaluation• Gaze-contingent Rendering• Interaction with Avatars• ...
[Stengel, Grogorick, Eisemannx2, Magnor – ACM MM 2015]
Augmented Reality
• Fusion of Virtual and Real World
[Baboud, Cadik, Eisemann, Seidel – CVPR’11]
DATA ANALYSIS AND VISUALIZATION
Motivation
• Need for assessment methods that allow for • objective, quantitative and valid measures • identifying the factors contributing to motor dysfunction
Motion Tracking
Microsoft Kinect
Intel RealSense
Perception Neuron
HTC Vive controller
AR technology Serious game
Assessment tool
Approach Cidota, Lukosch, Bank (2015)
Study on game engagement and usability
• Study with 8 healthy people older than 50 years
• Epson Moverio BT-200 OST-HMD with an Intel® RealSense F200 RGB-D camera
• Participants sorted a set of ten packages each
• Session recorded and debriefed
• Game Experience Questionnaire (GEQ)
• System Usability Survey (SUS)
Dezentje et al. (2015)
Results
• Liked the game
• Needs practice
• Difficult depth perception
• Difficult head-hand coordination
• Data from the sensors • positions and orientations of the joints
approx. 15MB per minute• RGB and RGB-D video stream
each approx. 4MB per minute • per patient and session more than 1GB
• Offline data analysis necessary for validation
Cidota et al. (2015)
Inertial Measurement Unit
• Record movement of the person
www.motionnode.com
Analyze such Data?
• Sensor output in a scatterplot• Often not very readable
• Dimensionality Reduction
Dimensionality Reduction - Example
Costly offline process
[Van der Maaten and Hinton 2008]
Interactive t-SNE Clustering[Pezzotti, Höllt, Lelieveldt, Eisemann, Vilanova - EuroVis2016]
TU Delft Sports Engineering Institute
Various Sensors and Data Sources
• Health Apps:
• FitBit
• Garmin
• SHealth
• iOS 9 – Health
• MapMyFitness
• …
Analysis and Visualization Tools for Complex or High-Dimensional Data
[Raidou, Eisemann , Breeuwer, Eisemann, Vilanova - Vis2015]
[de Hoon, van Pelt, Jalba, Vilanova – CGF15] [Kroes, Eisemannx2 – GI15]
OUTLOOK
Outlook
• New Sports Challenges
• Adaptive Environments
• New Sensors and On-the-fly Data Analysis
• Advanced Virtual Reality Equipment
• Engagement and Gamification
• Data Analysis and Visualization Solutions
Thank you very muchfor your attention!
Elmar EisemannComputer Graphics & Visualizationhttp://graphics.tudelft.nl/[email protected]
Stephan LukoschSystems Engineeringhttp://www.tudelft.nl/[email protected]
