VR-OOS System Architecture Workshop zu interaktiven VR-Technologien für On-Orbit Servicing
Robin Wolff DLR, Simulation and Software Technology
www.DLR.de • Chart 1 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Overview
- Motivation - Physical Mock-up - Servicing Tasks
- System Overview - Components - Requirements
- System Architecture - Distributed Simulation Framework - Simulation Loop - End-to-End Latency
- Simulation Module Implementations - Physics - Visualization
- Summary
www.DLR.de • Chart 2 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Motivation Physical Evaluation Mock-up
- Tele-Presence System demonstrated at ILA 2010 in Berlin
www.DLR.de • Chart 3 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Motivation Physical Evaluation Mock-up Virtual Evaluation Mock-up
- Tele-Presence System demonstrated at ILA 2010 in Berlin
www.DLR.de • Chart 4 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Motivation Servicing Tasks
- Remove Multi-Layer Insulation (MLI)
- Loosen / Tighten Screws
- Remove / Insert Modules (e.g. using a Bayonet Handle)
- Operate Switches
- Take Measurements (e.g. using a Voltmeter)
www.DLR.de • Chart 5 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
EVA Task Sheet
Virtual Satellite Mock-up Physical Satellite Mock-up
System Overview System Components
www.DLR.de • Chart 6 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Collision Detection & Force Computation
Physics Simulation
Visualization
VR Display
Haptic Device Virtual Service Robot Model
Virtual Target Satellite Model
Image
xuser xhd
xobject
xuser
xobject
xtracking
Fhd
Object Properties (Geometry, Material, Constraints, etc.)
System Overview Requirements
www.DLR.de • Chart 7 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Central Control / Manager
Collision Detection, Force Computation Rate: >1kHz
Physics Simulation Rate: ~60-100Hz
Network
…
Visualization Rate: ~30-120Hz
- Interactive Real-Time Simulation
- Fast Response Times necessary
- Complexity & Accuracy Trade-off
- Dedicated Machines used for Computation
System Architecture Distributed Simulation Framework
- Distributes Simulation across dedicated Machines
- Each Simulation runs at its own Rate and only sees its own “World”
- Simulations are synchronized via Network Messages
www.DLR.de • Chart 8 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
… Server Client Client Client
Physics
Scene Simulation
Communication
Visualization
Scene Simulation
Communication
Haptics
Scene Simulation
Communication
Manager
Scene Simulation
Communication
Network
- Wraps Common Functionality
- Unified Scene Management
- Asynchronous Message Passing
- Transparent Processing Loop
System Architecture Common Simulation Loop
www.DLR.de • Chart 9 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Process Updates
Interpret Updates
Collect Changes
Common
Extra
Scene Objects & State
Read & Modify
Recv. Send
Processing Loop
Distribute Changes Step Simulation
Scene Description (COLLADA)
Load Scene
System Architecture Common Simulation Loop
www.DLR.de • Chart 10 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Process Updates
Interpret Updates
Collect Changes
Common
Extra
Scene Objects & State
Read & Modify
Recv. Send
Processing Loop
Distribute Changes Step Simulation
Scene Description (COLLADA)
Load Scene
System Architecture Distributed Simulation Framework
- Includes Optimizations for Network Throughput - Communicates via UDP/TCP/IPC Channels - Unlimited Number of Communication Channels - Configurable Queuing Schemes (e.g. FIFO, Most-Recent) - Optional Spatial and Temporal Threshold Filters
- What about Network Issues?
- Latency, Jitter, Loss, Bandwidth
www.DLR.de • Chart 11 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
… Server Client Client Client
Physics
Scene Simulation
Communication
Visualization
Scene Simulation
Communication
Haptics
Scene Simulation
Communication
Manager
Scene Simulation
Communication
Network
System Architecture End-to-End Latency
www.DLR.de • Chart 12 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
(RTT between all hosts = 0.2ms)
Collision Detection & Force Computation
Physics Simulation
Visualization
VR Display
Haptic Device Virtual Service Robot Model
Virtual Target Satellite Model
Image
xuser xhd
xobject
xuser
xobject
xtracking
Fhd
Object Properties (Geometry, Material, Constraints, etc.)
host.59, host.62
host.62 host.61
System Architecture End-to-End Latency
www.DLR.de • Chart 13 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
A. Time between Hand Movement and feeling the Surface of static, not moving Objects
(RTT between all hosts = 0.2ms)
LatencyA = 0.80 ms
Collision Detection & Force Computation
Physics Simulation
Visualization
VR Display
Haptic Device Virtual Service Robot Model
Virtual Target Satellite Model
Image
xuser xhd
xobject
xuser
xobject
xtracking
Fhd
Object Properties (Geometry, Material, Constraints, etc.)
host.59, host.62
host.62 host.61
(thap= 0.18ms)
System Architecture End-to-End Latency
www.DLR.de • Chart 14 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
B. Time between Hand Movement and seeing the Hand move
(RTT between all hosts = 0.2ms)
LatencyB = 6.09 ms
Collision Detection & Force Computation
Physics Simulation
Visualization
VR Display
Haptic Device Virtual Service Robot Model
Virtual Target Satellite Model
Image
xuser xhd
xobject
xuser
xobject
xtracking
Fhd
Object Properties (Geometry, Material, Constraints, etc.)
xhd host.59, host.62
host.62
(thap= 0.18ms)
host.61
(tvis= 3.30ms)
System Architecture End-to-End Latency
www.DLR.de • Chart 15 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
C. Time until User sees pushed Object moving
(RTT between all hosts = 0.2ms)
LatencyC = 7.15 ms
Collision Detection & Force Computation
Physics Simulation
Visualization
VR Display
Haptic Device Virtual Service Robot Model
Virtual Target Satellite Model
Image
xuser xhd
xobject
xuser
xobject
xtracking
Fhd
Object Properties (Geometry, Material, Constraints, etc.)
host.59, host.62
host.62 host.61
(thap= 0.18ms)
(tvis= 3.30ms)
(tphy= 0.02ms)
System Architecture End-to-End Latency
www.DLR.de • Chart 16 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
D. Time between Hand Movement and feeling the Surface of moving Object
(RTT between all hosts = 0.2ms)
LatencyD = 2.82 ms
Collision Detection & Force Computation
Physics Simulation
Visualization
VR Display
Haptic Device Virtual Service Robot Model
Virtual Target Satellite Model
Image
xuser xhd
xobject
xuser
xobject
xtracking
Fhd
Object Properties (Geometry, Material, Constraints, etc.)
host.59, host.62
host.62 host.61
(thap= 0.18ms)
(tvis= 3.30ms)
(tphy= 0.02ms)
Implemented Simulation Modules Haptics, Extensions
- Haptics - Collision Detection and
Force Computation A. Improved Voxmap-Point-Shell (VPS) B. OpenHaptics Toolkit
- Interfaces - Bi-manual HMI, Phantom Omni, Falcon
- Extensions - Finger Tracking - Speech Recognition - Logging - More coming …
www.DLR.de • Chart 17 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
VPS Algorithm
DLR Bi-manual HMI Phantom Omni Novint Falcon
ART Finger Tracking
Implemented Simulation Modules Physics
- Uses Bullet Physics Engine - Real-Time n-Body Simulation - Optimized for Speed,
rather than Accuracy - Support for Rigid Bodies - Simplified Collision Detection - Universal Constraints - Support for Soft Bodies
www.DLR.de • Chart 18 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
- Alternatives:
Implemented Simulation Modules Visualization
- InstantReality - Developed at
Fraunhofer IGD - Widely used at DLR RM
- ViSTA VR-Toolkit - Developed at RWTH
Aachen (+10 Years) - Now in Cooperation
between RWTH & DLR - Open Source - Based on OpenSG
(soon OpenSceneGraph) - Support for Scientific
Visualization Methods
www.DLR.de • Chart 19 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Visualization using ViSTA VR-Toolkit
Implemented Simulation Modules Visualization
www.DLR.de • Chart 20 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
(Columbus EVA, ESA)
Implemented Simulation Modules Visualization
- OpenGL Shader Support
- Simple Shadow Mapping
- Realistic Star Background - Positions based on
Hipparchos and Tycho-2 Star Catalogues
- Realistic Earth - High-Resolution Textures - Night Lights - Clouds with Shadows - Atmospheric Scattering
www.DLR.de • Chart 21 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
OpenGL Shader (top), Realistic Stars (bottom)
Implemented Simulation Modules Visualization
www.DLR.de • Chart 22 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Rendering of Earth with High-Res. Textures, Night Lights, Atmospheric Scattering, Clouds with Shadows.
Implemented Simulation Modules Visualization
- NVIDIA OptiX (work in progress) - Toolkit for Real-Time
Ray-Tracing - Allows accurate Simulation
of Light Conditions
www.DLR.de • Chart 23 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Rendering Ray-Tracing Photograph
- Creates High-Quality Shadows, Reflection, Refraction
- Could also be used to generate virtual Camera Images to evaluate Image-Recognition Algorithms
Summary
- VR-OOS: a Virtual Reality Environment for On-Orbit Servicing
- Software Framework provides a generic Architecture for a distributed interactive Real-Time Simulation Environment
- Low-Latency Response Times
- Several Simulation Modules implemented and in use
- Live Demo Today! 15:15-16:30
www.DLR.de • Chart 24 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012
Contact
Dr Robin Wolff
German Aerospace Center (DLR)
Simulation and Software Technology Software for Space Systems and Interactive Visualization
Lilienthalplatz 7 38108 Braunschweig Germany www.dlr.de/sc
www.DLR.de • Chart 25 > VR-OOS System Architecture > Robin Wolff • VR-OOS Workshop 09/10.10.2012