National Shipbuilding Research Program
Reducing Cable Installation Costs using proven Virtual Reality and Augmented
Reality Advances – Phase 2
Restricted to US Citizens.
Gain AR & VR Experience Through Our Hands-On Demonstrations
Ben MarcusElectric Boat
Phone: (860) [email protected]
ARgos Phase 2 Recap11 October 2018
Distribution Statement A – Approved for public release
National Shipbuilding Research Program
Real-Time Manipulation of a Virtual Human and Cable System within ESI’s IC.IDO Virtual Reality System
• Enable designers to test cable installation sequences at design time
• Enable installers to train and practice pulling cable in a virtual environment
• Develop new workforce skills without requiring years of practical experience
• Reduce time and cost to pull cables by assisting installers in locating correct cable paths
Distribution Statement A – Approved for public release
Program Status
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramMilestones
Phase 1 Phase 2
Final Report Implementation
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramNSRP Legacy - Technology Transfer
SKM
spARky
ARgos
Digital Shipbuilding
Improving Workforce Development Initiatives Using Augmented Reality Technology
SMART: Shipyard Mobile Applications in Real Time Project
InfoTagger
Distribution Statement A – Approved for public release
ARgos Project Overview
Electric Boat
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramReality: virtual and augmented VR: Virtual reality
Device: head mounted display Location: an engineering office Users: engineers, planners User immersed in a full-scale digital model
AR: Augmented reality Device: tablet with front-facing video camera Location: work site Users: supervisors, mechanics User presented with virtual objects and information overlaid on
the environment
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramCable Life Cycle
Functional design Detailed design Planning Installation
Cable Diagram
Cableways
Arrangement Model
Complete
Cable Routing Cable Lay
Cable Installation Sequencing
Install Cable
Virtual Reality
Augmented Reality
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramDevelopment of Cabling Diagrams
Logical representation of electrical systems
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramDevelopment of Cableways Arrangement of Electrical Components
Part of arrangement process Start and end point of cables
Development of Cableways “Roads” Space reservations for paths that cables may run through
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramDevelopment of Cable Routes Define path for each cable through the cableways (Cable Route) Auto-Route use is based on shortest path Rule-based routing (EMI, max length, etc.)
Cable Route
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramCable lay
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramProduct Structure Example
Cable Installation Sequencing Cable Installation sequencing
Planners create product structure that reflects build plan
Cables are grouped together into work instructions
INSTALL CABLE AREA BINSTALL CABLE GROUP 1
CABLE 1CABLE 2CABLE 3
INSTALL CABLE GROUP 2CABLE 5CABLE 10CABLE 43CABLE 61
Work Instruction
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramCommunicate through shared experience using off-the-shelf Virtual Reality
VR enables high velocity learning
Collaborative Immersion Interactive experience
What you use:• Consumer class HMD• Single PC• Build-In head tracking• Interaction devices• Immersive User Interface
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramARgos AR Project Goals Goal: Develop a pre-production AR system that can identify cable hanger
locations in spaces under construction Primary challenge was to map the 3D space as seen by the device to the
3D space as captured in the CAD data.
Reduce the cost of locating shipboard items by using augmented reality technology to automatically find and identify shipboard items
Distribution Statement A – Approved for public release
ARgos Path Forward
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramPath Forward - ARgos Roadmap (MS 16)
Distribution Statement A – Approved for public release
Virtual Reality Status
ESI
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramVR Vision
Challenges
Drive down construction costs in the targeted realm of cable installation - Installation of cable aboard Navy ships is a substantial cost item- Cable installation often takes place with less direction from design than desired
Mitigate business risks associated with continuing reduction of experienced work force- Knowledge of ‘Optimal’ install plans for cables, particularly large diameter cables, is from hands-on experience - Situation is intensified by the dynamics of the aging shipyard work force
- The experience gap is difficult and costly to bridge with traditional methods
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramVirtual Reality Demonstration
Overview
In order to reduce costs and improve efficiency in context of ship cabling, several aspects need to be covered in a Virtual Reality tool to make this possible. Key capabilities developed to support the goals will be showing including:
• Cable refactoring• Interface tools to assist in understanding and optimizing cable placement/work instructions
• Realism• Contextual data handling from original data source, not reduced data• Direct interaction with cables, similar to actual cable interactions for installation tasks
• Interaction methods• Commercial Off The Shelf controllers• Gloves
Distribution Statement A – Approved for public release
Augmented Reality Status
AR Applications at Bath Iron Works
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramTrial Card Resolution Use Case
Field Engineer or Navy Inspector working trial cards: 1. I want to record specific defect location and details in real time to
send to Design for resolution at the time the defect is encountered. This provides common understanding between Navy and BIW of precise nature and location of defects.
2. I want better multi-site and multi-shift collaboration where 3D scans, waypoints and notes can be created on ship and then viewed from other locations at later times. Clarity is very important to avoid multiday turn around for answers.
3. I want to have a link back to trial card work list so that I can keep track of what I’ve worked.
Use Case Examples:A. A hard to reach area where 3” of weld is missing
B. Three hangers missing out of hundreds within a compartment
C. “I went out with the Navy Inspector today and he had a hard time locating the issue he identified”
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramAR Solution Summary
• Hardware: Selected large format Lenovo smart phone with depth sensor
• Software: APS developed “InfoTagger” application to BIW specifications
• Hardening: 3D printed carrier with light and laser attachments for ease of use and toughness
• Cyber Hardening: Secure BIW mobile network
• Implementation Status: Lab and Prototype testing complete. Pilot installation underway
Distribution Statement A – Approved for public release
National Shipbuilding Research Program
Pilot User Feedback Pilot was an overall success and met the objectives we
had set for ourselves.
Using this technology makes a typically frustrating part of our job less so, while completing it quickly and providing a higher level of detailed information to the mechanics.
About 50% reduction in time to resolve issues. Most of the time was spent locating trial card issues.
We chose interns who had no previous exposure to AR and they were able to use it without issue.
We introduced this to supervisors and QA inspectors and they immediately started thinking of new ideas that this could be used for.
Distribution Statement A – Approved for public release
National Shipbuilding Research Program
InfoTagger Demo
Distribution Statement A – Approved for public release
Workforce Development Status
D’Angelo TechnologiesHepinstall Consulting Group
Distribution Statement A – Approved for public release
National Shipbuilding Research Program
AR/VR Technology Enables High Velocity Learning in Shipbuilding
ARgos Workforce Development Model
Enables a System of Dynamic Discovery
• Develop workforce to “Start out with the Complete Move in Their Head”
Enables Collaborative
Problem Solving
• Seeing Problems & “Swarming” to Solve
Enables Wide Dissemination of
Knowledge
• Share and Sustain the Knowledge Base
Enables Leading by Developing
Capabilities in Others
• AR/VR investments develop workforce capacity to innovate and improve
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramState of the Art VR/AR Technologies• Advances in technologies have allowed VR and AR to become a more widely
accessible and viable for development solutions.• VR and AR can now be used with many commercial off-the-shelf devices as well
as dedicated hardware.
ARVR
HoloLensMeta 2
Magic Leap
HTC Vive
Oculus RiftAsus ZenFone ARDistribution Statement A – Approved for public release
National Shipbuilding Research ProgramDemonstration Highlights Cable Training Simulation• Cable handling procedures• HMD preferences • Instructional feedback• Subject matter expertise feedback and
knowledge capture • Optimization
Personal Protective Equipment (PPE)• Demonstration of integration with PPE• Smart system can determine if user is
wearing PPE before training is initiated • Recommended future step for ARgos
cable training simulation demonstrations and training
Cell Phone App Training• Demonstration of future applications
using mobile phones• Simplistic version of cable training
simulation • Feedback well received as good
recruitment type toolDistribution Statement A – Approved for public release
National Shipbuilding Research Program
Lessons LearnedShipyard Knowledge Capture Process • On-site visits beneficial• Subject Matter Expertise (SME) input critical for successful development and
implementation • SME review of system and recommendations for optimization to help improve system• Need well defined process for SME knowledge capture and dissemination into VR and AR
environments Head Mounted Display (HMD) Selection Process • Review of state-of-the-art HMDs• Strong knowledge base of HMDs and user experiences • Effort will benefit from multiple forms of AR and VR equipment (HMD, tablet, mobile phone
applications)
HTC Vive
Asus Zenfone AR
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramLessons Learned(cont.)Virtual Reality (VR) Development Process • VR demonstrations well received• Ideal to integrate glove based applications and haptic sensor feedback for VR• Training ideal in VR environment for both designers and builders (and
management)Augmented Reality (AR) Development Process• Demonstrations well received • Optimization focus – user work flows and experiences, additional use cases• Need to integrate development with trainingWorkforce Development AR/VR Applications• Phase II should bring in technical integration with the AR/VR aspects• Tie in “High Velocity Learning Techniques” within the VR Training Applications• AR and VR Development to integrate training components and applications
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramTeam Training HighlightsD’Angelo Technologies, LLC ESI Group
General Dynamics – Applied Physical Sciences
• Handling cables and understanding cable properties
• Cable installation process and environmental obstacles
• Environmental awareness in design and spatial requirements.
• AR application to visualize cable installation job• Identification of path obstruction problems
before installation
• Identification of intended cable pathways
• Installation order and organization of cables in hangers
• Cable refactoring techniques and pathway conflict resolution
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramD5T Cable Demo Highlights
Focuses:• Familiarization with cable
pulling• Understanding of the cable
pulling process• Simulating ship environments
for pullers and designers.
VR Cable Simulation Training Module
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramD5T Cable Demo Highlights
• Module used the HTC Vive• Works best with room=scale VR set up• Can be used in standing only set up (model has teleportation system)• The module has a total of seven scenes users can walk through including the
tutorial, menu, practice, and multiple environments.
Included Environments:• Ship Hangar• Submarine• Ship Bridge• Ship Below Deck
Overview
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramD5T Cable Demo Highlights(cont.)
• Handling cables and understanding the difficulty added to the installation process because of the heavy and inflexible nature of the cable material.
• The process of cable installation and how different environments have different obstacles.
• Environmental awareness regarding design to better understand the spatial requirements for the installation of cables and avoid non-ideal design configurations.
Training Goals
Distribution Statement A – Approved for public release
National Shipbuilding Research ProgramARgos @ SNAME
Distribution Statement A – Approved for public release
