1 Virtual Reality Integrated Weld Training A scientific evaluation of training potential, cost effectiveness and implication for effective team learning. Richard T. Stone 1,2 , Kristopher Watts 1 , Peihan Zhong 1 1 Department of Industrial and Manufacturing Systems Engineering, 2 Department of Mechanical Engineering, Iowa State University Contact Information: Richard T. Stone 3027 Black Engineering Building Department of Industrial and Manufacturing Systems Engineering Iowa State University Ames, IA 50014 515 294 3644 [email protected]
Transcript
1
Virtual Reality Integrated Weld Training
A scientific evaluation of training potential, cost effectiveness and implication for
effective team learning.
Richard T. Stone1,2, Kristopher Watts1, Peihan Zhong1
1 Department of Industrial and Manufacturing Systems Engineering, 2 Department of Mechanical
Engineering, Iowa State University
Contact Information: Richard T. Stone 3027 Black Engineering Building Department of Industrial and Manufacturing Systems Engineering Iowa State University Ames, IA 50014 515 294 3644 [email protected]
2
Abstract
Training in the welding industry is a critical and often costly endeavor; this study examines
the training potential, team learning, material consumption, and cost implications of using
integrated virtual reality technology as a major part of weld training. In this study, 22 participants
were trained using one of two separate methods (traditional training (TT) and virtual reality
integrated training (VRI)). The results demonstrated that students trained using 50% virtual reality
(VRI) had training outcomes that surpassed those of traditionally trained (TT) students across four
distinctive weld qualifications (2F,1G,3F,3G). In addition, the VRI group demonstrated
significantly higher levels of team interaction which lead to increased team- based learning.
Lastly, the material cost impact of the VRI group was significantly less than that of the TT group
even though both schools operated over a full two-week period.
Introduction
Welding is a skill, and as such requires that its practitioners be trained to a standard; this
kind of training requires time, money, and talent. For nearly as long as modern welding has
existed innovators have been exploring new ways to increase the effectiveness of its training.
Currently computer based VR training (CS Wave) and immersive VR training systems (VRTEX,
ARC+) have generated interest because they have the potential to reduce training costs (Refs. 1-
3). However, cost savings is only beneficial if the result is a competent welder who is trained in
a timely manner.
3
Prior to this study, the direct training impact of using VR technology as an integrated part
of weld training has not been evaluated. Published works pertaining to VR technology in
welding focus primarily on the training technology and its development, not the development of
the trainee (Refs. 4, 2). Many studies have focused on the general use VR in training operation
and results are far from conclusive. Some studies have shown that the use of VR technology
leads to reduced learning and transfer of skills (e.g. Refs. 5, 6). Other studies have shown that the
use of VR technologies in training is not significantly different from real world training (e.g.
Refs. 7, 8). Many studies have found that the use of VR technologies leads to a superior transfer
of skills when compared to traditional methods (Refs. 9, 10, 11, 12). There are many reasons for
this diversity of findings, like the methodology used for investigating the transfer of training
(Ref. 13). More commonly, however it is the fidelity of the different VR machines evaluated and
the degree to which the individual technologies were suited to their tasks that account for the
major sources of inter-study variation (Refs. 14, 15).
Modern technology has evolved to a point such that some virtual reality systems have the
ability to create high fidelity immersive environments (due in large part to advanced physic
engines and graphics rendering capabilities) coupled with an ability to achieve realistic
kinesthetic movements (due to magnetic displacement technologies allowing for 6 DOF
movement). These aspects of current VR welding simulators allow users to utilize kinesthetic
and cognitive learning in a way never before available in the virtual environment. In addition,
some VR systems such as the VRTEX 360 allow users to work in teams, with one member
observing welding progress while the other conducts the actual VR welds. This kind of system
further encourages team based interaction and learning among users. It must be noted that the
authors hold the VRTEX 360 as an example of a VR system capable of providing a level of
4
realism and kinesthetic feedback appropriate for this study. The authors do not endorse this
product over others that possess the aforementioned capabilities.
Prior to conducting this investigation, the authors hypothesized that: (1) VR integrated
training would result in superior training outcomes when compared to traditional methods, (2)
the use of a state of the art VR system would lead to increased levels of team interaction and
learning, and (3) weld training conducted with VR integrated technology would be significantly
less expensive than training conducted using traditional means.
Background
Transfer of training paradigm
The simplest way to evaluate the amount of learning that has taken place during the course of
a training program is to measure performance prior to training and compare it with performance
measures after training has taken place (Ref. 16). Often, training performance is measured in terms
of both operation completion time and accuracy. These measures can be translated into training
effectiveness ratios (TER) that enable comparison between training conditions.
The transfer of training paradigm requires a minimum of two groups of trainees, functioning
as an experimental group and a control group (Ref. 17). The group(s) given a new instructional
device (or alternative method of training) is the experimental group(s). The group given the
standard training (or no training) is the control group. In this experiment, the experimental group
used VR training technology 50% of the time and traditional training the remainder of the time
(VRI), whereas the control group used traditional means of training 100% of the time (TT). To
employ the transfer of training paradigm effectively, it is necessary to select appropriate
measurements to determine the extent to which training has been effective. In the case of this
5
study the qualification rate (the number of welders qualified for a specific position) was used as
the primary performance measure. The performance of the control group, measured in terms of
time and qualification rate, was used as a baseline. A positive transfer effect occurs when the
experimental group performs as well as or better than the control group. In the transfer of
training paradigm, the control group is automatically assigned a TER of zero. A TER greater
than zero represents a positive transfer effect; while a TER less than zero indicates a negative
transfer effect. The percent transfer is the absolute difference between control and experimental
group performance. The transfer of training paradigm is an effective tool in the assessment of
alternative training methods, and has commonly been used to determine the transfer effect
between virtual reality, augmented reality, and real training environments (e.g. Refs. 18, 19, 20,
15) particularly in laparoscopic surgery (e.g. Refs. 21, 22) as well as in aircraft simulation (Refs.
23, 24).
Team Interaction and Learning
Team learning occurs when multiple individuals carry out activities that enhance the
acquisition and development of competencies in all team members. Research has shown that
students who learn in team situations have a stronger tendency to learn from past experiences
and are more likely to take actions that lead to continuous development (Ref. 25). This has been
documented many times in various settings including many college classrooms (Refs. 26, 27) In
this study, the team learning questionnaire (TLQ) that was developed and validated by Breso et
al. in 2008 formed the basis for our team learning evaluation (Ref. 28). The TLQ evaluation was
modified so that the questions and content were specific to the domain of weld training. The
TLQ method of evaluation tracked three key dimensions of team learning and interaction that
were relevant to this study: (1) Continuous Improvement Seeking (the degree to which a team
6
can learn from previous experiences), (2) Dialogue Promotion and Open Communication (the
degree to which open and honest communication is encouraged and takes place within a
team),and (3) Collaborative Learning ( the degree to which team members are seen and used as
sources of knowledge by the rest of the team). Each dimension consists of a series of questions,
which the participant answers on a five-point scale (the higher the rating for a given question the
more positive the participant feels about the team learning for that question). In addition to TLQ
the authors of this study used continuous video and auditory recordings to assess the amount of
time students spent interacting within the weld booths.
Experiment
Training Facilities and Equipment
Both a traditional welding and a VR welding facility were constructed on the Iowa State
University Campus. The traditional facility housed six welding booths. Each booth was equipped
with the following: a new Lincoln Electric Power MIG 350MP welder with SMAW (stick metal
arc welding) attachments, two auto adjusting welding helmets, multiple sets of welding jackets and
gloves, power grinders, slag hammer, wire brushes, welding table, quenching buckets, and other
miscellaneous welding equipment. The welding facility was stocked with an ample supply of
runoff tabs, flat stock plates, groove plates, and 7018 electrodes.
The virtual reality weld training facility was located one floor below the traditional facility
and housed weld booths of the same size and dimension as their traditional counterparts. Each
booth contained a new Virtual Reality Welding Trainer with SMAW attachments and multiple sets
of welding jackets and gloves. The VRTEX 360 trainer was chosen because it is the highest
7
fidelity VR simulator currently available, and has design features that the authors felt would
greatly affect team based learning.
CWI
Achieving the rank of CWI represents a base standard for instructor capability. As such,
the capability of the CWI was considered to be a controlled variable. However, it is important to
note that individual teaching styles and capabilities are an important influencing factor in
knowledge acquisition. For this reason the experimenters observed 4 different CWI’s at 3 different
weld schools so as to learn what individual differences can be expected to exist between
instructors. Analysis revealed that the most significant factor was overall experience in teaching
(how long they have been instructing). For this reason the experimental protocol of this study
called for a CWI with at least 15 years of active teaching experience.
In this study, one paid CWI (who possessed greater than 15 years of instructing experience)
trained participants in both the TT and VRI groups. All CWI activities were closely monitored by
the experimenter to ensure that a consistent style of interaction and information exchange was
maintained between the CWI and participants in both groups. Lastly, post study questionnaires
sent to participants revealed that participants in the TW group rated their instructor’s capabilities as
a teacher at 4.2/5 and participants in the VRI group rated their instructors as 3.8/5. This indicated
that the perception of the instruction between the two groups was not significantly different. The
controls for the CWI were appropriate; if an alternative CWI with similar experience were to have
been used the overall outcome would be expected to remain the same.
Participants
There were 22 participants in total (21 males and one female). All participants committed
to 80 training hours over the course of two weeks. Participants were randomly assigned to one
8
of two groups. Group one (VRI) subjects were trained with 50 percent VR + 50 percent
traditional training, whereas group two (TT) subjects were trained using only the traditional
training system. Participants in this study were screened to ensure little to no welding experience
prior to the beginning of this study. The four participants with some previous experience were
evenly distributed between the two experimental groups. Participants in the TT and VRI groups
had an average age of 44 and 41 years respectively.
Independent and Dependent Variables
The primary independent variable in this experiment was training type at two levels,
representing the type of interface tested: Traditional Weld Training (TT) and 50% Virtual
Reality Training (VRI).
There were five major dependent measures in this investigation: percentage transfer,
training effectiveness ratio (TER), team learning, material consumption, and cost effectiveness.
Percentage transfer and TER are both training potential measures. As such, both were based on
the outcomes of participant qualification rates and training time. Qualification rates were
evaluated for each of four different weld positions tested in this study, including the 2F, 1G, 3F,
and 3G positions. Training time was defined as the total amount of time taken to train for a
qualification. Team learning was measured using the TLQ questioner and follow-up video
evaluation. Material consumption and cost effectiveness were functions of total plate, and
electrodes.
Experimental Procedure
Prior to experimentation, all participants gave informed consent, followed by individual
screening tests to ensure that they possessed normal visual acuity, depth perception, and hearing.
9
Upon completion of screening tests participants were randomly assigned to either the VRI or the
TT experimental group. The TT group trained at ISU for two weeks, and then one week later the
VRI group trained for two weeks.
In the traditional welding school (TT group), participants were trained in the principles
and practical application of welding techniques starting with the simplest position (2F), and
proceeding through to the most difficult (3G). The maximum amount of training time allotted
for teaching was fixed, this time included formal lectures and practical lab training conducted by
a certified AWS CWI. The CWI was responsible for evaluating welds to determine whether or
not a participant was ready to be tested prior to the end of their total allotted training time.
Following the training for each qualification, participants were given their test plate. If the test
plate for the qualification test passed the CWI’s visual inspection, it was sent to an independent
laboratory for structural testing. Qualification for certification was based on the results of this
structural testing. Immediately following the qualification tests for all four welds, participants
were administered TQL evaluations.
In the virtual reality integrated welding school (VRI), the experiment was conducted in
the same basic manner as the previous group. Both TT and VRI groups were given the same
overall training time opportunity for each weld type. The major difference between traditional
training and virtual reality integrated training was in the training system itself. Participants in the
VRI group spent only 50% of their time training (lectures and practical lab training) under the
direction of an AWS CWI for each weld type. The remaining 50% of their time was spent
training on the VR system. During VR training time, the participants (in pairs) used the VR
system to conduct virtual welds of each of the four weld types on which they would be tested. If
10
the participants were able to earn a machine generated quality score of 85% at least twice in a
row for a weld, they were permitted to discontinue their VR training time early.
Results
Training Potential
Training potential is defined by both the percent transfer and the transfer effectiveness ratio
(TER). These measures encompass both the differences in certification outcomes between the
groups as well as the differences in absolute training time between the groups.
Figure 1 shows the training differences in terms of qualification rate and training time for
the 2F position. Participants in the VR50 group (q. rate = 100%, M time = 12.27 hours)
outperformed the TW (q. rate = 81.8%, M time = 15.05 hours) group in terms of both
qualification rate and training time. The VRI group was found to have a 22.2% positive transfer
and a TER of 1.81 when compared to the TT group.
Figure 2 shows the training differences in terms of qualification rate and training time for
the 1G position. Participants in the VRI group (q. rate = 90.1%, M time = 11.72 hours)
outperformed the TT (q. rate = 54.5%, M time = 14.09 hours) group in terms of both
qualification rate and training time. The VRI group was found to have a 66.7% positive transfer
and a TER of 5.68 when compared to the TT group.
Figure 3 shows the training differences in terms of qualification rate and training time for
the 3F position. Participants in the VRI group (q. rate = 72.7%, M time = 11.60 hours)
outperformed the TT (q. rate = 45.5%, M time = 14.54 hours) group in terms of both
11
qualification rate and training time. The VR50 group was found to have a 60% positive transfer
and a TER of 5.17 when compared to the TW group.
Figure 4 shows the training differences in terms of qualification rate and training time for
the 3G position. Participants in the VRI group (q. rate = 45.5%, M time = 12.25 hours)
outperformed the TT (q. rate = 36.4%, M time = 15.31 hours) group in terms of both
qualification rate and training time. The VRI group was found to have a 25% positive transfer
and a TER of 2.04 when compared to the TT group.
Team Interaction and Learning
Team interaction and learning was assessed across three dimensions (1-Continuous
Improvement Seeking, 2-Dialogue Promotion and Open Communication, and 3-Collaborative
Learning), each representing a different aspect of cognitive capability. Interaction styles were
evaluated using video-based interaction analysis.
The VRI (M score = 4.47) group was not found to be significantly distinctive from the TT
(M score = 4.14) group in terms of continuous improvement seeking (T0.05, 1, 20 = -1.617, P =
0.121). Hence, both groups demonstrated a very strong desire to learn from their experiences
and to use what they learned to improve as individuals and as a team. This finding indicates that
the participants in both groups were equally willing to learn in the team context.
The VRI (M score = 4.63) group was found to be significantly more developed in terms
Dialogue Promotion and Open Communication than was the TT group (M score = 3.85) (T0.05, 1,
20 = -4.542, P < 0.001). Students in the VRI group were significantly more likely to engage in
task specific communication with their team member than were students in the TT group. Video
analysis revealed that the VRI group spent an average of 32% of their shared-booth virtual
12
reality training time engaged in training-relevant discussion (this discussion was primarily
related to the screen-observing student directing the student performing a virtual weld). This can
be compared to only 17% of the time spent in training-related discussion when sharing a booth in
the real world training facility (this discussion occurred primarily when the team member were
in-between passes). Video analysis demonstrated that participants in the TT group engaged in
training relevant discussion an average of 10% of the time when sharing a booth in the real world
training facility (this discussion occurred when the team member had completed a pass or a full
plate).
The VRI (M score = 4.73) group was found to be significantly more developed in terms of
Collaborative Learning than was the TT group (M score = 3.30) (T0.05, 1, 20 = -8.318, P < 0.001).
Students in the VRI viewed their team members as sources of knowledge to a greater extended
than did students in the TT group. The higher the level of collaborative learning in a team the
greater the likelihood that positive teamwork interaction took place and they learned from one
another.
Material Consumption
real world material usage
The VRI group used significantly less flat plates than the TT group (T0.05, 1, 20 = 4.607, P <
0.001). The VRI group used 210 flat plates compared to the TT group that used 288.
Also, the VRI group used significantly less groove plates than did the TT group. The VRI group
used 50 groove plates compared to 63 for the TT group (T0.05, 1, 20 = 2.711, P = 0.013). Similarly,
the VRI group used significantly less electrodes than did the TT group, 111.2 lbs for the VRI
group compared to 187.6 lbs for the TT group, (T0.05, 1, 20 = 8.958, P < 0.001).
virtual world stock material usage
13
The VRI group used significantly larger amount of overall flat plates (when considering both
virtual and real world plates) than the TT group. The VRI group used a total of 550 combined
(real + virtual) flat plates compared to the 288 real plates the TT group used (T0.05, 1, 20 = -12.343,
P < 0.001). The VRI group used significantly larger amount of overall groove plates than did the
TT group. The VRI group used a total of 82 combined plates compared to the 63 real plates the
TT group used (T0.05, 1, 20 = -8.542, P < 0.001). However, the VRI group did not use a
significantly larger number of electrodes than did the TT group. The VRI group used 205.2 lbs
of electrode versus 187.6 lbs used by the TT group (T0.05, 1, 20 = -1.386, P = 0.181). The
increased plate use in the VRI group reflects the fact that these students were able to conduct
more overall welds due to the fact the virtual environment allows for focused welding time
without the need for setup, tacking, etc. No difference in electrode usage was discovered
primarily because the VR environment does not suffer from sticking and associated electrode
abandonment, as does the real world condition.
Material Costing
The material costs in this study reflect the consumables purchase prices; it must be noted that
these prices may vary depending on a company’s vendor and purchasing agreements.
Additionally, prices reported in this study do not reflect shipping costs. Prices in this study are
