Validation of the UW Virtual Validation of the UW Virtual Reality TURP SimulatorReality TURP Simulator

Version 1.0Version 1.0

Presented By:

Timothy Kowalewski

Robert Sweet, MD

Peter Oppenheimer

Suzanne Weghorst

Jeff Berkley, Ph.D

What is TURP?What is TURP?

Transuretheral Resection of the ProstateTransuretheral Resection of the Prostate Gold-standard for treating obstructive Gold-standard for treating obstructive

urinary symptomsurinary symptoms Challenging to teach and learnChallenging to teach and learn

Goals of the ProjectGoals of the Project

Overall Project Goal:Overall Project Goal:• To help streamline urological procedure To help streamline urological procedure

education with the aid of simulation technologyeducation with the aid of simulation technology

Specific Goals of This Study:Specific Goals of This Study:• Establish validation status of Version 1.0, UW Establish validation status of Version 1.0, UW

TURP Simulator as a training and assessment tool.TURP Simulator as a training and assessment tool.

TURP: a good model for SimulationTURP: a good model for Simulation

Objective assessmentObjective assessment High risk to patientsHigh risk to patients

Apparent training gap Apparent training gap Common problemCommon problem

National Average Number of TURPS performed by residentsNational Average Number of TURPS performed by residents

Source: ACGMESource: ACGME

19901990 19921992 19941994 19961996 19981998

Sweet, et al. Journal of Endourology. October, 2002.

Mainstay operation Mainstay operation

Apparent demandApparent demand

Amenable to current Amenable to current technologytechnology

99%1%

99%: Validated Simulator Useful in Training

2%

75%

23%

75%: Validated Simulator Useful after Residency

……YESYES

98%2%

98%: TURP is the Gold Standard of care

TURP: a good model for SimulationTURP: a good model for Simulation

MethodsMethods

Simulator ConstructionSimulator Construction Virtual anatomy and visual elementsVirtual anatomy and visual elements

Real-time force / tactile feedbackReal-time force / tactile feedback

O.R. instrumentationO.R. instrumentation

External physical modelExternal physical model

Auditory cuesAuditory cues

MethodsMethods

Simulator ConstructionSimulator Construction Training VideoTraining Video Definition of task and errorsDefinition of task and errors

Statement of goal:Statement of goal:

“ “Efficiently resect as much tissue as possible while avoiding errors Efficiently resect as much tissue as possible while avoiding errors and minimizing blood loss, amount of irrigant used, coagulation and minimizing blood loss, amount of irrigant used, coagulation current and number of cuts…”current and number of cuts…”

Special Thanks to Anthony Gallager, Ph.D

MethodsMethods

Simulator ConstructionSimulator Construction Training VideoTraining Video Pre-Task QuestionnairePre-Task Questionnaire Demographics, gender, professional experience, education, ‘video Demographics, gender, professional experience, education, ‘video

game’ experience, TURP-related questions, etc… game’ experience, TURP-related questions, etc…

Stratification of databaseStratification of database

MethodsMethods

Simulator ConstructionSimulator Construction Training VideoTraining Video Pre-Task QuestionnairePre-Task Questionnaire Pre-Compiled, 5-minute TaskPre-Compiled, 5-minute Task Three trained, non-medical techniciansThree trained, non-medical technicians Consistent, pre-determined responses Consistent, pre-determined responses Privacy and AnonymityPrivacy and Anonymity

MethodsMethods

Simulator ConstructionSimulator Construction Training VideoTraining Video Pre-Task QuestionnairePre-Task Questionnaire Pre-Compiled, 5-minute TaskPre-Compiled, 5-minute Task

Post-Task QuestionnairePost-Task Questionnaire Critique of simulator according to its elementsCritique of simulator according to its elements Open feedback Open feedback

MethodsMethods

Simulator ConstructionSimulator Construction Training VideoTraining Video Pre-Task QuestionnairePre-Task Questionnaire Pre-Compiled, 5-minute TaskPre-Compiled, 5-minute Task

Post-Task QuestionnairePost-Task Questionnaire AUA Annual ConferenceAUA Annual Conference 72 Experts72 Experts 19 Trainees19 Trainees 19 Novices19 Novices

MetricsMetrics

Grams of tissue resectedGrams of tissue resected Amount of irrigant usedAmount of irrigant used Quantity of ambient bloodQuantity of ambient blood Number and severity of bleeding vesselsNumber and severity of bleeding vessels Instrument position and interactionInstrument position and interaction Number of cuts at tissueNumber of cuts at tissue ~ 12 Hz resolution~ 12 Hz resolution Combinations thereof …Combinations thereof …

Results: Results: Face ValidityFace Validity

Do you feel the UW TURP Simulator would be useful as a Do you feel the UW TURP Simulator would be useful as a training tool?training tool?

93%

7%YESNO

Results: Results: Face ValidityFace Validity

Would you like to see it implemented into the curriculum Would you like to see it implemented into the curriculum of residency programs?of residency programs?

88%

12%

YESNO

Results: Results: Face ValidityFace Validity

58%

40%

2%YESNOMAYBE

Do you feel it would be useful as a tool for accreditation?Do you feel it would be useful as a tool for accreditation?

Results: Content ValidityResults: Content Validity

Categorical Standardized Likert scale of global acceptability

0.650.32

1.93

0.27

2.2

1.231.46

-3

-2

-1

0

1

2

3

Visual Realism

TactileRealism

AuditoryRealism

Bleeding/Bubbles

Chips Instrumen-tation

Overall

Acceptability threshold

Slightly acceptable

Moderately acceptable

Totally acceptable

Slightly unacceptable

Moderately unacceptable

Totally unacceptable

Results: Results: Novices vs. ExpertsNovices vs. Experts

( p < )

0.0001 -0.0001+0.0001+

0.001+0.002+0.002+0.002+0.006+0.016+0.020 -0.032 -0.080+

0.097 -

Result of Levene-conditioned two-tailed T-test quantifies resolving potential of selected metrics to determine differences between novices and experts.

Results: Results: Construct ValidityConstruct Validity

Performance of Errors

0%5%

10%15%20%25%30%35%40%45%50%

Novice Trainee Expert

SphincterRedoutCapsule

Construct/Concurrent ValidityConstruct/Concurrent Validity (Cross-sectional)(Cross-sectional)

Metrics correlations mimicked the operating room.

Experts and trainees were more efficient at cutting than novices. (P<.01)

Experts were more efficient at coagulating bleeders than trainees who were more efficient than novices. (P<.05, P<.01)

Performance among experts did not exhibit “decay”

Video game experience influenced performance in novices and experts only.

ConclusionsConclusions

Established Face and Content ValidityEstablished Face and Content Validity Established Construct Validity Established Construct Validity (X-sectional)(X-sectional)

Addressed Concurrent Validity Addressed Concurrent Validity (incomplete)(incomplete)

NO assessment predictive validity or NO assessment predictive validity or longitudinal studieslongitudinal studies

- - - - - - For Version 1.0For Version 1.0 - - - - - -

Version 2.0Version 2.0Predictive and Construct ValidationPredictive and Construct Validation

Anatomy ID ModuleAnatomy ID Module

Coagulation ModuleCoagulation Module

Cutting ModuleCutting Module

Advanced Resection Advanced Resection ModuleModule