A Mixed Reality Approach for Merging Abstract and Concrete Knowledge

John QuarlesDepartment of CISE

Samsun LampotangDepartment of Anesthesiology

Ira FischlerDepartment of Psycholgy

Paul FishwickDepartment of CISE

Benjamin LokDepartment of CISE

Anesthesia Education

Current Learning Process

The Virtual Anesthesia Machine (VAM) – an abstract 2D

simulation

A real Anesthesia Machine – a physical simulation

Knowledge Types

Type Abstract Concrete

Skill Set Concepts that can be applied to many real machine models

Procedural, tactile, and psychomotor skills

Example Invisible gas flow Performing a procedure with a specific machine

Transfer Knowledge?

Problem: Merging Knowledge Types Anesthesia machine trainees must mentally

merge: Abstract Concepts Procedural Skills

Mixed Reality (MR) can merge real and virtual spaces Virtual space = Abstract Real Space = Concrete

Can MR’s Merging of spaces improve users’ merging of abstract and concrete knowledge?

Merging Abstract and Concrete

Merge the abstract physical simulation with the physical device

The Augmented Anesthesia Machine

Uses a magic lens approach to merge:Abstract VAM simulationReal Anesthesia Machine TUI

Overview

The Augmented Anesthesia Machine The Augmented Learning Process User Study: VAM vs. AAM

MR enables abstract + concrete knowledge merging

Previous Work

Tangible User Interfaces Two Main Parts:

Computational Media Physical Interface

Interface is meaningful Representative of a part of the computational media As opposed to an abstract interface (i.e. a mouse)

AAM Computational media = VAM Physical Interface = real anesthesia machine

[Ishii 1997, Ullmer 2000]

Previous Work

Magic Lenses2D to 3DApplications in AR/MR

AAMUses a magic lens to visually merge the

abstract and concrete spaces

[Bier 1993, Looser 2004]

Motivation

75% of Anesthesia Machine Related Accidents resulting in death or brain damage are due to user error.Many anesthesia providers do not fully

understand how the machine functions internally

VAM was created to reduce this number

Motivation for the VAM

Created by Dr. Samsun Lampotang Advantages of the VAM

visualize the invisible (i.e. gas flow) Spatial layout simplified for ease of visualization Web disseminated Increases understanding of internal function

[Fishcler et. al. 2006] 30000 Registered Users One Billion Hits per year

(http://vam.anest.ufl.edu) Problems still exist in knowledge transfer from

the VAM to the real machine

Motivation VAM to AAM: A Mapping Problem

A A

B

B

MR: Enabling Multiple Represetations Multiple Representations improve overall comprehension

of a concept. VAM: Abstract Representation Anesthesia Machine: Concrete Representation AAM: Combined Representation

Proposed Augmented Learning Process:

Abstract Representation

(VAM)

Combined Representations

(AAM)

Concrete Representation

(Anesthesia Machine)

The AAM Representation

Tracked 6DOF magic lens shows overlaid VAM simulation Real machine interaction

Anesthesia Machine Tracking System 2D Optical Tracking with OpenCV

Infrared Markers for knobs Infrared LEDs connected to buttonsColor tracking for gauges (bright red)

3 Unibrain webcams positioned around the machine in positions of minimal occlusion

Magic Lens Display

HP TC1100 3.7 lbs 10” screen

Not See through Uses a 3D model of the

machine, registered to the real machine

Easier to enable consistent registration

Less hardware than video see through

Magic Lens Tracking System

Outside-Looking-in Optical Tracking

Infrared Markers Specifications:

2 Unibrain Fire-I Webcams 640x480 at 30 fps

Tracking Volume: 3x3x3 m Accuracy: 1cm Jitter: 5mm Latency: 70 ms

User Study

Can AAM-Concrete help users to merge abstract and concrete knowledge?

Between subjects study (n=20)VAM Training GroupAAM Training Group

Population and Environment

20 Psychology Students4 males, 16 femalesReceived class creditKnew nothing about anesthesia machines

Conducted in a quiet lab environment

Procedure

Informed Consent

Introduction Machine

Functions

5 Training Exercises

VAM

AAM

Spatial Cognition

Tests

Self Evaluation of Training

Written Test

Real Machine Intro

VAM

AAM

Hands-on Fault Test

Questionnaires

Day 1 (90 min)

Day 2 (60 min)

Metrics

Efficiency of Training Time to complete 5 training exercises

Abstract Knowledge Acquisition Written Anesthesia Machine Test

Short Answer and muliple choice From the Anesthesia Patient Safety Foundation workbook

Concrete Knowledge Acquistion Hands-on fault test

Faulty inspiratory valve

Fault Test Results Hypothesis 1: The AAM is more effective at teaching

concrete concepts.

AAM group found the faults significantly more often than VAM group

Accept Hypothesis 1: The AAM trains concrete knowledge more effectively than the VAM

Group # Participants Successful

AAM 6 out of 10

VAM 1 out of 10

Written Test Results

Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts

There were no significant differences in Written test scores (p<0.21)

Training Time Results

Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts

VAM Group trained significantly faster (p<0.002)

Accept Hypothesis 2: The VAM trains abstract knowledge more effectively than the AAM

Discussion

Hypothesis 3: The AAM improves transfer to the real machine by enabling the merging of abstract and concrete knowledge.

AAM group performed significantly better in the fault test

To solve the fault test, participants had to merge: Concrete knowledge: notice the inspiratory valve was

missing Abstract knowledge: understand the effect on the gas

flow of the machine – harmful to the patient Accept Hypothesis 3: AAM helps to merge

abstract and concrete knowledge

Conclusions

leverage MR to merge simulation typesCombining Abstract simulation with the

corresponding physical device i.e. merging the VAM with the real machine

MR enables multiple representations MR can help users to merge abstract and

concrete knowledge Improves training transfer into real world

domains