Haptics in Medical Simulation - CISLcisl.stanford.edu/content/dam/sm/cisl/documents/About...In...

Haptics in Medical Simulation

Allison M. Okamura

Associate ProfessorDepartment of Mechanical Engineering

Design and Biomechanics GroupsStanford University

Collaborative Haptics and Robotics in Medicine (CHARM) Labhttp://charm.stanford.edu

Haptics Tissue modeling forhaptic simulation

Augmented reality Current Projects

Allison Okamura ◆ Stanford University ◆ 2012




hearing

sight

smell

taste

touch

which sense is most valuable

to you?

which would you relinquish

last?


what would life be like without touch?

Ian Waterman


why do we have brains?

Daniel Wolpert


sightcentralized

broadpassive

cognitive

touchdistributednarrowactivephysical


hap·tic ('hap-tik)adj. Of or relating to the sense of touch.

[Greek haptikos, from haptesthai, to grasp, touch. (1890)]

TemperatureTexture

SlipVibration

Force

Location/configurationMotionForce

ComplianceCutaneous

Kinesthesia

The haptic senses work together with the motor control system to:- Coordinate movement- Enable perception

Johansson and Westling


haptics making inroadsEntertainment

Nintendo Stanford/JHU Immersion

Education

Human-Computer Interfaces

BMWSamsung Boeing


Haptics



Tissue modeling forhaptic simulation

Haptics for medical simulationMain Challenge:Users should be allowed to interact with the simulator in any manner, and the simulator should respond appropriately.

(Unanswered) Question:How realistic does the simulation need to be to have a significant impact?

This talk addresses the mechanical and haptic interaction components medical simulation.


is the simulator is any good?• Face validity: does the system present an environment

resembling that which is encountered during a medical procedure?

• Content validity: is a skill measured by the system measured the specific skill desired, and not a different one?

• Construct validity: can the system capture the differences between experts and novices?

• Concurrent validity: to what extent does testing performance with the simulator yield the same results as other measures?

• Predictive validity: does performance/training with the simulator transfer to improvements in clinical practice?


Reality-Based Modeling

UBC videos courtesy Simon DiMaio


Cutting with Scissors

Blank Skin

Liver Tendon Forc

e

Scissor angle


Needle Insertion

Time

Forc

e Ve

loci

ty


Needle InsertionAllison Okamura ◆ Stanford University ◆ 2012

Simulation

Accuracy

Needle InsertionAllison Okamura ◆ Stanford University ◆ 2012

Good models also lead to new technology: Needle Steering

Alterovitz, et al. Webster, Reed, et al.



Current Projects


Augmented reality

Surgeon

PatientImage source: www.physicianphotos.com

Open SurgeryAllison Okamura ◆ Stanford University ◆ 2012

Surgeon

Instrument/Camera

PatientImage source: www.womenssurgerygroup.com

Minimally Invasive SurgeryAllison Okamura ◆ Stanford University ◆ 2012

© 2008 Intuitive Surgical, Inc.

Surgeon

Master Console

Teleoperated Robot-Assisted Minimally Invasive Surgery

Information-EnhancedRMIS (with Haptics)

Patient-Side Robot

Patient

Instrument/Camera


da Vinci Surgical System(Intuitive Surgical)

surgeon’sconsole

patient-siderobot


Experimental Platform

research version of the da Vinci surgical system


Force feedback:haptics and graphics


Effects of force feedback1. Haptic + Graphical2. Haptic Only3. Graphical Only

4. No Force Feedback

erro

r (d

eg)

feedback

Error (deg)

haptic feedbackreduces error


Differentiating material propertiesAllison Okamura ◆ Stanford University ◆ 2012

porcine kidney

arti!cial lumps

!ducial markers

!ducial markers

!ducialmarkers

!ducialmarkers

!ducial markers

!ducial markers

!ducialmarkers

!ducialmarkers

porcine kidney

X-ray imageCamera image

Lump detection taskAllison Okamura ◆ Stanford University ◆ 2012

Lump detection Results1 2

3 4


What about manipulation?

no overlay dot overlay

In collaboration with D. D. Yuh of JHMI Cardiac Surgery

Graphical force feedback results in lower peak forces, lower variability of forces, and fewer broken

sutures for novice robot-assisted surgeons



Augmented reality


Current Projects

In medical simulation today, haptic feedback is typically restricted to that provided by

• artificial materials (e.g., mannequins)

or

• resolved forces (e.g., force feedback haptic interfaces in virtual reality simulators)

Laerdal’s SimMan

Simon DiMaio, UBC


Kinetic System for Medical Simulation (KineSys MedSim)

In collaboration with T. Judkins (IAI), J. E. Colgate (Tangible Haptics), D. Gaba (Stanford)


Kinesthetic haptic display

Images courtesy T. Judkins (IAI)


Electrostatic tactile display

Disney Research

Tangible Haptics


Haptic Jamming: A Deformable Geometry, Variable Stiffness Tactile Display

using Pneumatics and Particle Jamming

Jim Gwilliam and Andrew Stanley

• Simultaneously controllable surface geometry and mechanical properties

• Uses pneumatics and “particle jamming”: a cell made from a flexible membrane filled with granular material solidifies when air is vacuumed out


Haptic Jamming DisplayAllison Okamura ◆ Stanford University ◆ 2012

Force deflection

curves before vacuuming

the cells

0 5 10 15 200

5

10

15

20

25

30

35

Forc

e (

N)

Deflection (mm)

Vacuumed After Inflation

No Inflation Max Inflation

0 5 10 15 200

5

10

15

20

25

30

35Inflated, not Vacuumed

Deflection (mm)

1.0" Cell

2.0" Cell

1.5" Cell

3.0" Cell

Force-deflection

curves after vacuuming

the cells



Augmented reality Current projects


JHU APL


AcknowledgmentsPast PhD/Postdoc Lab MembersJake Abbott (Univ. Utah)Nasir Bhanpuri* (USC)Steven Charles*+ (BYU)Jessica Crouch+ (Old Dominion Univ.)David Grow (New Mexico Tech.)Paul Griffiths+ (Intuitive Surgical)Netta Gurari (IIT)Katherine Kuchenbecker+ (UPenn)Mohsen Mahvash+ (Boston Univ.)Panadda Marayong (Cal St Long Beach)Steven Marra+ (JHU)Sarthak Misra (U. Twente)Kyle Reed*+ (USF)Lawton Verner (Intuitive Surgical)Robert Webster* (Vanderbilt Univ.)Tomonori Yamamoto (EndoMaster) *co-advised, +postdoc

… And manytalentedundergraduateresearchers!

Current Lab MembersTroy Adebar (Stanford)Amy Blank* (JHU)Nick Colonnese (Stanford)Tricia Gibo* (JHU)Jim Gwilliam* (JHU)Ann Majewicz (Stanford)Kirk Nichols (Stanford)Ilana Nisky+ (Stanford)Zhan Fan Quek (Stanford)Michele Rotella (Stanford)Sam Schorr (Stanford)Andrew Stanley (Stanford)Tom Wedlick (JHU)

CollaboratorsAmy Bastian (KKI/JHU Neurosci.)Michael Choti (JHU Surgery)Noah Cowan (JHU Mech. Eng.)David Gaba (Stanford Medicine)Ken Goldberg (UCB IEOR)Gregory Hager (JHU Comp. Sci.)Steven Hsiao (JHU Neuroscience)Michael Hsieh (Stanford Urology)Peter Kazanzides (JHU Comp. Sci.)KT Ramesh (JHU Mech Eng.)Danny Song (JHU Radiology)Russell Taylor (JHU Comp. Sci.)Louis Whitcomb (JHU Mech. Eng.)David Yuh (JHU Surgery)Intuitive Surgical, Inc.

SupportNational Science FoundationNational Institutes of Health (NIBIB, NCI, NINDS) Whitaker FoundationJHU Brain Science InstituteDARPA


Date post:	13-Aug-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Haptics in Medical Simulation - CISLcisl.stanford.edu/content/dam/sm/cisl/documents/About...In...

Documents