HAPTIC RENDERING OF CUTTING TOOLSINTERACTING WITH 3D VOLUMETRIC MODELS WITH

APPLICATION TODENTAL SURGERY SIMULATION

Examination Committee : Prof. Peter Haddawy (Chairman) Dr. Matthew Dailey, Dr. Manukid Parnichkun

Domain Expert : Dr. Siriwan Suebnukarn

Presenter : Kugamoorthy Gajananan

Outline

Introduction

Related Works

Methodology

Results

Conclusion

Background(1)

Traditional Healthcare Training Methods[E.g. Clinical and Surgical Skills in Dentistry]

– Mannequins (Plastic Models)– Live patient

Introduction

[1]

Background(2)

Drawbacks -Traditional Training Methods

Introduction

Expensive Equipments Lack of Cases

Ethical Concerns Busy Experts

Background(3)

Virtual Reality (VR) Simulators

Introduction

Operating Room

SurgicalEquipmentsPatients

[2]

VR based medical simulation provides a highly realistic surgical environment – Natural Interaction is guaranteed

Virtual Reality - 3D Volumetric Model

[Tuwien, 2008] [Nationmaster,2008]

Introduction

Haptic DeviceIntroduction

[4]

Haptic Example DemoIntroduction

[5]

Benefits of using a simulator

• No incremental cost • No Risk• Reusable ,Repeatable and Always available• Maximum teaching effectiveness(More cases).• Gather very precise data about the procedures.

– Assesment of skills.– Feedback in terms of teaching.

• Continuous monitoring of competence possible

Introduction

Available PrototypesRelated Works

(a).PerioSim (b). PerioSim (c). VRDTS (d).IDSS

a.

c.

b.

d.

Available Research Studies

a [Kim et al.2005]

b. [Eriksson et al.2006]

c. [Yau et al.2006]

d. [Wang et al.2003]

Related Works

The Goal and Objectives

• Real Time Manipulation of 3D Volumetric Data[Cutting/Drilling Simulation]– Volumetric data representation– Different shapes of virtual tools– Force feedback computation algorithms for

different shapes of virtual tools– Formal evaluation of the simulator

DemoMethodology

Snap Shot - Capsule Shape ToolMethodology

Snap Shot - Cylinder Shape ToolMethodology

Snap Shot - Cone Shape ToolMethodology

Snap Shot - Sphere Shape ToolMethodology

Graphic RenderingHaptic Rendering

Dental Simulator’s Architectural Design

Collision DetectionCollision Detection

Force Computation

Force Computation

Methodology

Surface Model

Simulation

1000 Hz 30 Hz

Volume Model

Haptic Interface

Main Technical Challenges• Real time cutting simulation

– Maintaining threads’ refresh rates at required level

• Compact data structure– Hybrid data structure : Volumetric and Surface

Representation of Tooth Model

• Efficient collision detection algorithm

• Fast Force feedback computation for different shapes of tool

Haptic - 1000 Hz Graphic - 30 Hz

Methodology

EvaluationMethodology

Dentists from Faculty of Dentistry in Thammasat University

Evaluation ResultsRealism of Visual Display N Mean Standard deviation

Tooth Model Realism Crown Roots Root Canal orifices

101010

4.805.205.20

1.320.920.84

3D Instruments Realism Capsule Cylindrical Conical Spherical

10101010

5.305.205.105.10

0.820.630.880.74

N - Number of participants

Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7.

Results

Evaluation ResultsRealism of Feel N Mean Standard deviation

Tooth Anatomy Different Stiffness Cutting Force Cutting Time

101010

4.404.105.30

1.320.920.84

3D Instruments Realism Capsule Cutting Cylindrical Cutting Conical Cutting Spherical Cutting

10101010

4.804.804.804.70

1.141.031.031.06

N - Number of participants

Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7.

Results

Evaluation ResultsUsefulness and Easiness N Mean Standard deviation

Usefulness Learning Dental Skills Assessing Dental Skills Teaching Tooth PreparationEasiness Comfortable Felt Focussed Friendly User Interface

101010

101010

5.604.905.60

4.404.604.90

0.721.280.52

1.070.840.99

N - Number of participants

Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7.

Results

Key Research Contributions (To be published)

Cylinder bur - roundCylinder bur

Taper cylinderRound bur

Force Feedback algorithms for different shapes of tool

Conclusion

Key Research Contributions(2)• Visual Display (Haptic integration with PolyVox

Surface extraction Library)

• Formal Evaluation

Conclusion

Potential Use

• Training tool for Thammasat University Dental Students

• Intelligent Dental Tutoring - Ongoing PhD research by Phattanapon Rhienmora

(Prof.Peter Haddawy’s research group)

Conclusion

Future Innovations

• General Techniques– Tooth Filling

– Bone Drilling (E.g Skull )

Conclusion

Thank you for your attention !

References

• [1] http://www.images.google.com• [2] Phattanapon Rhienmora’s proposal• [3] http://www.denx.com/• [4] http://www.sensable.com/• [5] http://www.reachin.se/

Acknowledgement

• Thank you Professor Peter Haddawy , Dr.Matthew Dailey , Dr.Manukid Parnichkun , Dr.Siriwan Suebnukarn for your guidence, advices and thouhtful discussions.

• Thank you Dr.Sumantha Guha for your guidence in Visualization.

• Thank you Dr.Chris Sewell for supporting my effort.

• Thank you Dr.David Williams for your great help.

• Thank you “Pat” for your valuable ideas for the force computation , continuous support, suggestions, critics and discussions.

Possible Publications

Additional Information

Data Acquisition

[Rhienmora, 2008]

Design and Implementation

• Tooth Model - Patient’s CT scan data

Tooth Resolutions :

a. 109 * 84 * 97

(0,100,150,255)

b. 103 * 94 * 161 (0 – 255)

c. 128 * 128 * 256 (0 – 255)

Data Representation(1)

• Volumetric Model – Voxels (256 * 256 * 256)– Pad up zeros on the orginal size (128 * 128 * 256)

Design and Implementation

[Ooeygui, 2008]

Block Volume

Data Representation(2)• Surface Model

– PolyVox devides the volume model into regions– PolyVox reconstruct surface mesh for each region

VolumeSurface

Marching Cube Algorithm

[Rhienmora , 2008]

Data Representation(3)

• Tool Models – Volumetric Sample Points

Design and Implementation

Haptic Rendering

• Collision Detection

Design and Implementation

Haptic Rendering• Force Computation –Underlying

model • Spring-damper model

F = k.∆x – b.vk stiffness b damping

constant

Haptic Rendering(1)• Force Vector Computation - Force Models for

Cylinder ,Capsule, Cone Shapes

Design and Implementation

Haptic Rendering(2)

• Force Computation - Magnitude– Magnitude based on voxel density

• Averaging the density values of the voxels with in the local area that a tool model intersects.

• Local area refers to the voxels collided by the immersed volume sample points.

• This implies the fact that stiffness calculated by averaging the stiffness values of the voxels with in the local area.

Design and Implementation

Haptic Rendering(3)• Force Computation - Magnitude

– Nonlinear magnitude computation• Based on the number of collided sample points of the tool. • Non-linear function of the number of immersed sample points.

• Resultant Force– The direction of force (unit vector) computed by force

response algorithm for different shapes of the tool.– This force vector scaled up by the magnitude calculated.

• Resultant force smoothed and filtered– Moving Window (size 100 ) Technique

Design and Implementation

Technical Challenges

• Synchronization of Haptic and Graphic threads• Stability• Impact of number of sample points of tool

models• Resolution of volumetric data• Limitation of the device

Design and Implementation

Tools and Libraries• OpenHaptics Toolkit -OpenHaptics SDK

(HDAPI)

• PolyVox Technology – Extract Surface Mesh

Design and Implementation

Evaluation Objectives

– Simulator realism for images of tooth model and instrument

– Simulator realism for the feel of the tooth and instruments while cutting – the perceived realism of Tooth cutting

– Ratings for simulator usefulness and easiness

Evaluation

Conclusions

Done

Done

Done

Evaluation

Done

Done

Key Research Contributions

• Volumetric Sample point technique used to implement different force models for different shapes of dental tool.

• Force feedback model weights the contribution of the intersected voxels' density to compute the force feedback's magnitude

• Force feedback model considers the number of immersed sample points to compute the force feedback's magnitude

• PolyVox integrated for the first time with haptic – very much improved graphic rendering ,Hyprid Data Structure

• Design of a formal evaluation

Future Work

• Whole Tool (Handle + Tip) Collision detection.• More advanced approaches for material

removal.• Volume feedback (Physical contact sound).• Undo/redo function.• Tooth filling.• Control the mirror using another haptic

device.

Commercial Systems

[3]

DentSim (DenX USA)

[3]

Related Works

Force computation• Implement force models for different shapes

of virtual dental tool

Introduction

[Petersik et al., 2003]

&

[Nicolasasenjoosorio, 2008]