Physically-based Simulation of Medical Procedures for Training and Planning The Challenge Computer simulations of medical procedures enable physicians and other clinicians to train in a controlled environment that exposes them to both common and rare patient cases without risks to patient safety. Studies indicate that surgical skills learned using computational simulators directly improve operating room performance by significantly decreasing procedure time and reducing the frequency of medical errors by up to sixfold compared to traditional training. Surgical simulations also have uses for pre-operative planning. We have focused on simulating and planning medical procedures involving needles. Numerous medical procedures, including brachytherapy cancer treatment, biopsies, and anesthesia drug injections, require inserting a needle tip to a specific target location inside the human body. This is difficult because inserting needles causes the surrounding soft tissues to deform. Ignoring these deformations can result in substantial placement error, resulting in failure of the procedure or increased side effects. To facilitate physician training and planning for needle-based medical procedures, we are developing an interactive simulation of needle insertion in soft tissues. Screenshots from our prostate brachytherapy simulator. A needle is inserted from the left through the epidermis into the prostate gland. We simulate both bevel-tip flexible needles (top) and symmetric-tip stiff needles (bottom). The Approach Medical simulations are challenging to develop because they require both physical realism and real-time interactive performance. We are developing 2D and 3D simulations of needle insertion procedures by modeling tissue deformations using a finite element method, modeling needle frictional and cutting forces, and using novel re-meshing to ensure conformity of the mesh to the curvilinear needle path. As part of a multi-institution collaboration between UNC- Chapel Hill, UC Berkeley, and Johns Hopkins University, we are developing a new simulator that models tissue deformation, needle elasticity, and their interaction. It allows us to realistically simulate the deflections that occur as thin needles travel through inhomogeneous tissues. A motivation for modeling needle elasticity is a new class of steerable needles that have a flexible shaft that curves as it penetrates soft tissue due to asymmetric forces exerted at the needle’s bevel tip. By twisting the needle as it is inserted, a physician can steer its tip around obstacles to reach clinical targets in soft tissues. It is not easy to learn how to control steerable needles, and realistic training simulations will accelerate their deployment in clinical practice. Several impediments make it difficult to simulate the interaction between a needle and soft tissues: a static spatial discretization (e.g. a fixed finite element mesh) does not easily support the accurate computation of contact forces and needle steering; the mismatch between needle stiffness and tissue stiffness hinders numerical stability; and the simulation must run at interactive rates. To address these challenges, we have introduced (1) a novel algorithm for local remeshing, (2) an efficient algorithm for coupling a 3D finite element simulation and Department of Computer Science University of North Carolina at Chapel Hill March 2012 • Simulating medical procedures enables pre- operative procedure planning and surgical training in virtual environments. • Simulating medical procedures is challenging due to tissue deformations and complex tool/tissue interactions. • We created a real-time, interactive simulation of needle insertion in deformable tissues. • Simulation of needle insertion in deformable tissue can be used to anticipate and correct for needle placement errors due to tissue deformations.
Transcript
Physically-based Simulation of Medical Procedures for Training and Planning
The Challenge Computer simulations of medical procedures enable physicians and other clinicians to train in a controlled environment that exposes them to both common and rare patient cases without risks to patient safety. Studies indicate that surgical skills learned using computational simulators directly improve operating room performance by significantly decreasing procedure time and reducing the frequency of medical errors by up to sixfold compared to traditional training. Surgical simulations also have uses for pre-operative planning.
We have focused on simulating and planning medical procedures involving needles. Numerous medical procedures, including brachytherapy cancer treatment, biopsies, and anesthesia drug injections, require inserting a needle tip to a specific target location inside the human body. This is difficult because inserting needles causes the surrounding soft tissues to deform. Ignoring these deformations can result in substantial placement error, resulting in failure of the procedure or increased side effects. To facilitate physician training and planning for needle-based medical procedures, we are developing an interactive simulation of needle insertion in soft tissues.
Screenshots from our prostate brachytherapy simulator. A needle is
inserted from the left through the epidermis into the prostate gland. We simulate both bevel-tip flexible needles (top) and symmetric-tip stiff
needles (bottom).
The Approach Medical simulations are challenging to develop because they require both physical realism and real-time interactive performance. We are developing 2D and 3D simulations of needle insertion procedures by modeling tissue deformations using a finite element method, modeling needle frictional and cutting forces, and using novel re-meshing to ensure conformity of the mesh to the curvilinear needle path.
As part of a multi-institution collaboration between UNC-Chapel Hill, UC Berkeley, and Johns Hopkins University, we are developing a new simulator that models tissue deformation, needle elasticity, and their interaction. It allows us to realistically simulate the deflections that occur as thin needles travel through inhomogeneous tissues. A motivation for modeling needle elasticity is a new class of steerable needles that have a flexible shaft that curves as it penetrates soft tissue due to asymmetric forces exerted at the needle’s bevel tip. By twisting the needle as it is inserted, a physician can steer its tip around obstacles to reach clinical targets in soft tissues. It is not easy to learn how to control steerable needles, and realistic training simulations will accelerate their deployment in clinical practice.
Several impediments make it difficult to simulate the interaction between a needle and soft tissues: a static spatial discretization (e.g. a fixed finite element mesh) does not easily support the accurate computation of contact forces and needle steering; the mismatch between needle stiffness and tissue stiffness hinders numerical stability; and the simulation must run at interactive rates. To address these challenges, we have introduced (1) a novel algorithm for local remeshing, (2) an efficient algorithm for coupling a 3D finite element simulation and
Interactive Simulation of Surgical Needle Insertion and SteeringNuttapong Chentanez
U.C. BerkeleyRon Alterovitz
U.N.C. Chapel HillDaniel RitchieU.C. Berkeley
Lita ChoU.C. Berkeley
Kris K. HauserU.C. Berkeley
Ken GoldbergU.C. Berkeley
Jonathan R. ShewchukU.C. Berkeley
James F. O’BrienU.C. Berkeley
AbstractWe present algorithms for simulating and visualizing the inser-tion and steering of needles through deformable tissues for surgi-cal training and planning. Needle insertion is an essential compo-nent of many clinical procedures such as biopsies, injections, neuro-surgery, and brachytherapy cancer treatment. The success of theseprocedures depends on accurate guidance of the needle tip to a clin-ical target while avoiding vital tissues. Needle insertion deformsbody tissues, making accurate placement di⇥cult. Our interactiveneedle insertion simulator models the coupling between a steerableneedle and deformable tissue. We introduce (1) a novel algorithmfor local remeshing that quickly enforces the conformity of a tetra-hedral mesh to a curvilinear needle path, enabling accurate compu-tation of contact forces, (2) an e⇥cient method for coupling a 3Dfinite element simulation with a 1D inextensible rod with stick-slipfriction, and (3) optimizations that reduce the computation time forphysically based simulations. We can realistically and interactivelysimulate needle insertion into a prostate mesh of 13,375 tetrahedraand 2,763 vertices at a 25 Hz frame rate on an 8-core 3.0 GHz In-tel Xeon PC. The simulation models prostate brachytherapy withneedles of varying sti�ness, steering needles around obstacles, andsupports motion planning for robotic needle insertion. We evalu-ate the accuracy of the simulation by comparing against real-worldexperiments in which flexible, steerable needles were inserted intogel tissue phantoms.Keywords: surgical simulation, needle insertion, real-time finiteelement methods, coupled simulationCR Categories: I.3.5 [Computer Graphics]: ComputationalGeometry and Object Modeling—Physically based modeling;I.3.7 [Computer Graphics]: Three-Dimensional Graphics andRealism—Animation; I.6.8 [Simulation and Modeling]: Types ofSimulation—Animation.
1 IntroductionNeedle insertion is an essential component of many clinical proce-dures such as biopsies, injections, neurosurgery, and brachytherapycancer treatment [Abolhassani et al., 2007]. The success of theseprocedures depends on how close the needle tip is maneuvered tothe target. It is crucial that the needle avoid bone and other criticalstructures and organs [Kohn et al., 2000]. Unfortunately, needle in-sertion deforms body tissues enough that poor accuracy is the normin practice. For example, experienced physicians inserting radioac-tive seeds into the prostate gland for brachytherapy prostate cancer
From the ACM SIGGRAPH 2009 conference proceedings.
Permission to make digital or hard copies of all or part of this work for personalor classroom use is granted without fee provided that copies are not made ordistributed for profit or commercial advantage and that copies bear this noticeand the full citation on the first page. To copy otherwise, to republish, to post onservers or to redistribute to lists, requires prior specific permission and/or a fee.ACM SIGGRAPH 2009, New Orleansc� Copyright ACM 2009
a)
b)
Figure 1: Screenshots from our prostate brachytherapy simulator.A needle is inserted from the left through the epidermis and dermisinto the prostate gland. a) Bevel-tip flexible needle. b) Symmetric-tip sti� needle.
treatment experience average placement errors of 6.3 mm, about15% of the prostate’s diameter [Taschereau et al., 2000].
Computer simulations of needle insertion procedures enable physi-cians and other clinicians to train in a controlled environment thatexposes them to both common and rare patient cases without risksto patient safety. Studies indicate that surgical skills learned us-ing computational simulators directly improve operating room per-formance by significantly decreasing procedure time and reducingthe frequency of medical errors by up to sixfold compared to tradi-tional training [Seymour et al., 2002; Satava, 2005; Gallagher et al.,2005]. Surgical simulations also have uses for pre-operative plan-ning [Alterovitz and Goldberg, 2008; Taylor, 2006].
We present a new simulator that models tissue deformation, needleelasticity, and their interaction. It allows us to realistically simulatethe deflections that occur as thin needles travel through inhomo-geneous tissues. A motivation for modeling needle elasticity is anew class of flexible, steerable needles recently developed in col-laboration between researchers at U.C. Berkeley and Johns Hop-
Interactive Simulation of Surgical Needle Insertion and SteeringNuttapong Chentanez
U.C. BerkeleyRon Alterovitz
U.N.C. Chapel HillDaniel RitchieU.C. Berkeley
Lita ChoU.C. Berkeley
Kris K. HauserU.C. Berkeley
Ken GoldbergU.C. Berkeley
Jonathan R. ShewchukU.C. Berkeley
James F. O’BrienU.C. Berkeley
AbstractWe present algorithms for simulating and visualizing the inser-tion and steering of needles through deformable tissues for surgi-cal training and planning. Needle insertion is an essential compo-nent of many clinical procedures such as biopsies, injections, neuro-surgery, and brachytherapy cancer treatment. The success of theseprocedures depends on accurate guidance of the needle tip to a clin-ical target while avoiding vital tissues. Needle insertion deformsbody tissues, making accurate placement di⇥cult. Our interactiveneedle insertion simulator models the coupling between a steerableneedle and deformable tissue. We introduce (1) a novel algorithmfor local remeshing that quickly enforces the conformity of a tetra-hedral mesh to a curvilinear needle path, enabling accurate compu-tation of contact forces, (2) an e⇥cient method for coupling a 3Dfinite element simulation with a 1D inextensible rod with stick-slipfriction, and (3) optimizations that reduce the computation time forphysically based simulations. We can realistically and interactivelysimulate needle insertion into a prostate mesh of 13,375 tetrahedraand 2,763 vertices at a 25 Hz frame rate on an 8-core 3.0 GHz In-tel Xeon PC. The simulation models prostate brachytherapy withneedles of varying sti�ness, steering needles around obstacles, andsupports motion planning for robotic needle insertion. We evalu-ate the accuracy of the simulation by comparing against real-worldexperiments in which flexible, steerable needles were inserted intogel tissue phantoms.Keywords: surgical simulation, needle insertion, real-time finiteelement methods, coupled simulationCR Categories: I.3.5 [Computer Graphics]: ComputationalGeometry and Object Modeling—Physically based modeling;I.3.7 [Computer Graphics]: Three-Dimensional Graphics andRealism—Animation; I.6.8 [Simulation and Modeling]: Types ofSimulation—Animation.
1 IntroductionNeedle insertion is an essential component of many clinical proce-dures such as biopsies, injections, neurosurgery, and brachytherapycancer treatment [Abolhassani et al., 2007]. The success of theseprocedures depends on how close the needle tip is maneuvered tothe target. It is crucial that the needle avoid bone and other criticalstructures and organs [Kohn et al., 2000]. Unfortunately, needle in-sertion deforms body tissues enough that poor accuracy is the normin practice. For example, experienced physicians inserting radioac-tive seeds into the prostate gland for brachytherapy prostate cancer
From the ACM SIGGRAPH 2009 conference proceedings.
Permission to make digital or hard copies of all or part of this work for personalor classroom use is granted without fee provided that copies are not made ordistributed for profit or commercial advantage and that copies bear this noticeand the full citation on the first page. To copy otherwise, to republish, to post onservers or to redistribute to lists, requires prior specific permission and/or a fee.ACM SIGGRAPH 2009, New Orleansc� Copyright ACM 2009
a)
b)
Figure 1: Screenshots from our prostate brachytherapy simulator.A needle is inserted from the left through the epidermis and dermisinto the prostate gland. a) Bevel-tip flexible needle. b) Symmetric-tip sti� needle.
treatment experience average placement errors of 6.3 mm, about15% of the prostate’s diameter [Taschereau et al., 2000].
Computer simulations of needle insertion procedures enable physi-cians and other clinicians to train in a controlled environment thatexposes them to both common and rare patient cases without risksto patient safety. Studies indicate that surgical skills learned us-ing computational simulators directly improve operating room per-formance by significantly decreasing procedure time and reducingthe frequency of medical errors by up to sixfold compared to tradi-tional training [Seymour et al., 2002; Satava, 2005; Gallagher et al.,2005]. Surgical simulations also have uses for pre-operative plan-ning [Alterovitz and Goldberg, 2008; Taylor, 2006].
We present a new simulator that models tissue deformation, needleelasticity, and their interaction. It allows us to realistically simulatethe deflections that occur as thin needles travel through inhomo-geneous tissues. A motivation for modeling needle elasticity is anew class of flexible, steerable needles recently developed in col-laboration between researchers at U.C. Berkeley and Johns Hop-
1
Department of Computer Science University of North Carolina at Chapel Hill March 2012
• Simulating medical procedures enables pre-
operative procedure planning and surgical training in virtual environments.
• Simulating medical procedures is challenging due to tissue deformations and complex tool/tissue interactions.
• We created a real-time, interactive simulation of needle insertion in deformable tissues.
• Simulation of needle insertion in deformable tissue can be used to anticipate and correct for needle placement errors due to tissue deformations.
a 1D elastic rod simulation with stick-slip friction, and (3) several generally applicable optimizations for reducing computation time for physically based simulations. Our remeshing algorithm efficiently relocates and creates nodes so they lie along a curvilinear needle path in a volumetric mesh, enabling the simulation to apply cutting and frictional forces along the needle shaft at mesh nodes while maintaining a high quality tetrahedral mesh for computing tissue deformations. Our optimizations fully exploit parallelization and the sparseness of finite element method meshes.
Our algorithms and enhancements enable us to realistically simulate needle insertion into deformable tissue at interactive frame rates. We achieve frame rates of 25 Hz on an 8-core 3.0 GHz Intel Xeon PC for a prostate mesh of 13,375 tetrahedra and 2,763 vertices. We use realistic material properties for human tissue, making it more challenging than the more compliant materials for which real-time performance is usually reported. As shown on the previous page, we applied our simulators to prostate brachytherapy with needles of different stiffness.
We are also developing a planning system for prostate brachytherapy cancer treatment. In this procedure, physicians use needles to implant radioactive seeds in the prostate. We combine our simulation of needle insertion with numerical optimization to compute needle insertion offsets that compensate for tissue deformations. We applied the method using 2D simulation to seed implantation during prostate brachytherapy to minimize seed placement error in simulation.
Current Project Members Ron Alterovitz (Principal Investigator), Assistant
Professor Sachin Patil, Graduate Research Assistant Research Sponsors National Science Foundation (NSF) under award IIS-0905344 on “Robust Intelligent Manipulation and Apprenticeship Learning for Robotic Surgical Assistants.”
Selected Publications Kyle B. Reed, Ann Majewicz, Vinutha Kallem, Ron Alterovitz, Ken Goldberg, Noah J. Cowan, Allison M. Okamura, “Robot-Assisted Needle Steering,” IEEE Robotics and Automation Magazine, vol. 18, pp. 35-46, Dec. 2011.
N. Chentanez, R. Alterovitz, D. Ritchie, J. Cho, K. Hauser, K. Goldberg, J. R. Shewchuk, and J. F. O’Brien, “Interactive simulation of surgical needle insertion and steering,” ACM Transactions on Graphics (Proc. SIGGRAPH), vol. 28, pp. 88:1–88:10, Aug. 2009.
R. Alterovitz, K. Y. Goldberg, J. Pouliot, and I.-C. Hsu, “Sensorless motion planning for medical needle insertion in deformable tissues,” IEEE Trans. Information Technology in Biomedicine, vol. 13, pp. 217–225, Mar. 2009.
R. Alterovitz and K. Goldberg, Motion Planning in Medicine: Optimization and Simulation Algorithms for Image-Guided Procedures, vol. 50 of Springer Tracts in Advanced Robotics (STAR). Berlin, Germany: Springer, July 2008.
K. Hauser, R. Alterovitz, N. Chentanez, A. Okamura, and K. Goldberg, “Feedback control for steering needles through 3D deformable tissue using helical paths,” in Proc. Robotics: Science and Systems, June 2009.
S. Patil and R. Alterovitz, “Toward automated tissue retraction in robot-assisted surgery,” in Proc. IEEE Int. Conf. Robotics and Automation (ICRA), May 2010.
M. Torabi, K. Hauser, R. Alterovitz, V. Duindam, and K. Goldberg, “Guiding medical needles using single-point tissue manipulation,” in Proc. IEEE Int. Conf. Robotics and Automation (ICRA), pp. 2705–2710, May 2009. Best Medical Robotics Paper Award Finalist
