Real-Time Graphic and Haptic Simulation of Deformable Tissue Puncture

A myriad of surgical tasks rely on puncturing tissue membranes (Fig. 1) and cutting through tissue mass. Properly training a practitioner for such tasks requires a simulator that can display both the graphical changes and the haptic forces of these deformations, punctures, and cutting actions. This paper documents our work to create a simulator that can model these effects in real time. Generating graphic and haptic output necessitates the use of a predictive model to track the tissue’s physical state. Many finite element methods (FEM) exist for computing tissue deformation ([1],[4]). These methods often obtain accurate results, but they can be computationally intensive for complex models. Real-time tasks using this approach are often limited in their complexity and workspace domain due to the large computational overhead of FEM. The computer graphics community has developed a large range of methods for modeling deformable media [5], often trading complete physical accuracy for computational speedup. Casson and Laugier [3] outline a mass-spring mesh model based on these principles, but they do not explore its usage with haptic interaction. Gerovich et al. [2] detail a set of haptic interaction rules (Fig. 2) for one dimensional simulation of multi-layer deformable tissue, but they do not provide strategies for integrating this model with realistic graphic feedback. Disciplines Computer Sciences | Engineering | Graphics and Human Computer Interfaces This journal article is available at ScholarlyCommons: http://repository.upenn.edu/hms/203 Real-Time Graphic and Haptic Simulation of Deformable Tissue Puncture Joseph M. Romano, Alla Safonova, and Katherine J. Kuchenbecker University of Pennsylvania, GRASP Laboratory and HMS Center {jrom, alla, kuchenbe}@seas.upenn.edu Background A myriad of surgical tasks rely on puncturing tissue membranes (Fig. 1) and cutting through tissue mass. Properly training a practitioner for such tasks requires a simulator that can display both the graphical changes and the haptic forces of these deformations, punctures, and cutting actions. This paper documents our work to create a simulator that can model these effects in real time. Generating graphic and haptic output necessitates the use of a predictive model to track the tissue’s physical state. Many finite element methods (FEM) exist for computing tissue deformation ([1],[4]). These methods often obtain accurate results, but they can be computationally intensive for complex models. Real-time tasks using this approach are often limited in their complexity and workspace domain due to the large computational overhead of FEM. The computer graphics community has developed a large range of methods for modeling deformable media [5], often trading complete physical accuracy for computational speedup. Casson and Laugier [3] outline a mass-spring mesh model based on these principles, but they do not explore its usage with haptic interaction. Gerovich et al. [2] detail a set of haptic interaction rules (Fig. 2) for onedimensional simulation of multi-layer deformable tissue, but they do not provide strategies for integrating this model with realistic graphic feedback. Tools and Methods Our simulation uses mass-spring meshes similar to those developed in [3] and implements the force rules for contact and puncture from [2]. We create three planar meshes, one for each tissue layer, which can then be deformed and punctured. The graphical output of our model is displayed to the user via an OpenGL window (Fig. 3). The user interacts with the mesh models via a Phantom Omni haptic device (Fig. 4); its motion drives the movement of the virtual needle, and its motors display contact forces to the user. The current implementation keeps the needle vertical at all times, but future versions will allow contact at any angle. Our software program performs its three main tasks (mesh update, graphic output, and haptic output) at different rates to achieve optimal performance for the user (Fig. 5). One thread updates the motion of the mesh models as quickly as possible, using a fourth-order Runge-Kutta integration algorithm. A simulation with 1875 total nodes was able to run at approximately 55 Hz on a modern desktop computer. This computational rate easily enables smooth graphical output at 30 Hz. In order to prevent discrete changes in force from being noticed by the user, haptic devices must update their force output at a rate exceeding human sensing capabilities (~600 Hz). Since our mesh model updates at only ~55 Hz, the 1 kHz haptic thread treats the mesh node positions as static between model updates. Using this approach we can provide smooth haptic feedback even though calculating the actual mesh deformation at such rates is not possible. The program computes the haptic output force by summing the simple positionand velocity-based terms from Fig. 2 with the net normal force acting on the closest mesh node, smoothed via a low-pass filter. Between mesh updates, the forces generated from the Fig. 2 rules are summed and then applied together to the appropriate nodes during the mesh's next RK4 integration step. A layer is punctured if the contact node experiences a net normal force beyond a critical value. Results The application of damping forces to the mesh model results in graphically pleasing effects at the tissue surfaces. The surface appears to ‘stick’ to the needle as it is inserted and retracted from the model, and a haptically realistic viscous damping sensation is created. Datasets of the user’s position and applied force were recorded for several interactions with our simulation (Fig. 6). Distinct ‘popping’ actions can be felt when tissue layers are broken, as expected when using the rules seen in Fig. 2. Our simulation was executed on several different computing platforms in order to test the combined effect of the haptic and graphic interaction on commonly available computing resources, as seen in Fig. 7. Conclusion Our new simulation techniques are suitable for immediate implementation on available computing hardware. The reaction of the model is similar to physical needle puncture data taken by other groups such as [2]. We believe that our method of rendering haptic and graphical output from a mass-spring model can enable today’s simulations to perform the real-time dynamic reactions that they are lacking. Although we have not yet performed a formal user study, we believe this enhanced output will be of considerable use to the simulation community.