Deliverable 5.5 RASimAs Revised...

FP7-ICT-2013-10 – 5.5 – RASimAs Revised Specification

Project No.

610425 Deliverable Report

D5.5, 31/01/2016, Revision: Final Version

Page 1 of 20

RASimAs_D5.5_RASimAs Revised Specification_Final2 16/02/2016

Deliverable 5.5

RASimAs Revised Specification

Dissemination

Level Type Delivery Month

Confidential (CO)

Restricted (RE)

Public (PU)

Report (R)

Prototype (P)

Other (O)

27

Deliverable D5.5

Milestone not applicable

Work Package

Leader SINTEF

Task/Deliverable Leader SG + SINTEF

Deliverable Due

Date 31.01.2016

Date of Submission 01.02.2016

Version 1.0

Keywords Simulator; assistant

Internal Report Review Done by management body


Project No.



Page 2 of 20


Version Control

Version Date Author (Name, Institution) Comments

1.0 30.01.2016 Hassan Syed - SG Initial version, RASim

2.0 30.01.2016 Frank Lindseth – SINTEF Initial version, RAAs based on

D5.3

3.0 01.02.2016 Julia Oliveira – UKA-IMI Format

3.1 11.02.2016 Thomas Deserno – UKA-IMI Final check

3.2 15.02.2016 Frank Lindseth – SINTEF Final version

1.X = 1st version circulating between the members / 2.X = 2nd version following comments of members

/ 3.X = 3rd final version


Project No.



Page 3 of 20


Table of Contents

1 ABSTRACT .......................................................................................................................... 6

2 INTRODUCTION ................................................................................................................... 6

2.1 Context ....................................................................................................... 6

2.1.1 SIMULATOR ............................................................................................... 6

2.1.2 ASSISTANT ................................................................................................ 7

2.2 Objectives ................................................................................................... 8

2.2.1 Deliverable description ............................................................................... 8

3 RASIM SPECIFICATIONS ..................................................................................................... 8

3.1 Modular View of Simulator .......................................................................... 8

3.2 Client Server Architecture ........................................................................... 9

3.3 Simulator Core System Flow .................................................................... 10

3.4 H3D Module .............................................................................................. 11

3.5 SOFA Module ........................................................................................... 12

3.6 Ultrasound Simulation Module .................................................................. 13

3.7 Course-ware Module ................................................................................ 14

3.8 Ultrasound-guided Framework ................................................................. 15

3.9 Electrical Nerve Stimulator-guided Framework ......................................... 15

3.10 RASim Implementation ............................................................................. 16

3.10.1 Hardware Specification ............................................................................. 16

3.10.2 Software Implementation .......................................................................... 19

3.11 Deviations/Problems ................................................................................. 19

4 RAAS SPECIFICATIONS ..................................................................................................... 20

4.1 Deviations/Problems ................................................................................. 20

5 FURTHER WORK................................................................................................................ 20

6 PUBLICATION/DISSEMINATIONS ......................................................................................... 20

7 REFERENCE DOCUMENTS ................................................................................................. 20


Project No.



Page 4 of 20


List of Figures

Figure 1. Simulator training environment system overview. ..................................................... 7

Figure 2. Modular diagram for proposed training environment. ............................................... 9

Figure 3. Simulator training environment client server architecture. ...................................... 10

Figure 4. H3D core simulation based system flow diagram. .................................................. 11

Figure 5. H3D core module communication with SOFA and US module. .............................. 12

Figure 6. SOFA module communication via H3D with other modules. .................................. 13

Figure 7. Ultrasound module communication with H3D module and others via H3D. ............ 14

Figure 8. Course-ware module communication with H3D API and simulator. ........................ 15

Figure 9. Workbench computer for Simulator core. ................................................................ 16

Figure 10. Prototype view (Input and output devices) and base platform. ............................. 17

Figure 11. A basin shaped leg placed in middle of the base platform to augment the Leg

Femoral block area. The basin offers replaceable oasis foam in the middle section. ..... 17

Figure 12. The needle (Braun 50 mm) is attached in front of to the haptics stylus allowing for

movement in x, y, z, direction, pitch, yaw and limited rotation. ....................................... 18

Figure 13. 3D printed copy of a real probe with sensor embedding location for the Ascension

tracker solution. ............................................................................................................... 19

Figure 14. USB based rotating knob for electrical stimulation impulse control. ..................... 19


Project No.



Page 5 of 20


Acronyms

Courseware: An application to maintain/track the user training scenarios and profiles and

serves as primary user interface.

EN: electrical nerve stimulation.

H3D: H3DAPI is an open source haptics software development platform that uses the open

standards OpenGL and X3D with haptics in one unified scene graph to handle both graphics

and haptics. H3DAPI is cross platform and haptic device independent. It enables audio

integration as well as stereography on supported displays. (http://www.h3dapi.org)

RAAs: Regional Anaesthesia Assistant system

RASim: Simulator prototype in the RASimAs project.

RASimAs: Regional Anaesthesia Simulator and Assistant.

RA: Regional Anaesthesia.

SOFA: Simulation open framework architecture, SOFA is an Open Source framework

primarily targeted at real-time simulation, with an emphasis on medical simulation.

(http://www.sofa-framework.org)

US Images: Ultrasound images.

US Module: Ultrasound guided needle insertion simulation module providing virtual

ultrasound images for needle insertion assistance.

Web services: A Web service is a method of communication between two electronic devices

over a network. It is a software function provided at a network address over the web

interface.


Project No.



Page 6 of 20


1 Abstract

This document (Deliverable 5.5) aims at presenting the RASimAs Simulator and Assistant

specification. Regarding the simulator, the revised specifications were based on the

adjustments on previously defined specifications Deliverable 5.1 (RASim specifications). We

came up with various hardware solutions for the RASim prototype and according to the

clinicians suggestions, we focused on the optimal positioning of haptic device, improvements

for the needle attachment tool, mannequin redesign and portable base platform setup.

Hence, this document defines the revisions in the software architecture and hardware

platform design leading to the improvement in RASim prototype. With respect to the

assistant, this document is similar to Deliverable 5.3 (RAAs Specification).

2 Introduction

2.1 Context

2.1.1 SIMULATOR

This document aims at presenting the RASimAs simulator specification which is an extension

to the deliverable 5.1 (specifications of RASim). The purpose for this report is to layout the

revisions and improvements done in the specifications of simulator and it modules used in

development of the platform. This document further defines the change/revision in the

module integration, interfaces and communication layers between the modules to be

developed in the RASim platform.

The system has been proposed by experts to replicate the basic regional anesthesia

practices. Figure 1 shows the RASimAs training environment overview, also referred to as

RASim (for Regional Anesthesia Simulator). The user interacts with the system through a

work bench client computer. The system consists of three components, namely the input

interface, the output interface, and the core simulation software. The input interface primarily

connects the input devices, such as the haptic devices, the 3d tracker, the rotating knobs etc.

The output interface is responsible for showing updated views to its users.


Project No.



Page 7 of 20


Figure 1. Simulator training environment system overview.

The output views are divided in three views, the first is patient view that shows the scene

being rendered, including the virtual patient and the needle. Second is the ultrasound view

showing the update for ultrasound images while moving the ultrasound tracker probe. The

third view is the course-ware view, which holds the users data and details such as available

scenarios, profile records, and server updates (manage curriculum/training lessons, replay

lessons, scoring). The system back-end (core) is the simulator program. Further details are

given in Section 2 (modular view) and section 4 simulator core.

2.1.2 ASSISTANT

In the RASimAs project, a RAAs system for assisting the physician in performing ultrasound-

guided RA procedures has been developed. In this document, an in-depth specification of

the Assistant is provided. The Assistant is a single-rack system consisting of an ultrasound

scanner with embedded tracking of both the probe and needle, as well as a high-end RAAs

computer. The Assistant software provides fully automatic real-time functionality for placing

the probe correctly relative to the anatomy and interpreting the US images. It is addressing

the main challenges in ultrasound-guided RA as stated by the clinical partners in the project.

By finding an optimal ultrasound view and identifying the target nerve and key landmarks,

like arteries in real-time 2D US images, the Assistant is aiding the operator to overcome

these challenges. This is further enhanced by a 3D view showing 3D reconstructions of

important structures like the artery generated from ultrasound data and a generic model of

the surrounding anatomy that automatically snaps into place for anatomical reference when

enough ultrasound data has been acquired.


Project No.



Page 8 of 20


2.2 Objectives

2.2.1 Deliverable description

As stated in the Description of Work, the deliverable that constitutes this plan is described as

follows:

D5.5 RASimAs Revised Specification

Adjustments on previously defined specifications coming from the clinical

evaluation and leading to improvements of the prototype.

3 RASim Specifications

3.1 Modular View of Simulator

Figure 2 sketches the modular architecture of the simulator environment. H3D viewer serves

as core API to communicate with additional modules and metadata exchange. H3D as

simulator core running on CPU level is responsible for metadata exchange between the other

modules. It simulates information such as scene being rendered, the needle position, the

probe position, the knob status and haptic feedbacks. This simulation rendering is mainly

done in collaboration with the SOFA module for soft muscular deformations, the US module

for simulation of ultrasound guided images, the courseware module for keeping track for the

user metrics (profile and scenarios) and the electrical nerve stimulator module to stimulate

electrical impulse guided needle insertions.


Project No.



Page 9 of 20


Figure 2. Modular diagram for proposed training environment.

In later sections of this document revised version of every module is defined in further

details. No such deviation or any significant revision was done for this part (i.e. the design

follows the specification presented in Deliverable D5.1). An additional interface has been

provided for courseware and Ultrasound module to exchange metadata. This interface has

boosted the simulation performance with real time ultrasound displays. This interface is

shown in the above figure.

3.2 Client Server Architecture

The interaction of the simulator (training environment) with the server is described in the

system architecture document for RASimAs (Deliverable D2.2, “System Architecture”). The

communication between the server and training environment is based on web-services

(Figure 3). A local copy of server side database (available x3d models) in the training

environment is maintained to help the system run in offline mode (when network services are

not accessible). The training system is responsible to connect with the remote server

database and download updates and models required by consuming web-services.

No such deviation or any significant revision was done for this part (i.e. the design follows the

specification presented in Deliverable D5.1). However, cloud based synchronizing service

has been added as an addition feature to the web services hosted by the remote server. This


Project No.



Page 10 of 20


feature allows all the training environment to sync simultaneously with the services being

offered by the server.

Figure 3. Simulator training environment client server architecture.

3.3 Simulator Core System Flow

As depicted in Figure 4 a minor change has been suggested in the core simulation flow

environment compared to the first specification of RASim. An interface has been introduced

in between the ultrasound module and courseware to communicate through TCP/IP.

Courseware is the module responsible for displaying the ultrasound images therefore to

improve the performance of the system the courseware has been given direct access to the

ultrasound module Images/metadata.


Project No.



Page 11 of 20


Figure 4. H3D core simulation based system flow diagram.

3.4 H3D Module

The H3D API controls the simulator core. It is responsible for the interaction between the

other functions modules (SOFA, US, Course-ware). Initially the user interacts with the

course-ware module user interface to select user desired scenarios and proceeds with

loading process. The H3D module depicted in Figure 5, loads the selected scenario in 3D

simulation environment and proceeds with the initialization of the haptics force feedback

controls, the connections and data exchange between the SOFA module for soft body

simulations/deformations and initialization of 3D tracker device. No such deviation or any

significant revision was done for this part (i.e. the design follows the specification presented

in deliverable 5.1).


Project No.



Page 12 of 20


Figure 5. H3D core module communication with SOFA and US module.

3.5 SOFA Module

In the RASim platform SOFA handles all the mechanical / soft body deformations. Its main

tasks consists in computing and managing:

The deformation of the skin and other tissue layers, using Co-rotational tetrahedral

Finite Element Methods,

The needle manipulation on the different tissue types, with skin penetration and

tissue friction,

The physiological response of the nerves

The muscles contraction, depending on the physiological response and soft tissue

properties,

The user interactions and haptic force feedback.

Multiple parameters can be customized such as the stiffness of the muscles, the

characteristics of the needle, etc.

Figure 6 outlines the interaction between SOFA and H3D core, the control of the needle is

managed by H3D which can support various haptic devices such as the Sensable Omni

currently used for testing purpose. SOFA is heavily based on a mapping mechanism, where

mechanical, visual and collision models are different but linked together in one mechanism.


specification presented in deliverable 5.1).


Project No.



Page 13 of 20


Figure 6. SOFA module communication via H3D with other modules.

3.6 Ultrasound Simulation Module

The ultrasound (US) simulation module is the primary component to generate the virtual US

images needed for the training of ultrasound guided needle insertion procedure. The

ultrasound generator module utilizes a fast ray tracing based approach to simulate US

interactively. The process is divided in two stages: a configuration step that is executed just

once and the simulation step, which will be executed for each frame.

During the configuration step, the US module receives meta-data from the H3D module and

loads simulation parameters, tissue properties and the anatomy models from disk. These are

then processed to setup and fill according tissue textures and data structures needed during

the simulation process.

In each simulation step, the actual position and orientation of the virtual probe and the needle

as well as the current deformation state of the anatomy in form of a tetrahedral mesh (result

of the SOFA module), are received from the H3D module.

The position and orientation of the virtual US probe determines the origin and direction of the

rays that are traced through the scene under consideration of the deformation state. As

illustrated in Figure 7, the US module communicates with the H3D module using prefixed

H3D interface.

Since the Ultrasound generation module is based in GPU computing and our simulation

virtual patient view is running on CPU. An additional interface based on TCPIP

(subscriber/receiver) communication has been implemented to let the courseware directly


Project No.



Page 14 of 20


subscribe with the US module and receive the ultrasound generated images. The main

purpose of this interface is to boost the simulation performance making it real time.

Figure 7. Ultrasound module communication with H3D module and others via H3D.

3.7 Course-ware Module

The course-ware module is the main component that renders the Ultrasound view as well as

records the metrics and send it to the server side database. Initially, the user is required to

log into the course-ware module and receive the statistics/profile data from server. The data

will include the user training hours and metrics. Based on the user level and metrics, course-

ware module will provide available training scenarios for users to load and practice number

of hours on a given protocol. The course-ware module is designed to connect with the H3D

as core module and load meta-data for selected scenario through server storage. This

module performs the user authentication process by contacting the User management

subsystem of the RASim platform (deliverable document 2.2). This subsystem is responsible

for the storage of the users’ credentials information and users profiles.

The system flow diagram for the Courseware is shown in Figure 8, other than given an

interface to communicate directly with ultrasound module for receiving raw image data

(giving a boost to the real-time ultrasound simulations view) no such deviation or any

significant revision was done for this part (i.e. the design follows the specification presented

in deliverable 5.1).


Project No.



Page 15 of 20


Figure 8. Course-ware module communication with H3D API and simulator.

3.8 Ultrasound-guided Framework

The ultrasound guided regional anesthesia procedure has been center of attention in RASim

development. This module has been development as most important procedure in simulation

environment.

Other than a few changes in modules integration, no such deviation or any significant

revision was done for this part (i.e. the design follows the specification presented in

deliverable 5.1).

3.9 Electrical Nerve Stimulator-guided Framework

Regional Anesthesia (RA) is a challenging medical task whose success relies on accurate

localization of peripheral nerve proximity. Often, electrical stimulation is used to guide a

needle in localizing the nerves. To allow medical staff practice this difficult procedure, RASim

involves simulator build with electrical nerve stimulator (EN) module. This modules performs

real-time simulation that emulates the physiological behaviors during the electrical

stimulation.


specification presented in deliverable 5.1).


Project No.



Page 16 of 20


3.10 RASim Implementation

The interaction of the RASim (training environment) with the server is described in section 3.

To fulfill the simulation requirements, following are the revised specifications of hardware and

software implementation:

3.10.1 Hardware Specification

a) Computer

A machine is to host the RASimAs development platform (client machine). Following are the

minimum requirements for the machine.

The following are the minimum hardware and operating system requirements for the training

environment (Figure 9):

Windows-7 / Linux platform.

Tested with GTX980 graphics card (CUDA capability 3.0, 3D vision support).

Minimum Intel i7 core processor.

500GB, hard disk space.

8GB Ram.

Figure 9. Workbench computer for Simulator core.

b) Platform

The base platform involves positioning of the haptics device and the mannequin relative to

each other with an ease to perform the regional anesthesia procedure. To serve this

purpose, the prototype includes 3D magnetic tracker (Transmitter) solution to work with

ultrasound probe. A plain wooden plate used as base for placing the devices on it.

The haptic device by default placed on right corner for right handed persons. However, the

device can be moved on left hand side for left hand users. The middle of base platform is

portion for mannequin imitating the leg portion of femoral block area (Figure 10).


Project No.



Page 17 of 20


Figure 10. Prototype view (Input and output devices) and base platform.

c) Mannequin

To represent the patient area of needle insertion having capabilities to simulate real human

tissue and to further hold the needle in position during needle insertion simulation. The base

platform was designed keeping in view the support for both left and right handed procedures.

The middle part of the base platform has been developed to imitate the femoral block area of

human Leg. Keeping in view that the needle insertion procedures might need replacement of

mannequin after number of needle pops, RASim base platform offers portability and replace

ability for oasis foams (low cost) after few hundred needle insertions. The new base platform

(mannequin) is shows in Figure 11.

Figure 11. A basin shaped leg placed in middle of the base platform to augment the

Leg Femoral block area. The basin offers replaceable oasis foam in the middle

section.

d) Views (Monitor display)

To display H3DAPI simulation view, Patient, US view, Course-ware view.

Standard/3D computer monitor is proposed as hardware solution.

No such deviation or revision was done for the display units.

e) Haptic Device


Project No.



Page 18 of 20


Initial tests were carried out by the experts based on the specification in deliverable 5.1 using

haptic device with needle mounted/attached on the side. It was discovered that the needle

insertion procedure was not user friendly, the haptic stylus was interfering with the ultrasound

probe, as well as it was not easy to hold the needle (confusing with stylus) since most of test

users were holding the stylus pen instead of the needle tool.

Keeping in view the above reported issues a new design requirement was presented to

overcome these issues. The new requirements demanded that needle be attached at the tip

of the stylus to make it easy to insert under the plane of the ultrasound probe and not letting

it interfere the probe. Based on the design requirement an upgrade to the needle tool was

done as shown in Figure 12.

f) Ultrasound Probe

To simulate the probe and the probe interaction during RA procedure. A 3D printed probe

embedded with magnetic tracked sensor has been designed for the RASim prototype. The

Ascencion 3D-Guidance trakSTAR magnetic tracker (model 800 http://www.ascension-

tech.com/medical/trakSTAR.php) has been placed inside the probe giving 6 DOF freedom to

move probe as done in realistic environments shown in Figure 13.

Figure 12. The needle (Braun 50 mm) is attached in front of to the haptics stylus

allowing for movement in x, y, z, direction, pitch, yaw and limited rotation.


Project No.



Page 19 of 20


Figure 13. 3D printed copy of a real probe with sensor embedding location for the

Ascension tracker solution.

g) Electric Knob

To simulate and control the electrical nerve stimulation a simple rotating usb based knob is

used for RASim prototype which can be seen in Figure 14. No such deviation or revision was

done for the knob.

Figure 14. USB based rotating knob for electrical stimulation impulse control.

3.10.2 Software Implementation

Mostly the software implementation/integration has been done according the

specifications given in deliverable 5.1 and 5.2. No significant changes were made in the

implementation/integration procedure.

3.11 Deviations/Problems

The RASim prototype has passed the implementation stage and is being tested at this stage.

No such deviation or problems reported at this stage.


Project No.



Page 20 of 20


4 RAAs Specifications

Please see Deliverable D5.3 (RAAs Specification) for information on the requirement

specifications of the RAAs based on the guidance system specifications and user needs.

4.1 Deviations/Problems

No significant deviations from original plan. Further details regarding the main challenges

related to the assistant can be found in MS6 document (part 4).

5 Further work

Some of the future work that we would like to address is as follows:

Assemble the two new clinical assistants into one-rack systems (will be done according

to plan).

Support the clinical use of the systems (e.g. symposium in Leuven).

Improve the system based on clinical feedback and additional data (if available).

Improve needle guidance and address anesthetics spread verification (if remaining

resources and data from the whole procedure are available).

Additional publications and disseminations related to the assistant.

Address other nerve blocks (in addition to femoral nerve blocks) (if resources and relevant

data are available).

6 Publication/Disseminations

Several publications, proceedings, abstracts, posters, presentations, popular science articles

and videos related to the assistant already exist. See the MS6 document for a more detailed

list of all the contributions that have been accepted.

7 Reference Documents

Deliverable 4.5 – Guidance Systems Specification

Deliverable 5.4 – RAAs Prototypes (with “user manual” report.

Deliverable 8.4 and 8.5 – Exploitation plan 1 and 2 for the assistant.

Milestone 6 - RAAs function available on the portable prototypes.

RAAs system introduction video: https://www.youtube.com/watch?v=ptRF6dv43HA

https://www.youtube.com/watch?v=ptRF6dv43HA

Date post:	31-Jul-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times