Multi-Modality Communication Systems for Robotic Telesurgery
Syed Saqib Hussain Shah 1 and Jackrit Suthakorn
2
1, 2 Center for Biomedical and Robotics Technology (BART LAB), Department of Biomedical Engineering,
Faculty of Engineering, Mahidol University.
Abstract. Remote surgery combines elements of robotics, cutting edge communication technology such as
high-speed data connections and elements of management information systems. Several network and systems
option provide different Quality of Service (QoS) depending on several limitations and difficulties, such as,
transmission delays, bandwidth constraint, and connection rupture packet lost. In this paper an attempt has
been made to configure the best possible mode of communication to mock-up system which provides faster
data transmission with minimum time latency. Here, a concept of tele-surgery via real-time multi-modality
telecommunication networks and systems is presented. To test the effectivity of the proposed systems LAN,
WAN, the 3rd generation network and broad band communication system while short distance and long
distance internet communications test were examined, as result longer distance face little discomfort whereas
short distance does not have any such issues. The survey was conducted in consultation with
anesthesiologists and surgeons at Ramathibodi hospital, Thailand.
Keywords: Telesurgery, Latency, Robotics assisted surgery, Telecommunication surgery, Internet based
surgery.
1. Introduction
In today’s globalised technological scenario one is able to get the whole world under one’s grip by a
single click of the mouse. Invention of computers has made everything possible. Today we find its use in
almost all the fields changing the humans’ life style whether it at work force or personal. One such area in
the official is the field of medicine. Technology has evolved the communication from mechanical to
electronic broadcast using optic wires, radio signals and the internet making the world into a close knit
family. Tele-surgery or Remote surgery is the performance of robotics over a distance with a communication
link enabling the extension of the surgeon’s surgical expertise to rural area community cutting down the
transport expenses. The telesurgery conducted is a composition of a master and slave where the master site is
operated by human and slave site follows it. Zeus Robotic system which consisted of two
telecommunications computers are the first one to be introduced in this area where each computer is
connected to local site. The advantages of this system was that it had a robust error checking and fail safe
performance of standard Zeus even while operating in a tele-operative mode. Further it could be easily
switched into a tele-surgical Zeus-TC [1].
The first remote human telesurgery was successfully accomplished by operation Lindberg where the
operation surgeon was shifted to a distant location from the surgical room to perform a minimally invasive
surgery on a patient. The integrated telecommunication system enabled the surgeons from the master site to
collaborate with assistant surgeons from slave site. Telephones were used as voice communication, fax
machines as enable written communication, video call enabling visual interaction and robotics enabling
physical interaction over a distance [2]. The first telesurgery was performed by the National Aeronautics and
Space Administration in 1972 and since then we could witness a lot of development. M.Anvari.et.al. [3]
Corresponding author. Tel.: +66-2-889-2138 Ext 6446; fax: +662-441-4254.
E-mail address: [email protected].
International Proceedings of Chemical, Biological and Environmental Engineering, Vol. 89 (2015)
DOI: 10.7763/IPCBEE. 2015. V89. 12
72
explains the importance of communication for tele-surgery using commercial fiber optics. Tele-surgery using
research based laparoscopic robot was developed and brought to application by S.S.Sastry.et.al. [4]. .
Expert Site/Master Site Surgical Site/Slave Site
Communication
Fig. 1: BART Lab Telesurgical system
Because of telesurgery’s limitations such as absence of haptic information from the target area, use of
two dimensional video images by surgeons to observe patient and time delay on telecommunication and lack
of quality service due to internet to establish robotic technology as a standard treatment becomes difficult.
Hence Communication plays a very important role here [5]. Though it has many advantages it has
disadvantages too. Transmitting time delays, bandwidth constraint and connection rupture packet loss are its
major challenges. This paper discusses the advantages in general and the major challenges in specific and
tries to overcome the challenges and provide a solution for it. Various studies have shown that data loss is a
main challenge to determine the quality of service in time latency sensitive multimedia application is quite
challenging [5]. Understanding that the internet is the best way for transmitting data between human system
interface and the remote teleoperation [6] at the same time determining the quality of service in time latency
sensitive multimedia application is quite challenging an attempt has been made to configure the best possible
mode of communication to mock-up system which provides faster data transmission with minimum time
latency. Based on BART Lab Telesurgical system as shown in Fig. 1 [7] a setup of a mock up master and
slave system was established [8], [9]. Initially a pick and pace task was conducted. Two commercial robots
namely Phantom Omni and Motoman HP3NX 100 were connected to evaluate the multi-modality
communication system. To test the effectivity of the multi-modality communication systems LAN,
WAN ,the 3rd generation network and broad band communication system while short distance and long
distance internet communications were examined.
This paper is organized as follows. In Section 2, the workspace analysis were sensible phantom Omini
workspace and motorman HPS workspace were explained. Further, in this section, the control algorithm for
the position control is also explained. Then, the effect of multimodality system was analysed experimentally
and the results are discussed in Section 3 and 4, this is the main contribution of this paper. Finally, this paper
is summarized in Section 5.
z
y
xϴ4
{BASE}
z
y
xϴ1
z
y
xϴ2
z
y
xϴ3
z
y
xϴ5
z
y
xϴ6
{EE}
H1
'1
H2
'2
H3
'3
H4
'4
H5
'5
H6
'6
Fig. 2: Phantom Omni haptic device Fig. 3: Phantom Omni MATLAB workspace
2. System Modelling Workspace Analysis
73
2.1. Sensable Phantom Omni Workspace
For the Tele-surgery system, sensible phantom Omni was used as the master site surgical console. The
workspace and frame relation between each joints of Omni are shown in Fig. 2 and Fig. 3.
Fig. 3 Show the relationship between homogeneous transformation matrix of HEEBASE . It is computed to
show the position and orientation of the end effector with respect to the base frame as (1).
THTHTHTHTHTHHHEEBASE
1
1
1
1
1
1
1
1
1
1
1
167
6
6
56
5
5
45
4
4
34
3
3
23
2
2
12
1
1
17 (1)
2.2. Motoman HP 3 Workspace
For our experiment, we used Motoman HP3RX 100 as a surgical Robot. Fig. 4. Shows the relationship
of frame of motions for each joint and the workspace measurements were well explained in Fig. 5. The
equation of motion for the Motoman HP 3 is shown in equation (1). It shows the position and orientation of
the end effector with respect to the base frame. Fig. 6 shows the mock-up system for teleoperation using
haptic device as master and motoman HP 3 manipulator as slave.
x
Y
z
{EE}
ϴ3
ϴ1
ϴ2
ϴ4
ϴ5
ϴ6
L1
L2
L3L4
L5
{BASE}
x
Y
z
x
Y
z
x
Y
z
x
Y
z
x
Y
z
H1
'1
H2
'2
H3
'3
H4
'4
H5
'5
H6
'6
Fig. 4: Kinametics of Motoman HP3 RX 100 Fig. 5: Motoman HP3 RX 100 workspace
Fig. 6: Mock up system for Tele-Communication
2.3. Workspace Comparison with Control Algorithm
For comparison between Motoman and Omni workspace, we used position to position command
algorithm, as shown in Table 1.
Table 1: Comparison of Motoman and Omni workspace (Numbers show in degree) Degree of
Freedom
Omni Motoman
#1st DOF ±55 ±170
#2nd DOF +3/+104 -45/+150
#3rd DOF +49/+103 -142/+210
#4th DOF ±138 ±190
#5th DOF ± 65 ±125
#6th DOF ± 146 ± 360
In the experiment, author have used position to position command equation, as shown in (2)
74
Here, master robot workspace position is given by [Ym Zm Xm] where, K is the diagonal matrix (Ky KZ Kx)
of constant scaling terms, and [XS YS ZS] is the slave robot work space position. For Tele-operation system,
slave robot follows the master robot, which is described by the equation (3).
kuP (3)
where P is the Position Vector [XS YS ZS]T; k is position vector of K matrix and u is position vector of [YS ZS
XS]T
3. Experimental Setup
Multi-modality Communication Test was conducted to show the importance of modality and how it
would affect the Transfer of data during telesurgey and the experimental platform which is shown in Fig. 7.
Motoman NX 100 arm robot was used as a slave located in where Mahidol under the control of the Master
robot located in Ramathibodi Hospital. A pick and place task test was performed in order to check the
precision and accuracy of the work done by robots where surgeons were asked to move the five colour
blocks .i.e., red, black and white bocks from position A to position B and then again set it back to its original
position as shown in Fig.8.
This was designed as to hone the ability of the surgeon to understand motor movements in two
dimensions .The data was sent as surgical data expert surgeon visual data and voice communication from the
master site while the slave site included surgical robot data, endoscope camera, patient and room over view,
voice communication date and anesthesiologist data as shown in Fig. 9. As the performance and the success
of telesurgery depends on the well classified communication. LAN, WAN, Third generation Network and
broadband Internet communication systems a multi -modality communication test was conducted as
explained in Fig. 10.
The master site was set up at Ramathibodi Hospital, Dept. of Neurology and 24 km away from the slave
site in the BART Lab, Faculty of Engineering, Mahidol University, Salaya. Testing multi-modality
communication system is one of most important factor, for this method where we can find out each modality
bandwidth and latency on each modality, while understanding each modality bandwidth and latency on the
system, while performing surgery author get tested CISCO VPN Switcher, which apply two different
communication system on network, while one communication is fail to perform CISCO switch from backup
line to not let it fail the system, switching one line to another this CISCO required between 2-5 second, Fig.
11. shows the backup system for both expert site and surgical site.
Master Site Slave Site
Multi-Modality
Communication
Fig. 7: Experimental Platform
4. Results and Discussion
Five modality tests were done on the same systems where WIRESHARK the commercial network
analyzer software was used to capture network ability while performing telesurgey shown in Fig. 12. Each
data was tested 5 times between 10-30 minutes. Significant difference between each modality revealed. i.e.
between the LAN and Wireless LAN setup where master and slave sites are connected to each other in the
same network, Wireless LAN found an average delay of 5ms which is a smaller difference than the LAN
network. Both networks experience unnoticeable delay while controlling the robot. While using the other 3
modalities the results reveal much higher latency in each network. The five modalities LAN, Wireless LAN,
m
m
m
x
z
y
s
s
s
X
Z
Y
K
K
K
Z
Y
X
00
00
00
(2)
75
Internet, 3G and long distance were tested in the same rook, whereas long distance tests were performed
between a Master and Slave site, which were 24 km apart.
Fig. 8: Slave side pick and place task Fig. 9: Multi-modality communication system
Fig. 10: Multi-modality communication system individual communication network
76
For communication in short distance again trails where conducted for this average time was 700
milliseconds. Maximum time recorded for task was 4.2 seconds and minimum time recorded was 120
milliseconds. In the Fig. 13 and Fig. 14 it shows that some of the experimental subjects took little long time
to understand 2D direction movements of the experimental task which results in the increase of average time
little more than expected. Anvari et.al [3] uses the fibre optic communication system in between Master and
slave side and expensive method too.
0
0.1
0.2
0.3
0.4
0.5
0.6
Ist 2nd 3rd 4th 5th
Tim
e D
ura
tio
n (
In M
illis
eco
nd
)
Number of Test of each modality
Communication Modality
LAN
WiFi
Internet
3G
LD
Fig. 11: Communication backup system for safety Fig. 12: Different modality of communication
0
0.5
1
1.5
2
2.5
3
3.5
4
0 2 4 6 8 10 12 14 16 18 20
Tim
e D
ura
tio
n in
Min
ute
Number of Trial by different Users
Pick and Place Task at Long Distance
0
0.5
1
1.5
2
2.5
3
3.5
4
0 2 4 6 8 10 12 14 16 18 20
Tim
e D
ura
tio
n in
Min
ute
Number of Trial by different Users
Pick and Place Task at Short Distance
Fig. 13: Long Distance Pick and Place object Test Fig. 14: Short Distance Pick and Place Test
5. Conclusion
We understand that invention of telesurgery is a boon to all those whole lot who are not in reach of
medical treatment. We understand that only when the doctors are in coordination with the robot with the help
of Internet the telesurgery can be successful. The Graphic User Interface designed here to display vital and
other physiological science for anaesthesiologist monitoring is the first of its kind in the field of telesurgery
itself. The impact of this field is very high because of its real time patient monitoring during telesurgery as it
not only improves the quality of surgical procedure but also helps to reduce any possibility of human error.
From the above experiment we can infer that telesurgery can be performed on various modalities of
communication like the 3G and Broadband Internet even if the modality fails surgery need not be a familiar
as there is a possibility of the control to switch to another modality without any impediments are losses of
data. When compared long distance and short distance performance we notice that longer distance face little
discomfort whereas short distance does not have any such issues. Success depends on the training and
adaptation to the handling of the instruments before performing any real time telesurgery no matter from
whatever distances the surgeon performs robotic surgery.
6. Acknowledgement
77
This project is supported by the National Research University Grant through Mahidol University. The
first author would like to thank Ramathibodi Hospital’s surgical staff. The first author would like to thank
Mr. Sakol Nakdhamabhorn for his great help of tele-communication control, Ms. Nantida Nillahoot for her
great help of unilateral tele-operation and also his fellow BART LAB members for their technical support.
7. References
[1] Taylor, R.H., Stoianovici, D., “Medical Robotics in Computer-Integrated Surgery,” IEEE Trans. on Robotics and
Automation, 19(5), Oct. 2003.
[2] Marscausx J., Leroy, J., Rubino, F., Smith, M., Vix, M., Simone, M., Mutter, D., “Transcontinental Robot-
assisted Remote Tele-surgery: Feasibility and Potential applications” Ann. Surg., 235(4), 2002, pp. 487-492.
[3] Anvari, M., McKinley, C., Sein, H., “Establishment of the World’s First Telerobotic Remote Surgical Service for
provision of Advanced Laparoscopic Surgery in a Rural Community” Ann. Surg., 241(3), March 2005.
[4] Sastry, S. S., et al., “A Laparoscopic Telesurgical Workstation”, IEEE Transactions on Robotics and Automation,
Vol. 15, No. 4, August 1999.
[5] Moustris, G., et al., Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the
literature. The International Journal of Medical Robotics and Computer Assisted Surgery, 2011. 7(4): p. 375-392.
[6] Kang, Y., et al., "Control Design for Tele-Operation System with Time-Varying and Stochastic Communication
Delay" International Journal of Innovative Computing, Information and Control, Volume 8, Number 1(A), January
2012.
[7] Suthakorn, J., “A Concept on Cooperative Tele-Surgical System Based on Image Guiding and Robotic
Technology,” Proc. 2012 Pan American Health Care Exchange (PAHCE 2012), USA, March 2012.
[8] Itsarachaiyot, Y., Pochanakorn, R., Nillahoot, N., Suthakorn, J., “Force Acquisition on Surgical Instruments for
Virtual Reality Surgical Training System,” Proc. 2011 Int’l Con. on Computer Control and Automation (ICCCA
2011), Korea, May 1-3, 2011.
[9] Direkwatana, C., Suthakorn, J., “On the Design and Development of A Novel 4-DOF Wire-Driven Laparoscopic
Surgical Robotic System, “MU-LAPAROBOT”,” Proc. The 7th Asian Conference on Computer-Aided Surgery
(ACCAS 2011), Bangkok, Thailand, August 26-27, 2011.
78