NA-MIC National Alliance for Medical Image Computing OpenIGTLink Update NA-MIC Summer Project Week...

NA-MICNational Alliance for Medical Image Computing http://na-mic.org

OpenIGTLink Update

NA-MIC Summer Project Week 2010

Junichi Tokuda, Ph.D.

Department of Radiology

Brigham and Women’s Hospital

NCIGTNational Center for Image Guided Therapy

http://ncigt.org

National Alliance for Medical Image Computing http://na-mic.org

National Center for Image Guided Therapyhttp://ncigt.org

Backgrounds

• Hardware/software in IGT– Position tracking sensors– MR/CT/Ultrasound scanners– Robotic devices– Planning and navigation software

• Data types– Images– Positions / Transforms– Software / hardware status, etc…

• Communication standard for IGT?



Goals of OpenIGTLink

• Available – Research and commercial

• Simple– From embedded system to HPC

• Extensible– Variety of data types

• Reliable– Used clinically!



Protocol − Overview

• Message-based Protocol– No session / messages are independent– Allows application-specific message types

Header

IMAGE Type Body

Body

TRANSFORM Type Body

COMMAND Type Body

….

Body Size



Protocol − Body

• Position / Transform

x y z ox oy oz w

position orientation (quaternion)

tx ty tz sx sy sz nx ny nz x y z

€

tx sx nx xty sy ny ytz sz nz z0 0 0 1

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟



Protocol − Body

• Image – 2D/3D– Integer / floating point– Scalar / vector– Partial Image update i

j

k

Nj

(0, 0, 0)

(Ni -1, Nj -1, Nk -1)Nk

Mj

Ni

Mi

(oi +Mi -1, oj +Mj -1,ok+Mk -1)

Mk

(oi, oj, ok)

Generic info size transform partial volumeindex/size

Pixel data



The OpenIGTLink Library

• Multi-platform C/C++ Library– Windows, Linux, Mac OS X, Solaris– Used in QNX, iPhone OS, vxWorks etc..

C++ socket & thread class C message structure

C++ message class

Application




• Code snippet (9 lines to send position)




• Various “hello world” examples– Tracker simulator– Imager simulator– Data receive server– etc.



• Import to / export from MRML scene– Image (vtkMRMLScalarVolumeNode)– Transform (vtkMRMLLinearTransformNode)

3D Slicer OpenIGTLink I/F

Tracker

OpenIGTLink

Imager

OpenIGTLink

…

3D Slicer

Connector node 1

Connector node 2

Ext. software

OpenIGTLink

Connector node N

MRML Scene

Module 1

Module 2

Module N

Image

LinearTransform

Image

LinearTransform

Ethernet




• Import / export



3D Slicer + BrainLAB

Elhawary H, Norton I, Patel P, Rigolo L, Papademetris X, Hata N, Golby AJ (BWH / Yale / BrainLAB)



3D Slicer + BrainLAB

• Research prototype (Slicer) +

FDA-approved IGS system (BrainLAB)

3D Slicer

OpenIGTLink

BioImage Suit

OpenIGTLVVLink

VVLink

BrainLABVectorVision

Image / Tracking data

(VVLink)

Image / Tracking data

(OpenIGTLink)

Research PrototypeIGS system

Bridge Software



MRg Prostate Biopsy Robot

Fischer G, Tokuda J, Song S, Iordachita I, DiMaio S, Cho N, Csoma C,Bleivns J, Burdette EC, Fichtinger G, Hata N, Tempany C

(Johns Hopkins, BWH, Queen’s, Acoustic MedSystems)



MRg Prostate Biopsy Robot

• Integration of Slicer, robot and scanner

3D Slicer

OpenIGTLink

Robot Proxy

OpenIGTLink Pipe

Controller

Pipe

Scanner Proxy

OpenIGTLink

MRI Scanner

ProprietaryProprietary

Real-time image

Imaging slice position

Current position

Command



Neurosurgical Manipulator

Arata J, Tada Y, Yamada A, Tanaka Y, Fujimoto H(Nagoya Institute of Technology, Japan)



Neurosurgical Manipulator

• System diagram



Motion Correction for MRgFUS

• MR Image Motion Correction by US-based motion tracking

Schwartz BM, McDannold NBrigham and Women’s Hospital

MRI Scanner

No Motion w/ motionUncorrected

w/ motionCorrected

ScannerHardware Imaging slice position

OpenIGTLink



Cardiac EP Catheter + 3D Slicer

Jagadeesan J, Hata NBrigham and Women’s Hospital



Cardiac EP Catheter + 3D Slicer

• Matab/Simulink in backstage…



Next Steps

• OpenIGTLink Protocol Version 2 (draft)– Backward compatibility with version 1

– Query support

– Better IGS system support

– Other message types?



Query Support

• “Get” message to request data

• “Start”, “stop” and “pause” messages to control data streaming

• “Status” message to respond to query



IGS Systems Support

• Remote Image List– Display a list of images available in the

remote software

• Tracking data– Multi-channel, device name

• Fiducial, trajectory– Share surgical planning information



Other Message Types?

• Remote Execution– Run Tcl/Python on 3D Slicer from remote– Remote procedure call

• XML data / Associative array– General data types– Versatility but compromised performance



MATLAB Interface

• MATLAB – 3D Slicer communication



OpenIGTLink + ZeroMQ

• A layer between OpenIGTLink and TCP/IP• Support different message patterns

– Point-to-point– Publish-subscribe– Request-reply– Parallelized pipeline

• Proposed in CTKhttp://www.commontk.org/index.php/Documentation/Messagingby Patrick Cheng (Georgetown)

Ethernet

IP

TCP

ZeroMQ

OpenIGTLink



Acknowledgements

• BWH. Nobuhiko Hata, Ron Kikinis, Clare Tempany, Alexandra Golby, Haiying Liu, Tina Kapur, Jayender Jagadeesan, Haytham Elhawary, Benjamin M. Schwartz, …

• Isomics, Inc. Steve Pieper• Kitware, Inc. Luis Ibanez• JHU. Sam S. Song, Nathan Cho, Iulian Iordachita, Peter Kazanzides, …• Queen’s. Univ. Gabor Fichitinger• Georgetown Univ. Ziv Yaniv, Patrick Cheng, Kevin Cleary• Acoustic MedSystems Inc. Jack Blevins, E. Clif Burdette• Worcester Polytechnic Institute. Gregory Fischer• AIST, Japan. Kiyoyuki Chinzei• Nagoya Inst of Tech. Junpei Arata, Atsushi Yamada, Hiroaki Kozuka, Hideo

Fujimoto• Nagoya Univ. Yuichiro Hayashi



Grant Support

• This work is supported by 1R01CA111288, 5U41RR019703, 5P01CA067165, 1R01CA124377, 5P41RR013218, 5U54EB005149, 5R01CA109246, R01EB006494 and R21EB007770 from the NIH. This study is also in part supported by NSF 9731748, CIMIT, Intelligent Surgical Instruments Project of METI (Japan).



Publications

• Tokuda J., Fischer G.S., Papademetris X., Yaniv Z., Ibanez L., Cheng P., Liu H., Blevins J., Arata J., Golby A.J., Kapur T., Pieper S., Burdette E.C., Fichtinger G., Tempany C.M., Hata N. OpenIGTLink: an open network protocol for image-guided therapy environment. Int J Med Robot. 2009 Dec;5(4):423-34. PMID: 19621334. PMCID: PMC2811069.

• Tokuda J., Fischer G.S., DiMaio S.P., Gobbi D.G., Csoma C., Mewes P.W., Fichtinger G., Tempany C.M., Hata N. Integrated navigation and control software system for MRI-guided robotic prostate interventions. Comput Med Imaging Graph. 2010 Jan;34(1):3-8. PMID: 19699057. PMCID: PMC2815337.

• Arata J., Kozuka H., Kim H.W., Takesue N., Vladimirov B., Sakaguchi M., Tokuda J., Hata N., Chinzei K., Fujimoto H. Open core control software for surgical robots. Int J Comput Assist Radiol Surg. 2010 May;5(3):211-20. PMID: 20033506.



For more information…

OpenIGTLink ML: [email protected]





History

• Dec 2007: Discussed in IGT Week• Jan 2008: Defined in All Hands Meeting• May2009 : Supported in 3D Slicer 3.4• Jun 2010: Protocol Extension




• Performance Optimization by Steve

Before After

Date post:	04-Jan-2016
Category:	Documents
Upload:	chester-price
View:	215 times
Download:	0 times

NA-MIC National Alliance for Medical Image Computing OpenIGTLink Update NA-MIC Summer Project Week...

Documents