SimITK and SimVTK:SimITK and SimVTK:ITK and VTK in SimulinkITK and VTK in Simulink

DG Gobbi, P Mousavi, KM Li, J Xiang, A Campigotto,DG Gobbi, P Mousavi, KM Li, J Xiang, A Campigotto,A LaPointe, G Fichtinger, P AbolmaesumiA LaPointe, G Fichtinger, P Abolmaesumi

Medical Image Analysis Laboratory (MedIA)Medical Image Analysis Laboratory (MedIA)http://www.media.queensu.cahttp://www.media.queensu.ca

School of ComputingSchool of ComputingQueen’s UniversityQueen’s University

Motivation Motivation

Visual programming for ITK and VTKVisual programming for ITK and VTK Drag & drop approach to application developmentDrag & drop approach to application development Increase understandability & maintainabilityIncrease understandability & maintainability Instructional valueInstructional value

MATLAB is our friendMATLAB is our friend Use MATLAB’s Simulink visual environmentUse MATLAB’s Simulink visual environment Hence, avoid reinventing the wheelHence, avoid reinventing the wheel Allow mix of MATLAB code and ITK / VTKAllow mix of MATLAB code and ITK / VTK

Related WorkRelated Work

Visual Programming of VTK and ITKVisual Programming of VTK and ITK ITKBoard (CSIRO)ITKBoard (CSIRO) SciRun (Utah)SciRun (Utah) MeVisLab (MeVis Research GmbH)MeVisLab (MeVis Research GmbH) caBIG XIP (caBIG, Siemens, Washington U. in St Louis)caBIG XIP (caBIG, Siemens, Washington U. in St Louis)

ITK and VTK in MATLAB (without Simulink)ITK and VTK in MATLAB (without Simulink) MATITK (Chu and Hamarneh)MATITK (Chu and Hamarneh) VTK plus MATLAB – variousVTK plus MATLAB – various

Us: Visual ITK/VTK Programming in MATLABUs: Visual ITK/VTK Programming in MATLAB

MATLAB’s Simulink MATLAB’s Simulink EnvironmentEnvironment

Library Model

Our Own Simulink LibrariesOur Own Simulink Libraries

3D Processing with ITK3D Processing with ITK

Combine ITK Processing and Combine ITK Processing and VTK RenderingVTK Rendering

Input:cthead1.png

Output:VTK Window

Method: Wrapping ProcessMethod: Wrapping Process

Simulink wrappers for ITK, VTKSimulink wrappers for ITK, VTK Each Simulink block is an “S-Function” with Each Simulink block is an “S-Function” with

callbackscallbacks S-Function callbacks are C++ code, calls ITK or VTKS-Function callbacks are C++ code, calls ITK or VTK

CMake build processCMake build process Start with an XML description of each filter (ITK)Start with an XML description of each filter (ITK) oror Start with class header files (VTK) Start with class header files (VTK) CMake calls perl script to generate C++ codeCMake calls perl script to generate C++ code CMake then calls perl to generate .mdl library fileCMake then calls perl to generate .mdl library file Finally, build libraries with VC++ or gccFinally, build libraries with VC++ or gcc

Method: Data Flow for Method: Data Flow for SimITKSimITK

SimITK, SimVTK use different architectureSimITK, SimVTK use different architecture ITK is primarily image basedITK is primarily image based VTK handles many data set typesVTK handles many data set types

SimITK Data FlowSimITK Data Flow ITK images -> MATLAB arrays (“info” and “data”)ITK images -> MATLAB arrays (“info” and “data”) Use shared memory for efficiencyUse shared memory for efficiency Conversion of ITK parameter data typesConversion of ITK parameter data types

ITK uses several specialized types e.g. versorITK uses several specialized types e.g. versor

SimITK Helper ClassSimITK Helper Class

InputPortConversion

itkImage

Run ITK Image Filter

itkImageConversion

OutputPort

Parameter Variables

Conversion

Spacing & Origin

Data

Spacing & Origin

Data

Input

Output

Parameters

Method: Data Flow for Method: Data Flow for SimVTKSimVTK

SimVTK Data FlowSimVTK Data Flow Use MATLAB arrays to pass vtkObject pointersUse MATLAB arrays to pass vtkObject pointers

Define “VTKOBJECT” MATLAB type for safety Define “VTKOBJECT” MATLAB type for safety When run starts, Simulink builds VTK pipeline on-the-flyWhen run starts, Simulink builds VTK pipeline on-the-fly Then, Simulink calls Update or Render at each time stepThen, Simulink calls Update or Render at each time step

SimVTK filter parametersSimVTK filter parameters VTK parameter types need no conversionVTK parameter types need no conversion VTK and MATLAB both use basic C data typesVTK and MATLAB both use basic C data types

Registration ExamplesRegistration Examples

Registration in ITKRegistration in ITK Immediate, interactive feedbackImmediate, interactive feedback Great way to tune parametersGreat way to tune parameters Easy to swap out componentsEasy to swap out components

Intervention ExamplesIntervention Examples

Computer Assisted Intervention in Computer Assisted Intervention in VTKVTK Incorporate real-time data inputIncorporate real-time data input Use “Subsystems” to manage Use “Subsystems” to manage

complexitycomplexity

DiscussionDiscussion

““Diagrams” that actually run as codeDiagrams” that actually run as code Easy to verify that code matches designEasy to verify that code matches design Great teaching toolGreat teaching tool Understandability improves maintainabilityUnderstandability improves maintainability

UtilityUtility Simulink models can run as stand-alone execSimulink models can run as stand-alone exec Nearly all VTK, ITK classes can be wrappedNearly all VTK, ITK classes can be wrapped

Future WorkFuture Work

WrapITKWrapITK To automate ITK build (VTK is already done!)To automate ITK build (VTK is already done!)

GUI integrationGUI integration Slicer module interface in SimulinkSlicer module interface in Simulink KWWidgets (later Qt, maybe FLTK)KWWidgets (later Qt, maybe FLTK)

Community BuildingCommunity Building Mailing list, websiteMailing list, website We want to hear from you!We want to hear from you!