SNM 2008 - Info SNM What IHE Delivers1

Integrating the Healthcare EnterpriseIntegrating the Healthcare Enterprise

Image FusionImage Fusion

Todd KantchevTK Medical Systems

Jerold WallisWashington University

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 2

Image FusionImage Fusion

Scenario: I want to view two studies (e.g. a PET and a CT)

overlayed on the screen

and

I want to look at it again later, maybe even on another screen

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Your Problems with Image FusionYour Problems with Image Fusion

“I registered these before. Why do I have to do it again?” Registrations not transferable between systems

“Why can’t I see it on another workstation?” Presentations not transferable between systems

“Why can I only do PET / CT?” No general purpose registration

Impact time lost to repeated work; delayed care inability to access needed views

3

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 4

How should it work?How should it work?

I scan on a dual system or several systems

Some datasets are pre-registered

The tech does additional registrations or tweaks the pre-registered sets

On my workstation, I adjust the blending and read the images

Later, on another workstation, I retrieve & view perhaps tweak the registration perhaps tweak the blending perhaps register additional datasets

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 5

IHE SolutionIHE Solution

(FUS) – Image Fusion

Record Registration between any datasets Record Blending / Presentation parameters Retrieve & View on any compliant workstation

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Spatial Registration ObjectSpatial Registration Object

(Not required for already-registered PET-CT data from a hybrid scanner )

SROSROImage 1Image 1

Image 2Image 2

TransformationTransformation

6

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 7

Spatial Registration ObjectSpatial Registration Object

Encodes transformation to align images in a common registered space

Identifies the registered space

References the images to be aligned

Not required for already-registered PET-CT data from a hybrid scanner

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Blending Presentation StateBlending Presentation State

Image 1Image 1

Image 2Image 2

SROSRO

TransformationTransformationBPSBPS

Window – 1Window – 1

Window – 2Window – 2

Color TableColor Table

TransparencyTransparency

8

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 9

Blending Presentation ObjectBlending Presentation Object

References one set of underlying images (e.g. CT)

References one set of superimposed images (e.g. PET)

References (optionally) a Registration Object to align both image sets before blending.

Encodes the window level for each set

Encodes the blending factor (0 to 100%) of the overlay transparency

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Essential Clinical FeaturesEssential Clinical Features

MPR

Fusion

Side-by-side Registered

10

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 12

Image Fusion Image Fusion Actors and TransactionsActors and Transactions

Acquisition Modality

Evidence Creator

Image Manager

↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored↓ [RAD 56] Spatial Registrations Stored↓ [RAD 10] Storage Commitment

↓ [RAD 14] Query Images↓ [RAD 16] Retrieve Images↓ [RAD 15] Query Presentation States↓ [RAD 17] Retrieve Presentation States↓ [RAD 58] Retrieve Spatial Registrations

Image Archive

[RAD 8] Modality Images Stored ↓[RAD 57] Blending Presentation States Stored ↓[RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓

Image Display

Vol 1-20.1 Actors and Transactions in Image Fusion Profile

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 13

Image Fusion Image Fusion Actors and TransactionsActors and Transactions

Acquisition Modality

Evidence Creator

Image Manager

↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored↓ [RAD 56] Spatial Registrations Stored↓ [RAD 10] Storage Commitment

↓ [RAD 14] Query Images↓ [RAD 16] Retrieve Images↓ [RAD 15] Query Presentation States↓ [RAD 17] Retrieve Presentation States↓ [RAD 58] Retrieve Spatial Registrations

Image Archive

[RAD 8] Modality Images Stored ↓[RAD 57] Blending Presentation States Stored ↓[RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓

Image Display

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 14

Load CT and PET images

Align images (automatically or manually) and fuse with desired blending factor

Store the alignment, including

Registration object Blending State (defining the blending

factor, window levels and referencing all other objects)

Storage can be on PACS or disk. Images do not need to be stored if already stored.

Fusion- The Creating Process

CTCTCTCT

PETPETPETPET

Blending StateBlending StateBlending StateBlending State

Registration ObjectRegistration ObjectRegistration ObjectRegistration Object

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 15

Acquisition ModalitiesAcquisition Modalities

Hybrid Modalities are required to store Blending Softcopy Presentation State for the datasets, which are in the same Frame of Reference. Typically, the Referenced Spatial Registration Sequence is absent since no transformation needs to be applied to register the referenced images before display.

All modalities on the IHE TF are now explicitly required, when creating image objects, not to use the same Frame of Reference for multiple image datasets unless all the images are in the same Reference Coordinate System.

4.57.4.1.2 Message Semantics – BSPS Stored

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

TESTTEST

FUS_MOD_STORE: Modality Stores Images and BSPS to Central Archive

Special Instructions

Acquistion Modalities should complete this test on the first day of the Connectathon to enable Evidence Creators, Image Display and Image Archives to complete their tests that depend on this one.

No formal evaluation ? Validate BSPS?

16

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 17

Image Fusion Image Fusion Actors and TransactionsActors and Transactions

Acquisition Modality

Evidence Creator

Image Manager

↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored↓ [RAD 56] Spatial Registrations Stored↓ [RAD 10] Storage Commitment

↓ [RAD 14] Query Images↓ [RAD 16] Retrieve Images↓ [RAD 15] Query Presentation States↓ [RAD 17] Retrieve Presentation States↓ [RAD 58] Retrieve Spatial Registrations

Image Archive

[RAD 8] Modality Images Stored ↓[RAD 57] Blending Presentation States Stored ↓[RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓

Image Display

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 18

Evidence CreatorsEvidence Creators

Create and store Blending Softcopy Presentation objects as required Referenced Spatial Registration Sequence (0070, 0404) references the Spatial

Registration object, unless spatial registration is not required The Presentation State Relationship Macro in the Blending Sequence (0070,

0402) lists all the images or frames to which blending is to be applied Encodes presentation (blending, hue, windowing etc.)

Create and store Spatial Registration Objects as required Contains one or more Registration Sequences, each defining the transformation

of the corresponding Original Dataset into the Registered Space. Stored in the most recent of the Studies containing the registered images and in

a different series from images and presentation states.

Optionally create derived (re-sampled) images.

4.56.4.1.2 Message Semantics – Spatial Registration Stored

4.57.4.1.2 Message Semantics – BSPS Stored

See also 4.18.4.1.2.5 for storage of Derived Images

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Test: FUS_MOD_ECTest: FUS_MOD_EC

FUS_MOD_EC: Evidence Creator performs registration and blending on Modality image series

Images will be on the Connectathon CENTRAL_ARCHIVE after Modalities complete their FUS_MOD_STORE test.

This is a test between Modality and Evidence Creator actors

19

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Demonstrate that Evid. CreatorDemonstrate that Evid. Creator

…register image datasets which do not share a Frame of Reference, using translations, rotation & scaling, as required

…(recommended) that it notes the method of registration within the Spatial Registration object.

…it can set the contrast/window settings for the underlying grayscale dataset, and … it can set the windowing and color map for the overlay dataset

… it can set the transparency for the overlay dataset

20

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

FUS_MOD_ECFUS_MOD_EC

Create several examples of the presentation state, with different registration and window/blending settings (eg, different transparency or color maps)

Store them to the archive for use by Displays.

21

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 22

Image Fusion Image Fusion Actors and TransactionsActors and Transactions

Acquisition Modality

Evidence Creator

Image Manager

↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored↓ [RAD 56] Spatial Registrations Stored↓ [RAD 10] Storage Commitment

↓ [RAD 14] Query Images↓ [RAD 16] Retrieve Images↓ [RAD 15] Query Presentation States↓ [RAD 17] Retrieve Presentation States↓ [RAD 58] Retrieve Spatial Registrations

Image Archive

[RAD 8] Modality Images Stored ↓[RAD 57] Blending Presentation States Stored ↓[RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓

Image Display

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 23

Image Manager/ArchiveImage Manager/Archive

Able to store, query and retrieve Blending Softcopy Presentation States, supporting all required keys, already defined in the TF. Note that SOP Class UID (a required matching key for the SCP) is a useful way to limit responses to queries for Blending Softcopy PS.

Able to retrieve Spatial Registration Objects.

Retrieval of images must be supported at Series Level.

4.15.4.1.2 Message Semantics- Query Softcopy PS

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 24

Image Fusion Image Fusion Actors and TransactionsActors and Transactions

Acquisition Modality

Evidence Creator

Image Manager

↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored↓ [RAD 56] Spatial Registrations Stored↓ [RAD 10] Storage Commitment

↓ [RAD 14] Query Images↓ [RAD 16] Retrieve Images↓ [RAD 15] Query Presentation States↓ [RAD 17] Retrieve Presentation States↓ [RAD 58] Retrieve Spatial Registrations

Image Archive

[RAD 8] Modality Images Stored ↓[RAD 57] Blending Presentation States Stored ↓[RAD 56] Spatial Registrations Stored ↓[RAD 10] Storage Commitment ↓

Image Display

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 25

Image DisplayImage Display

Able to retrieve, read and apply to images Blending Softcopy Presentation States and, if referenced, Spatial Registration Objects as required

Able to retrieve the referenced images, if not already present locally, and perform image registration as required if necessary

Display images, supporting specific display features (fusion, side by side, MPR, adjustment of blending, windowing- aka NM, navigation through spatial and time dimensions, display content of specific attributes etc.)

4.17.4.2.3 Retrieve/View Presentation States.

4.58.4.1.2 Retrieve Spatial Registrations

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Essential Clinical FeaturesEssential Clinical Features

MPR

Fusion

Side-by-side Registered

26

SNM 2008 – InfoSNMSNM 2008 – InfoSNM 27

50%50%

Blending PresentationBlending PresentationOverlay TransparencyOverlay Transparency

20%

50%100%

80%

50%

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- Side by SideExample- Side by Side

Side-by-side can be up/down (PET/CT)

Side-by-side can omit fusion image when displayed in this format

Recommended – MIP image capability (upper left)

29

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- Side by SideExample- Side by Side

Side-by-side can be up/down, can be multiple levels (eg 3 levels as shown here)

Side-by-side can include fusion image (preferred)

30

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- Side by SideExample- Side by Side

Side-by-side can include multiple CT window settings (as shown here)

Side-by-side can include fusion image (preferred)

31

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- Side by SideExample- Side by Side

PET-only image (upper right) can be pure BW (preferred as on prior slide, but not mentioned in profile), but can use same LUT as fused image as shown here

32

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- MPRExample- MPR

MPR screen must be able to show 3 planes

MPR screen need not show side-by-side as well, but can do so (here with pure CT image at upper left, and fused below)

33

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- MPRExample- MPR

Must be able to adjust blending (transparency)

Same screen adjusted to be mostly CT

34

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- MPRExample- MPR

Must be able to adjust blending (transparency)

Same screen adjusted to be mostly PET

35

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- MPRExample- MPR

Different vendor example for MPR fused

36

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Fusion controlsFusion controls

Must have controls for Windowing CT (using standard center/width controls) Windowing PET/NM (using separate upper & lower level controls , not

center/width controls Adjusting transparency Navigating volume in three planes Changing color LUT table for overlay is recommended but not required

37

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- ControlsExample- Controls

38

Must have controls for Windowing CT (using standard center/width controls) Windowing PET/NM (using separate upper & lower

level controls , not center/width controls Adjusting transparency Navigating volume in three planes Changing color LUT table for overlay is recommended

but not required

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- ControlsExample- Controls

39

Must have controls for Windowing CT (using standard center/width controls) Windowing PET/NM (using separate upper & lower

level controls , not center/width controls Adjusting transparency Navigating volume in three planes Changing color LUT table for overlay is recommended

but not required

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Example- ControlsExample- Controls

40

Must have controls for Windowing CT (using standard center/width controls) Windowing PET/NM (using separate upper & lower

level controls , not center/width controls Adjusting transparency Navigating volume in three planes Changing color LUT table for overlay is recommended

but not required

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Selecting StudiesSelecting Studies

Users must be able to select the softcopy presentation state (BSPS)

More than one BSPS may be present

If not already present, any images or spatial registration objects referenced in the Presentation State shall be retrieved by the Image Display. (For clarification on this, see closed issue 19 and section 4.17.4.2.3 )

41

SNM 2008 – InfoSNMSNM 2008 – InfoSNM

Full Profile on IHE SiteFull Profile on IHE Site

http://www.ihe.net/Technical_Framework/index.cfm#radiology

http://www.ihe.net/Technical_Framework/index.cfm#radiology

Open link on page above for FUS profile

42