In nuclear medicine we are more enthused about volume rendering than surface rendering, because our borders are so fuzzy, and (of greater impor- tance) our unique medical information is contained within those borders. J.M. Links, 1995 Chapter 4 Normal Fusion for 3D Integrated Visualization of SPECT and MR Brain Images Abstract Multimodality visualization aims at efficiently presenting integrated information obtained from different modalities, usually combining a functional modality (SPECT, PET, fMRI) with an anatomical modality (CT, MRI). This chapter presents a technique for 3D integrated visua- lization of SPECT and MR brain images, where MRI is used as a framework of reference for the display of the SPECT data. Methods: A novel technique for 3D integrated visualization of functional and anatomical information, called Normal Fusion, is presented. With this technique local functional information is projected onto an anatomic structure. Results: The Normal Fusion technique is applied to three cases of SPECT/MRI integration. The results are presented, discussed and evaluated for clinical relevance. Conclusions: The results for 3D integrated display of SPECT and MR brain images indicate that the Normal Fusion technique provides a potentially comprehensive and diagnostically valuable presentation of cerebral blood perfusion in relation to the anatomy of the brain.
Transcript
In nuclearmedicinewe are more enthusedaboutvolumerenderingthansurfacerendering, becauseour borders are so fuzzy, and (of greater impor-tance)ouruniquemedicalinformationiscontainedwithin thoseborders.
J.M. Links,1995
Chapter 4
Normal Fusion for 3D IntegratedVisualization of SPECT and MRBrain Images
AbstractMultimodality visualizationaims at efficiently presentingintegratedinformation obtainedfromdifferentmodalities,usuallycombiningafunctionalmodality(SPECT, PET, fMRI) withananatomicalmodality(CT, MRI). Thischapterpresentsatechniquefor 3D integratedvisua-lizationof SPECTandMR brainimages,whereMRI is usedasa framework of referenceforthedisplayof theSPECTdata.Methods: A novel techniquefor 3D integratedvisualizationof functional and anatomicalinformation, called Normal Fusion, is presented.With thistechniquelocal functionalinformationis projectedontoananatomicstructure.Results: TheNormalFusiontechniqueis appliedto threecasesof SPECT/MRIintegration.Theresultsarepresented,discussedandevaluatedfor clinical relevance.Conclusions: Theresultsfor 3Dintegrateddisplayof SPECTandMR brainimagesindicatethattheNormalFusiontechniqueprovides a potentially comprehensive and diagnosticallyvaluablepresentationof cerebralbloodperfusionin relationto theanatomyof thebrain.
42 NormalFusionfor 3D IntegratedVisualizationof SPECTandMR Brain Images
4.1 Introduction
Theuseof multiple imagingmodalitiesfor clinical examinationsis graduallyincrea-sing (Maisey et al. 1992,Viergever et al. 1992). Thegoalof multimodalityvisual-ization is to comprehensiblyconjoin the diagnosticinformationof different imag-ing modalities,andto communicatethe integratedinformationto the referringspe-cialists. Thecombinationof complementarydatafrom multiple modalitiesmayre-vealadditionaldiagnosticinformationascomparedwith interpretationdirectly fromthe individual imagingmodalities(Levin et al. 1989,Pelizzariet al. 1989,Holmanet al. 1991,Valentinoet al. 1991,Hill et al. 1992). We distinguishtwo typesofmultimodalityintegration,viz.; i) thecombinationof anatomicaldatafrom differentmodalities,andii) thecombinationof functionalwith anatomicaldata.
An exampleof theintegrationof multimodalanatomicalinformationis thefusionof CTandMRI in skull basesurgery, whereit isusedtodeterminethepreciselocationof a lesion (MRI data)with respectto bone(CT data) in order to obtain a moreaccuratediagnosisand treatment(Ruff et al. 1993). Other examplesof CT/MRIfusion,e.g., for radiationtherapy planning,canbefoundin (ChenandPelizzari1989,VandenElsenandViergever 1994,VanHerk andKooy 1994,VandenElsenet al.1995,Maintzetal. 1996a);in all thesecasesthecomplementarityof theinformationobtainedfrom CT andMRI is utilized.
Multimodality displaycanalsointegratefunctionalinformationfrom, e.g., PET,SPECT, EEG,MEG, MRSI, or fMRI with anatomicalinformationfrom MRI or CT(Gevins et al. 1990,Schneideret al. 1990,Holmanet al. 1991,Evanset al. 1991,Knufmanet al. 1992,Viergever et al. 1992).Theanatomicalmodalitythenprovidesa frameof referencefor spatiallycorrectinterpretationof thefunctionalinformation.Valentinoet al. (1991)states:”In brain imagingin particular, the accuratedisplayof functionalandanatomicimagedatais essentialin identifyingsitesof normalandpathophysiologicfunctionin thebrain.”
This chapteraddressesa novel techniquefor 3D integrateddisplayof SPECTandMR brain images.WhenusingSPECTin isolation,investigationof functionalprocessesandthecorrelationwith anatomicalstructuresis hamperedby thelow spa-tial resolution(MazziottaandKoslow 1987,Kundel1990,Zubalet al. 1995,Evanset al. 1996). Two optionscanbe appliedto facilitate the investigation of SPECT;i) theuse(andmanipulation)of color encodingfor thedisplayof SPECTdata,soasto employ thepotentialof thevisualsystemmoreeffectively (Kundel1990,Stapletonet al. 1994),andii) visually comparingSPECTbloodperfusionin pertinentcerebralregionswith thehomologousregionsof thecontralateralhemisphere(Kundel1990,Stapletonetal. 1994,Zubaletal. 1995).To furtherimproveunderstandingof theun-derlyingrelationshipsbetweenfunctionandanatomy, it is essentialthatanatomicalinformation,e.g., acquiredwith CT and/orMRI, is usedasa framework for SPECTinformation(Britton 1994).
haveusedlinkedcursors,alternatepixel display, color integrationprocedures,andat-lasses(Schadetal.1987,Weissetal.1987,Pelizzarietal. 1989,Hawkesetal.1990).Integratedvisualizationtechniquesfor SPECTandMR imagesincludeswork doneby Condon(1991),who implementedfive techniquesthatusedSPECTinformationasanextradimension(eitherasheight,coloror time) to a2D MR image.
Techniquesfor 3D multimodalityvisualizationof functionalandanatomicaldatahave beenusedmainly for PET/MRI using windows (Levin et al. 1989), opacityweighteddisplay(Evansetal. 1996),cutplanes(Evansetal. 1991)or mappingfunc-tional activity onto the brain surface(Levin et al. 1989,Hu et al. 1990,Valentinoet al. 1991).The3D presentationof SPECTinformationcombinedwith anatomicalinformationhasbeenprimarily focusedon thevisualizationof alreadydetectedab-normalities,sothatstandardvolumevisualizationtechniquescanbeapplied(Wallis1992).
Thepresentstudydiscussesa novel technique,calledNormalFusion,to simul-taneouslydisplayfunctionalandanatomicaldata.The—preliminary—evaluationofthe methodfocuseson threecasesthat investigate the relationbetweenbehavioraldisordersandfunctional/morphologicalbrain damage.The casesare: 1) a patientwith a frontal lobetumor(Hulshoff Pol et al. 1995),2) a patientwith autisticbehav-ior, and3) a patientwith theGilles dela Tourettesyndrome(TS).Theobjectivesofour work areto investigatewhetheri) themultimodalinformationcanbepresentedsimultaneouslyandcomprehensively, andii) additionalinformationcanbeobtainedfrom simultaneouspresentationof thedata.
Theorganizationof thechapteris asfollows: This introductiononmultimodalityvisualizationof SPECTandMRI is followedby a brief overview of theacquisitioncharacteristicsandthetoolsthatwereused.An explanationof theemployedvolumevisualizationmethodprecedesa descriptionof the principlesandindividual meritsof the Normal Fusiontechnique. The visualizationresultsfor the threecasesarepresentedandevaluated,followedby theconclusions.
4.2 Acquisition
Informationon brain anatomywasacquiredfrom a T1-weighted3D gradient-echoMR image.TheMRI dataof thewholeheadconsistedof contiguousaxialslices(128for case1, 131for case2, and127for case3) of 1.3mm thicknesswith TR=30ms,TE=13ms,256� 256matrix,and230mmFOV. TheMR imageswereacquiredwitha whole-bodyPhilipsGyroscan0.5Teslausinga standardheadcoil. Informationonfunctionalprocessesin thebrainwasobtainedfrom a HMPAO–SPECTscan,whichportraysthe cerebralblood perfusion(Peraniet al. 1988). The SPECTdatawasreconstructedto contiguousaxial slices(36 for case1, 46 for case2, and44 for case3) with a64 � 64matrix,aslicethicknessof approximately7.1mm,aplaneresolutionof 7.5mm FWHM, acquiredwith a Picker PRISMTM three-detectorgammacamera
44 NormalFusionfor 3D IntegratedVisualizationof SPECTandMR Brain Images
usinga long-boreultrahigh resolution,low energy fanbeamcollimator.
4.3 Processing and visualization
Registrationof thedatasetswasdoneusingexternalarrow-shapedskin markersandin-housedevelopedsoftware(VandenElsen1993). We choseto matchto theMRIdatato avoid degradationof the cortex renderings.We usedANALYZETM (Robb1990)for thesegmentationof thedatasets.For themultimodality3D visualization,we usedthesoftwarepackageVROOM (Zuiderveld 1995),developedat our depart-ment;it isessentiallyacollectionof C++ classesaimedatmultimodalityvisualization.
4.4 3D integrated visualization
Realisticimagesof 3D medicalvolumedataon a computermonitorcanbeobtainedwith a processcalledvolumevisualization.Thedemandfor volumevisualizationof3D imagingdatain routineclinical work is rapidly increasing.This is truenot onlyfor theanalysisof thedataby theradiologistor thenuclearmedicinephysician,butalso,andmaybemoreimportantly, for communicationwith the referringphysicianor surgeon. For example,brainsurfacestructuresaregenerallyhardto identify forlack of anatomicalinformationwhenusing2D imagesonly. With thehelpof a 3Drenderingof the brain,gyri andsulci aremucheasierto trace,which alleviatesthestudyof brainanatomy(Kundel1990,HohneandHanson1992,Kikinis etal. 1992).
4.4.1 Visualization of anatomical surfaces
Volumevisualizationrelieson shadingtechniquesto modelthelight absorption,re-flection,andtransmissionalongsurfaces. In general,photorealismis not required,whichis why simpletechniquescanbeusedto achieveadequatevisualizationspeeds.
A furthersimplificationof theprocessis theuseof orthographicinsteadof per-spectiveprojection.Thisconsiderablysimplifiesandspeedsupthevisualizationpro-cess,which is why orthographicprojectionis still oftenused.
In general,modern3D visualizationtechniquescalculatethe surfacedirectionfrom theoriginal grey data. Thesurfacenormalcanbe representedby thenormal-izedgrey level gradient(Hohneetal.1990).For everypointonasurfacethisgradientcanbecalculatedfrom thegrey level dataof its neighbors(e.g., six first orderneigh-borsor in a secondorder (3 � 3 � 3) neighborhood).Normalizationof the gradientsubsequentlyyieldsthe(outward)surfacenormal.
4.43D integratedvisualization 45
A simplelight reflectionmodelsufficesin mostcases.Themostusedlight modelis thatof Phong(Phong1975),which separatesthereflectedlight into threecompo-nents,viz.; i) anambient,ii) a diffuse,andiii) a specularcomponent.Furthermore,asignificantreductionin visualizationspeedcanbeobtainedby approximationof thespecularcomponent(Schlick1994).
Thedemandfor 3D volumevisualizationis evenmorestringentfor multimodaldatasets,wheremental3D reconstructionof themultivariateinformationis nigh im-possible.In thischapter, wepresentanovel techniquefor 3D integratedvisualizationof SPECTandMRI. The techniqueNormalFusioncolor encodeslocal SPECTac-tivity ontothebrainsurfacerenderedfrom MRI.
4.4.2 Normal Fusion
MRI
orGradient
Inwardnormaloutward normal
Gradient
SPECT
Figure 4.1 Principle of the Normal Fusiontechnique. Thegradientcalculatedin a standard volumetricrenderingprocedure of MR imagesis usedto evaluatethecorrespondingSPECTdata.
46 NormalFusionfor 3D IntegratedVisualizationof SPECTandMR Brain Images
A B C
D E F
0 255
Figure 4.2 Seepage65.
A B
Figure 4.3 NormalFusionvisualizationusingthemaximumvalueof theSPECTsamplesto color encodethe surface. Frame(A): Color encodingwith the lookuptableproposedin thepresentchapter. Frame(B): Color encodingwith theoriginalheated-objectscale.
4.43D integratedvisualization 47
��� Figure 4.2 Top part: Surfacecolor encodingof SPECTactivity on a brainsurfacefor a patientwith a right frontal lobetumor(seearrow)(A),a patientwith autisticbehavior(B), anda patientwith TS(C). For thesethreecases3Drenderingsof theright andlefthemispherewithdifferentdepthrangesarepresented;0–1mm(top row), 0–5mm(middlerow), and0–10mm(bottomrow). Themaximumvalueover the depthrange wasusedto color encodethe correspondingsurfacevoxel. Bottompart: Influenceof the integrationdirectionon themultimodalityvisualization.Therenderingsin Frames(D)and(E) aretheresultof fusionemployingtheviewingdirection.Bothimagescolor encodethehighestactivity onto thesurface, Frame(D) traversesthewholedataset,Frame(E) usesonly thefirst 10 mm.Frame(F) resultsfromtheNormalFusiontechniquedepictingthehighestactivity over a depthof10 mm. Theportrayedlookuptable wasthe basisfor the color encoding;scalingwasadjustedfor each caseaccordingly.
HMPAO–SPECTimagesportray the cerebralblood perfusion(Roperet al. 1991,Matsudaet al. 1993,FaberandFolks 1994),which is tightly coupledwith theactiv-ity of thetissue.Activity in thebrainis mainly locatedin thegrey matter(Valentinoet al. 1991),whoseprincipalpart is thefoldedsurfacelayer(about2–10mm thick).Variousmethodsthatmapfunctionalactivity of thegrey matterontothebrainsurfacerenderedfrom anatomicaldatahave alreadybeenproposed(Levin et al. 1989,Huet al. 1990).Thesetechniquessharethecharacteristicthatthefunctionalactivity be-low thesurfaceis sampledalongtheviewing direction.Fromthesesamplesa valueis calculatedwhich is color encodedonto the surface. The resultingvisualizationsprovide theobserver with specificinformationconcerningbrainfunctionin relationto anatomy. However, theobtainedfunctionalinformationis a representationof in-formationintegratedalongtheviewing directionratherthantheactuallocal activitybeneaththe brain surface. We developeda techniqueto investigate the functionalactivity moreaccuratelymakinguseof thegradientthat is calculatedin thevolumevisualizationprocess.
NormalFusionspawnsa secondaryray at thesurfacealongthereversegradientor inwardnormaldirection(seeFigure4.1).With SPECT/MRIvisualization,weusethissecondaryrayto projectlocal functionalbrainactivity from SPECTontothesur-facerenderedfrom MRI–T1 data. The valueat a surfacevoxel is calculatedfromsamplepointsof SPECTin thedirectionof theinwardnormal;integrationdepthandsamplingrateareuserdefinable.Thesurfacevoxel hasbeencalculatedin two ways(othermethodscanbeimplementedeasily),viz.; i) by takingtheaverageof thesam-plevalues,andii) by takingthemaximumintensityvaluealongthesampledirection(MIP). Theobtainedvaluerepresentsthecerebralbloodperfusionjustbelow thesur-face.Thecalculatedactivity is thencolor encodedontothevolumetricrenderingofthecortex asderivedfrom theMRI data.
Theintegrateddisplayshown in Figure4.2 (FramesA, B, andC) is theresultof
48 NormalFusionfor 3D IntegratedVisualizationof SPECTandMR Brain Images
theNormalFusionprocedureappliedto theSPECT/MRIdatasetsof thethreecasesusing different depths(top row for a depthof 1 mm, middle row for 5 mm, andthebottomrow for 10 mm), onesamplepermm, andcolor encodingthemaximumSPECTvalueontothesurfaceof thebrain.
The resultsof the patientwith the frontal lobe tumor (case1) arepresentedinFigure4.2A. The left hemisphere(right column)portraysSPECTactivity (cerebralbloodperfusion)beneaththesurfaceof anassumedhealthy brainhemisphere.Therighthemisphere(left column)showshow thedeteriorationof gyri andsulci,asisvis-ible from theMRI cortex data(seearrow), matchestheabnormalregionvisible fromthe color encodedSPECTactivity. The abnormalgyral patternis hypoperfusedinaccordancewith thepresenceof a tumorunderneaththis surfaceregion. We observethat a strip of increasedactivity, correspondingwith increasedblood perfusionbe-neaththesurface,surroundsthedamagedregion. Furthermore,theright hemisphereandcerebellumshow anoverall increasein activity comparedto theleft hemisphereandcerebellum,apparentlynodiaschysisis present.
Theresultsof case2, a patientwith autisticbehavior areshown in Figure4.2B.It shows several differencesbetweenleft andright hemisphere,viz.; i) frontal lobehyperperfusion,left hemisphereslightly higherthanthe right, and ii) left temporalhypoperfusion.Although the gyrationappearednormalon the 2D MR images,the3D volumerenderingis suggestive of anabnormalgyral patternof theleft temporallobe.Noteworthy is anabsenceof perfusionin theareaof Wernicke.
Theresultsof case3, a TS patient,arepresentedin Figure4.2C. Thetop of thebrain wasnot scanned.Several differencesbetweenleft andright hemispherecanbe noted,viz.; i) a stronghot-spotin the right lateralfronto-orbitalregion which isalreadyclearlyvisibleatadepthof 1 mm, ii) increasedactivity in theleft dorsalpari-etal lobeover a diffusearea,andiii) increasedactivity in theleft dorsalcerebellum,with anormalright cerebellum.
The findingsof all threecaseswerebasedon both the 2D SPECTdataandthe3D NormalFusionimages.Someof thereportedfindingsweredifficult to establishuponexaminingthe2D SPECTslicesonly. Moreover mentalreconstructionof the3D activity in relationto theanatomyappearedadifficult task.
StandardSPECTlookup tableswerenot suitablefor our applications,becausetheirlow-activity colorsareshadesof black.An instructivesimultaneouspresentationof the cortical hot-spotsin SPECTin relationto the MRI datawasobtaineduponadjustingtheheated-objectscalesuchthat the low-activity color waschangedfromblack to blue andthe scalewasreverted(seeFigure4.3). While this lookup tableis valid for thesethreecases,visualizationof othercases(e.g., for visualizationofcold-spots)mayrequirea differentlookuptable. Theoptionto (rapidly) changethecolor encodingof the calculatedsurfacevalueby manipulationof the lookup tableappearsavaluableextensionof thisvisualizationtechnique.
We have studiedthe effect of differentdepthsand samplingrateson the finalvisualization,as well as different strategies for deriving a surfacevalue from the
4.43D integratedvisualization 49
activity beneaththe surface. With the samplingrate,a trade-off betweenaccuracyandspeedexists. Thesamplingrateshouldbehigh enoughto accuratelysampletheSPECTdata,but preferablyaslow aspossiblefor a quick assessmentof thevisual-izationresult.Theevaluationstrategy dependson theinformationtheobserver is in-terestedin. For instance,MIP appearsto bethemethodof choiceto revealhot-spots,but the averagingmethodmay be moresuitableto give an impressionof the entirerangeof sampledSPECTintensities.Otherdataandapplicationsmaydemanddiffer-entcalculationparadigms.Changingthedepth,or better, thedepthrange,overwhichtheSPECTdataaresampledoffersthepossibilityto revealsuperficialor moredeeperlying regionsof activity. With case3, thehot-spotin theright lateralfronto–orbitalregioncouldalreadybeappreciatedusingonly thefirst voxel of SPECTinformationbelow thebrainsurface.Otherhot-spotsdid notemergeuntil thedepthwasincreasedto 10mm(seeFigure4.2C) or beyond.
Oneof themainadvantagesof theNormalFusionmethodis that it follows thecurvatureof the brain to calculatethe regional activity of subcorticalcells, whichmakes the visualizationindependentof the viewing direction. Earlier techniques(e.g., for PET/MRI see(Levin et al. 1989,Hu et al. 1990))madeuseof theviewingdirectionto integrateinformationontothesurfaceof thebrain.To illustratetheeffectof the integration direction, we renderedthe left hemisphereof case3 with threemethods. The first methodusedthe viewing direction throughthe entire volume(seeFigure 4.2D), the secondusedthe viewing direction with a depthof 10 mm(seeFigure4.2E), andthethird usedtheinwardnormaldirectionwith a depthof 10mm (seeFigure4.2F). For all threemethodswe usedonesamplepermm,while themaximumvaluewascalculatedto colorencodethesurface.
In Figure4.2D two clearly delineatedhot-spotsarevisible in the left inferiorfrontal andleft inferior temporalregion. Thefirst hot-spotstemsfrom theright lat-eral fronto-orbitalregion (seeFigure4.2C) andthesecondhot-spotis activity fromthefiducialmarkerattachedto theright temporo-mandibularskinsurface.In thepari-etal andoccipital region thereareotherhot-spotsthat resultfrom right hemisphereactivity. Theeffectof activity in theright hemisperevisibleon theleft hemisphereisgreatlyreducedwhenthedepthis decreasedascanbeseenin Figure4.2E. Still, activ-ity depictedonthesurfaceof agyrusmayresultfrom activity locatedin aneigboringgyrus. This effect canbestbeappreciatedin a movie-sequencewheretheangle-of-view is changed.Thenthelocation,form,andcolorof thehot-spotschangesfor eachangle.Theactivity of thecerebellum,which canbeusedasa reference(Hashikawaet al. 1995),alsochangeswith theviewing angle.Thedependency on theangle-of-view provedlarge(esp.whenusingtheentirevolume),which reducestheability tocorrectlylocalizeactivity. Figure4.2F depictstherenderingwith theNormalFusiontechnique,whichis insensitiveto theviewing direction.Fromeachviewing anglethecolorson thesurfaceremainexactly identical.
An extensive clinical evaluationof multimodalityvisualizationtechniquesis inprogress(seeChapter6). This will requiremany moredatasetsthat arecurrently
50 NormalFusionfor 3D IntegratedVisualizationof SPECTandMR Brain Images
collected. Below, we give a first impressionof the validity of the Normal Fusionapproachbaseduponthethreecasesshown in thissection.
4.5 Assessment of clinical relevance
A preliminaryevaluationof the Normal Fusiontechniquewas carriedout by fivenuclearmedicinephysiciansof theUniversityHospitalUtrecht.They answeredasetof simple,yet fundamentalquestionssoasto assessthepossibleclinical benefitsofthetechnique.Sincewe reporton threecasesonly andanticipateda trainingeffect,we decidedto conducttheevaluationtwice for eachobserver with at leastoneweekbetweenthetwo sessions.
Thecaseswerepresentedto eachof thenuclearmedicinephysiciansseparatelyandconsistedof the usualSPECTdatain the familiar settingof their own depart-ment.Theorderwasidenticalfor eachnuclearmedicinephysician,viz.case1, case2,andthencase3. First,thephysicianwasaskedto performaroutineclinical screen-ing of thepatientdatawith theoriginal clinical informationat hand.For case1 thisinformationwas:Operationbecauseof a histologicallyconfirmedright frontal aste-rocytomegradeII (seealsothe2D imagedatain Figure4.1). Question:How is thesurroundingcerebralbloodflow? For case2: Dysfunctionrelatedto autism. Ques-tion: Abnormalitiesfrontal regionandbasalganglia?For case3: Tourettesyndrome.Question:Perfusionabnormalitiesin basalganglia?
Subsequently, preprocessedNormal Fusion imageswere addeddepicting thehighestSPECTactivity on thebrainsurfacefor four depths(1, 5, 10, and15 mm.)andfor four viewing angles(from thefrontal,right, left, andcaudalsideof thebrain).The right andleft views over depthsof 1, 5, and10 mm for the threecasesarede-pictedin Figures4.2A, B, andC. Thefollowing list of questionshadto beansweredfor thepresentationof eachcase:
1. Do you find it difficult to relatethe information acquiredfrom the NormalFusiontechniqueto thetraditional2D SPECTimagesfor thiscase?
2. Is theNormalFusiontechniquebeneficialin establishingananatomicalframe-work for thefunctionalSPECTinformationfor thiscase?
3. Do youfind thisframework usefulfor theinvestigationof SPECTfor thiscase?
4. Do youexpectthatthis typeof presentationwill facilitateestablishingthedif-ferentialdiagnosisfor thiscase?
5. Do you expectthatcommunicationto thereferringclinician will beimprovedwith this typeof presentationfor thiscase?
6. Do you expectto usetheNormalFusionpicturesfor a generalimpression(anoverview) wheninterpretingthe2D SPECTimagesfor thiscase?
Question mean s.d. mean s.d. mean s.d. mean s.d. mean s.d. mean s.d.
Difficult to relateNF to 2D?
3.6 2.0 4.2 0.8 5.0 - 5.0 - 5.0 - 5.0 -
Establishesanatom-ical framework?
1.2 0.4 2.4 2.0 1.2 0.4 1.0 - 2.2 1.6 1.0 -
Is framework use-ful?
2.4 1.1 3.4 1.5 1.8 1.8 1.0 - 2.4 1.5 1.6 0.9
Facilitates differen-tial diagnosis?
2.4 0.9 3.4 1.3 3.0 1.9 1.4 0.5 2.4 1.7 1.8 0.4
Communication im-proved?
1.0 - 3.4 1.8 2.4 1.5 1.0 - 2.6 1.8 1.2 0.4
Use as overview? 1.2 0.4 2.4 1.7 2.6 1.5 1.0 - 2.4 1.5 1.2 0.4
Table 4.1 Results(twosessions)of theclinical evaluationfor threecases:Case1is a patientwith a right frontal lobetumor, case2 is a patientwith autisticbehavior,and case3 is a TSpatient. A full-lengthversion of the questionscan be foundinthe text andtheanswers couldbeeither1=definitelyyes,2=probablyyes,3=neu-tral, 4=probablyno,or 5=definitelyno. An arithmeticmean(mean)with standarddeviation (s.d.)over all observers wascalculated.In all questionsbut thefirst, theresponse”1” is consideredpositivefor theNormalFusion(NF) technique;in thefirstquestiontheresponse”5” is consideredpositive.
Although the dataarecategorical, we decidedto calculatean arithmeticmeanwith standarddeviation for computationaleaseandto facilitatecomprehension.Thenuclearmedicinephysicianswerenot trainedto interprettheNormalFusionimagesin clinical practice. At first it proved difficult to fully understandand categorizethe informationconveyed by the Normal Fusionimages. After sometraining, theinformation from the Normal Fusionimageswaseasily interpretedand integratedwith theinformationfrom the2D SPECTimages.Especiallyfor cases1 and3 (seeTable1), consistency accrossthenuclearmedicinephysiciansincreasedconsiderablyfrom session1 to 2 while this wasalsotruefor four out of five observersfor case2.Thelessfavorableresultsfor theautisticpatientwerecausedby thedeviatingopinionof oneobserver, who attestedthat theNormalFusionimagefor this caseconveyedclinically irrelevantor incorrectinformation.
Initially, thenuclearmedicinephysiciansstatedthey would usetheNormalFu-sionimagesonly whentheclinical questioncalledfor investigationof corticalactiv-ity. After the training phasethey reportedthey would usethe techniqueto quicklyinvestigatecorticalactivity evenif theclinical questiondid notpoint to thecortex astheprimarysiteof interest.Severalof theobserversreporteda desireto manipulatethecolorencodingscaleof theNormalFusionimagesfor animprovedunderstandingof thedata.
The cliniciansappreciateddirect (3D) visual comparisonwith the homologousregion of the other hemisphere,especiallywhen the 2D SPECTimageswere not
52 NormalFusionfor 3D IntegratedVisualizationof SPECTandMR Brain Images
properlyaligned.Then,investigationof the2D SPECTimagesprovedproblematic,becausecomparisonof activity with themirror hemispherewasdifficult. TheNormalFusiontechniqueis not affectedby misalignmentof theoriginal2D SPECTimages,becauseof theprovidedanatomicalframework,whichalleviatesthecomparisonwiththepertinentanatomicalregion.
Overall, theresultsof session2 (seeTable1) indicatethat theobserversconsid-eredtransferof informationfrom theNormalFusionimagesto the2D SPECTdataeasy. The techniqueprovidesan anatomicalframework which may help not onlyin establishingthedifferentialdiagnosis,but alsoin communicatingto thereferringclinician.
4.6 Conclusions
Thereis a growing needfor multimodality visualizationtools in the clinic to gainmoreinsightinto theintricateinformationconveyedby multimodaldatasets.Useof3D multimodalityvisualizationtechniquesfor SPECTandMRI mayfacilitatediag-nosisandcommunicationby increasingtheappreciationof thespatialrelationshipsof theimages.
We developeda novel methodfor theintegrated3D visualizationof informationacquiredwith SPECTandMRI. Thismethod,NormalFusion,calculatestheregionalbloodperfusionbeneaththesurfaceandcolor encodesthis valueontotheMRI cor-tex rendering. Functionalinformationof the surfacelayer of cortical grey matteris presentedwithin ananatomicalframeof reference.Thecurvatureof thebrain isfollowed,whichmakesthevisualizationindependentof theviewing direction.
Experiencewith this techniqueusing clinical datasetsis promising. Informa-tion thatwasdifficult to find whendiagnosingSPECTfrom theindividualsliceswasbroughtout by the multimodal presentations.A simple clinical evaluationof theNormal Fusiontechniquewasconducted,which indicatedthat communicationandanatomicallocalizationmay well benefitfrom this technique.The promisingeval-uation resultscall for a rigorousvalidationof the diagnosticvalue of the NormalFusiontechnique,or rathermoregenerallyof simultaneousdisplayof functionalandanatomicalinformation.
Acknowledgements
We areindebtedto our colleaguesP.C. Anema,P.C. van Barneveld, F.J. Beekman,J. Buitelaar, W.I. de Bruin, E. Buskens,P.A. van den Elsen,J.H. de Groot, J.W.van Isselt, R.T.M. Jonk, J.M.H. de Klerk, J.B.A. Maintz, L.C. Meiners,M. Met-selaar, andG.R. Timmensfor their contributions. We gratefully acknowledgetheresearchlicenceof ANALYZETM, provided by Dr R.A. Robb,Mayo Foundation/Clinic, Rochester, Minnesota.
