QIBA CT Volumetrics - QIBA CT Volumetrics - Cross-Platform Study (Group 1C) Cross-Platform Study (Group 1C)

December 23, 2009December 23, 2009

CT Cross-platform Sizing of Phantom LesionsCT Cross-platform Sizing of Phantom Lesions

Quantitative Imaging Biomarker AllianceQuantitative Imaging Biomarker Alliance

Charles Fenimore, NISTCharles Fenimore, NIST

ChargeCharge1.1. To agree on the To agree on the scanner settings and other scanner settings and other

protocol elementsprotocol elements under which imagery is to under which imagery is to be collected.be collected.

2.2. To agree on requirements of To agree on requirements of phantomsphantoms to be to be imaged/measured.imaged/measured.

3.3. To agree on the To agree on the platforms and centersplatforms and centers to be to be selected for imagery collection.selected for imagery collection.

4.4. To agree on the site(s) and methods for the To agree on the site(s) and methods for the reader studyreader study..

5.5. To identify the To identify the measurements and the measurements and the algorithmsalgorithms for use in image processing. for use in image processing.

6.6. To To specify the analysisspecify the analysis of the measurements. of the measurements.

GoalsGoals

1.1. Measure the volume of nodules on CT imagery collected Measure the volume of nodules on CT imagery collected from several CT scanners and sites. Protocol is to be from several CT scanners and sites. Protocol is to be defined in relation to QIBA Vol CT 2 specifications. Protocol defined in relation to QIBA Vol CT 2 specifications. Protocol is to include a branch specifying performance.is to include a branch specifying performance.

2.2. Measure image noise and other image quality factors and Measure image noise and other image quality factors and determine their impact on the measurement of volume.determine their impact on the measurement of volume.

3.3. Analyze the Analyze the accuracy and precisionaccuracy and precision of volume of volume measurements for all design factors including: site, device, measurements for all design factors including: site, device, scanner settings, & reader/algorithm/software.scanner settings, & reader/algorithm/software.

4.4. Determine the Determine the minimum detectable level of changeminimum detectable level of change that that can be achieved when measuring nodules in can be achieved when measuring nodules in phantom phantom datasetsdatasets..

Goals 1 and 2Goals 1 and 2

1.1. Measure the volume of nodules on CT Measure the volume of nodules on CT imagery collected from several CT imagery collected from several CT scanners and sites, etc.scanners and sites, etc.

2.2. Measure “image noise” and determine its Measure “image noise” and determine its impact on the measurement of volume, impact on the measurement of volume, to facilitate scanner comparison.to facilitate scanner comparison.

Realization of these goals is discussed in the next few slides.Realization of these goals is discussed in the next few slides.

QIBA CT 1-C ProtocolQIBA CT 1-C Protocol

AcquisitionAcquisition– The phantomThe phantom– Image datasetsImage datasets– The imaging sitesThe imaging sites– System parametersSystem parameters– Markup sitesMarkup sites

QIBA CT 1-C Acquisition ProtocolQIBA CT 1-C Acquisition ProtocolPhantom: FDA lung & NIST pocket phantomsPhantom: FDA lung & NIST pocket phantoms– The FDA phantom accomodates 8 attached nodules, some are 20 The FDA phantom accomodates 8 attached nodules, some are 20

mm in size. A typical configuration might be:mm in size. A typical configuration might be: Left: 10 & 20 mm spheres @ - 10 & +100 HULeft: 10 & 20 mm spheres @ - 10 & +100 HU Right: 10 & 20 mm spiculated @ - 10 & +100 HU, 10 mm ovoid, Right: 10 & 20 mm spiculated @ - 10 & +100 HU, 10 mm ovoid,

lobulated, spiculatedlobulated, spiculated

Options: ovoid & lobulated nodules are also available. 9-mm spheres Options: ovoid & lobulated nodules are also available. 9-mm spheres are coming, providing graded sizes (5, 8, 9, 10 mm).are coming, providing graded sizes (5, 8, 9, 10 mm).

– The The NIST pocket phantom is a free standing Lego structure holding NIST pocket phantom is a free standing Lego structure holding teflon balls (6 mm in diameter, > 1000 HU). It is to be attached to teflon balls (6 mm in diameter, > 1000 HU). It is to be attached to the exterior of the lung phantom. Specific request to include high the exterior of the lung phantom. Specific request to include high contrast (+100 HU) 20 mm sphere in interior. These two phantoms contrast (+100 HU) 20 mm sphere in interior. These two phantoms provide information about the system step response.provide information about the system step response.

Potential imaging & reading sitesPotential imaging & reading sitesPotential imaging sitesPotential imaging sites

– FDA - Philips 16 detector rowFDA - Philips 16 detector row– Nick Petrick / Marios Gavrelides, contactNick Petrick / Marios Gavrelides, contact

– Duke – GE Lightspeed 16 and 64Duke – GE Lightspeed 16 and 64– Ehsan Samei, Division Chief, Clinical Imaging PhysicsEhsan Samei, Division Chief, Clinical Imaging Physics– 1 GE LightSpeed VCT (64 slice), 4 GE LightSpeed 16 (16 slice, 1 with 1 GE LightSpeed VCT (64 slice), 4 GE LightSpeed 16 (16 slice, 1 with

fluoroscopic capability)fluoroscopic capability)

– Johns Hopkins – Siemens, ToshibaJohns Hopkins – Siemens, Toshiba– Mahadevappa Mahesh, Chief PhysicistMahadevappa Mahesh, Chief Physicist

– UCLA UCLA – Mike McNitt Gray, medical physicistMike McNitt Gray, medical physicist

– UM-MC, Philips 64UM-MC, Philips 64– Eliot Siegel / Joseph Chen, radiologistsEliot Siegel / Joseph Chen, radiologists

– Siemens (Germany), offer of multiple scannersSiemens (Germany), offer of multiple scanners– Mathias ThornMathias Thorn

Potential reading sites, CROsPotential reading sites, CROs– Rad Pharm – Initiated discussion of markupRad Pharm – Initiated discussion of markup– Additional sites: Perceptive Informatics, Biomedical SystemsAdditional sites: Perceptive Informatics, Biomedical Systems

QIBA CT 1-C ProtocolQIBA CT 1-C Protocol11stst branch = a standard protocol branch = a standard protocol

Scan 2 - 3 times. No more the one or two are to be Scan 2 - 3 times. No more the one or two are to be read.read.

Use the developing QIBA protocol (akin to ACRIN Use the developing QIBA protocol (akin to ACRIN 6678) to specify kVp, slice thickness, mAs, 6678) to specify kVp, slice thickness, mAs, rotation time, pitch, reconstruction kernel (affects rotation time, pitch, reconstruction kernel (affects MTF). MTF). Include water and ACR phantoms Include water and ACR phantoms to to characterize the resolution and noise levels under characterize the resolution and noise levels under this branch of the protocol.this branch of the protocol.

Sample Protocol Chart for ACRIN 6678Sample Protocol Chart for ACRIN 6678

QIBA CT 1-C ProfilesQIBA CT 1-C Profiles22ndnd branch = performance specified branch = performance specified

Specify easily implemented PERFORMANCE metrics of Specify easily implemented PERFORMANCE metrics of resolution and noise.resolution and noise.

Guidance provided on:Guidance provided on: kVp (affects contrast difference between materials)kVp (affects contrast difference between materials) Slice thickness, recon interval (affects z-axis resolution & Slice thickness, recon interval (affects z-axis resolution &

noise)noise) Rotation time and pitch (coverage, breath hold, etc.)Rotation time and pitch (coverage, breath hold, etc.)

Performance required on:Performance required on: Recon kernel performance - EXAMPLE: Recon kernel performance - EXAMPLE:

– Choose kernel such that you can see 6 or 7 (but no more than 7) Choose kernel such that you can see 6 or 7 (but no more than 7) lp/cm on ACR phantom….orlp/cm on ACR phantom….or

– 10% MTF should be between 6 and 7 lp/cm10% MTF should be between 6 and 7 lp/cm mA level performancemA level performance

– Choose effective mAs level so that std dev is between 20 and 30 Choose effective mAs level so that std dev is between 20 and 30 HU in a 20 cm water phantomHU in a 20 cm water phantom

QIBA CT 1-C Markup ProcedureQIBA CT 1-C Markup Procedure

Adapt the Rad-Pharm process used in QIBA Adapt the Rad-Pharm process used in QIBA volume-CT 1-A and 1-B to acquire RECIST volume-CT 1-A and 1-B to acquire RECIST (1D), WHO (2D), & volume (3D) reads.(1D), WHO (2D), & volume (3D) reads. 5 - 6 readers5 - 6 readers 2 reads for each nodules (one repeat).2 reads for each nodules (one repeat).

– Can we avoid repeat reads for the full set of series?Can we avoid repeat reads for the full set of series? The 1A study may provide prior estimates of intra- The 1A study may provide prior estimates of intra- and inter-rater variability.and inter-rater variability.

– Is it possible to reduce the number of reads?Is it possible to reduce the number of reads?

QIBA CT 1-C ProfileQIBA CT 1-C ProfileNotional sizing of studyNotional sizing of study

4 sites; 4 sites;

2 branches to protocol (QIBA-2 profile & 2 branches to protocol (QIBA-2 profile & performance-based);performance-based);

8 nodules8 nodules

2 – 3 acquisitions, of which 1 or 2 are read2 – 3 acquisitions, of which 1 or 2 are read

2 repeat reads2 repeat reads

Full factorial design implies 256 nodule reads per Full factorial design implies 256 nodule reads per radiologist - too large. Aim for less than full-radiologist - too large. Aim for less than full-factorial design.factorial design.

Goal 3Goal 3

Compare the Compare the accuracy and precisionaccuracy and precision of of radiologists’ measurements of radiologists’ measurements of RECIST and Volume for these RECIST and Volume for these phantom datasets. phantom datasets.

a)a) RECISTRECIST vs. vs. volume.volume.b)b) Investigate Investigate variance & biasvariance & bias..c)c) Inter-system variation.Inter-system variation.d)d) Intra-system variation.Intra-system variation.

Goal 4Goal 4

4. Determine the 4. Determine the minimum detectable minimum detectable level of changelevel of change that can be achieved that can be achieved when measuring nodules in when measuring nodules in phantom phantom datasetsdatasets..

Required resourcesRequired resources

Use of Use of FDA phantom, water and ACR FDA phantom, water and ACR phantoms. Also NIST pocket phantom.phantoms. Also NIST pocket phantom.

Select clinical image Select clinical image collection sitescollection sites through QIBA-CT group: Offers from FDA, through QIBA-CT group: Offers from FDA, Duke, John’s Hopkins. Other possible Duke, John’s Hopkins. Other possible sites: UMMC, MSKCC, Siemens.sites: UMMC, MSKCC, Siemens.

Mark up similar to QIBA CT 1-A Mark up similar to QIBA CT 1-A proceduresprocedures at Rad-Pharm. Generates at Rad-Pharm. Generates

RECIST (1D)RECIST (1D) WHO (2D)WHO (2D) Segmented volume.Segmented volume.

Next stepsNext steps

Refine Questions and Experimental Design. Refine Questions and Experimental Design. – Do we need the first branch, “standard” protocol?Do we need the first branch, “standard” protocol?– Do we want the lesion mask available for study?Do we want the lesion mask available for study?

Complete clinic recruitment. Develop Complete clinic recruitment. Develop protocol with associated medical physicists.protocol with associated medical physicists.

Confirm availability and arrange reading with Confirm availability and arrange reading with Rad-Pharm. Other CROs are possible. Rad-Pharm. Other CROs are possible.

Schedule use of FDA phantom.Schedule use of FDA phantom.