Monthly Program UpdateMarch 8, 2012

Andrew J. Buckler, MSPrincipal Investigator

WITH FUNDING SUPPORT

PROVIDED BY NATIONAL

INSTITUTE OF STANDARDS AND

TECHNOLOGY

Agenda

• Summary and close-out of the „Winter 2012“ development iteration– Covering what’s been accomplished from

multiple points of view

• Preview of „Spring 2012“ development iteration– With focus on directions in

StudyDescription and „ISA“ storage model, evaluation of workflow engine.

22

3

Winter 2012 (n=47)Autumn 2011 (n=19)

Spring 2012 (n=32) Unstaged (n=19)

Overal

l

Speci

fy

Form

ulate

Execu

te

Analyze

Packag

e

Studies

02468

10121416

OpenResolved

Overal

l

Speci

fy

Form

ulate

Execu

te

Analyze

Packag

e

Studies

Unresolve

d02468

10121416

OpenResolved

Overal

l

Speci

fy

Form

ulate

Execu

te

Analyze

Packag

e

Studies

02468

10121416

OpenResolved

Overal

l

Speci

fy

Form

ulate

Execu

te

Analyze

Packag

e

Studies

02468

10121416

OpenResolved

333

• Ramp-up of formal development environment, (including issue tracking)

• Initial Specify (including QIBO and Knowledgebase)

• Major update to Execute: Metadata extraction Better Batchmake GUI

• Initial Formulate• Specify now creates

instances in knowledgebase

• Change studies• Scripted reader studies• Export to Analyze• Import from Formulate• Evaluate workflow

application “Iterate”

• Clojure DSL for executable specifications

• Major update to Analyze• RDF-compliance in Specify• Formulate using SPARQL• Service APIs for Iterate

3A Pilot

3A Pivotal

ISA Storage Model

Analyze project

Specify/ Formulate

project

User: Lab Protocol• Develop and run queries based on data

requirements – Use of Formulate

• Load Reference Data into the Reference Data Set Manager

– Example Pilot3A Data Processing Steps

• Server-Side Processing using the Batch Analysis Service

– Package Algorithm or Method using Batch Analysis Service API

– Prepare Data Set • Create Ground Truth or other Reference Annotation

and Markup • Importing location points and other data for use

– Writing Scripts – Initiate a Batch Analysis Run

• Perform statistical analysis – Analyze Use Instructions

Developer: Design Documents• User Needs and Requirements Analysis • Architecture • Application-specific Design

– Specify • "Specify" Scope Description (ASD) • "Specify" Architecture Specification (AAS) • "Quantitative Imaging Biomarker Ontology (QIBO)" Softwa

re Design Document (SDD)

• "Biomarker DB" (a.k.a., the triple store) Software Design Document (SDD)

• AIM Template Builder Design Documentation:

– Formulate • "Formulate" Scope Description (ASD) • "Formulate" Architecture Specification (AAS) • "NBIA Connector" Software Design Document (SDD)

– Execute • "Execute" Scope Description (ASD) • "Execute" Architecture Specification (AAS) • Reference Data Set Manager (RDSM) Software Design Doc

ument (SDD)

• Batch Analysis Service Software Design Document (SDD)

– Analyze • "Analyze" Scope Description (ASD) • "Analyze" Architecture Specification (AAS)

– Package • "Package" Scope Description (ASD) • "Package" Architecture Specification (AAS)

4444

(Form of) Early Analysis Results

3A Challenge Series1. Median Technologies2. Vital Images, Inc.3. Fraunhofer Mevis4. Siemens5. Moffitt Cancer Center6. Toshiba

5555

Pilot

Pivotal

Investigation 1

Train

Test

Pilot

Pivotal

Investigation

Train

Test

Pilot

Pivotal

Investigation

Train

Test

Pilot

Pivotal

Investigation n

Train

Test

Pr im

ar y

Seco n

dar y

• Defined set of data• Defined challenge• Defined test set policy

First Participants7. GE Healthcare8. Icon Medical Imaging9. Columbia University10. INTIO, Inc.11. Vital Images, Inc.

Standardized Representation of Quantitative Imaging

Statistical Validation Services for Quantitative Imaging

66666

OK. Now into the details for Spring 2012 Iteration: Starting with what we said in January…

7777

Formulate

Statistical Analysis Results (Relation

strength)

Annotation and Image Markup,

Non-imaging Clinical Data

Primary Data: Images and other

Raw Data

Reference Data SetsQIBO

Specify

RDF Triple Store

CT Volumetry CT

obtained_by

Tumor growth

measure_of

TherapeuticEfficacy

used_for

Analyze

Y=β0..n+β1(QIB)+β2T+ eij

Execute

Feedbac k

Feed

bac

k

…and where we left off in February…// Business RequirementsFNIH, QIBA, and C-Path participants don’t have a way to provide precise specification for

context for use and applicable assay methods (to allow semantic labeling):BiomarkerDB = Specify (biomarker domain expertise, ontology for labeling);

Researchers and consortia don’t have an ability to exploit existing data resources with high precision and recall:ReferenceDataSet+ = Formulate (BiomarkerDB, {DataService} );

Technology developers and contract research organizations don’t have a way to do large-scale quantitative runs:ReferenceDataSet .CollectedValue+ = Execute (ReferenceDataSet.RawData);

The community lacks way to apply definitive statistical analyses of annotation and image markup over specified context for use:BiomarkerDB.SummaryStatistic+ = Analyze ( { ReferenceDataSet .CollectedValue } );

Industry lacks standardized ways to report and submit data electronically:efiling transactions+ = Package (BiomarkerDB, {ReferenceDataSet} );

888888

Rotate it to align with the horizontal rather than vertical presentation of our splash screen…

// B

usin

ess

Requ

irem

ents

FNIH

, QIB

A, a

nd C

-Pat

h pa

rtici

pant

s do

n’t

have

a w

ay to

pro

vide

pre

cise

sp

ecifi

catio

n fo

r con

text

for u

se a

nd

appl

icab

le a

ssay

met

hods

(to

allo

w

sem

antic

labe

ling)

:Bi

omar

kerD

B =

Spec

ify (b

iom

arke

r do

mai

n ex

perti

se, o

ntol

ogy

for l

abel

ing)

;Re

sear

cher

s an

d co

nsor

tia d

on’t

have

an

abili

ty to

exp

loit

existi

ng d

ata

reso

urce

s w

ith h

igh

prec

isio

n an

d re

call:

Refe

renc

eDat

aSet

+ =

Form

ulat

e (B

iom

arke

rDB,

{Dat

aSer

vice

} );

Tech

nolo

gy d

evel

oper

s an

d co

ntra

ct re

sear

ch

orga

niza

tions

don

’t ha

ve a

way

to d

o la

rge-

scal

e qu

antit

ative

runs

:Re

fere

nceD

ataS

et .C

olle

cted

Valu

e+ =

Ex

ecut

e (R

efer

ence

Dat

aSet

.Raw

Dat

a);

The

com

mun

ity la

cks

way

to a

pply

defi

nitiv

e st

atisti

cal a

naly

ses

of a

nnot

ation

and

im

age

mar

kup

over

spe

cifie

d co

ntex

t for

us

e:Bi

omar

kerD

B.Su

mm

aryS

tatis

tic+

= An

alyz

e ( {

Ref

eren

ceD

ataS

et .C

olle

cted

Valu

e } )

;In

dust

ry la

cks

stan

dard

ized

way

s to

repo

rt

and

subm

it da

ta e

lect

roni

cally

:efi

ling

tran

sacti

ons+

= P

acka

ge

(Bio

mar

kerD

B, {R

efer

ence

Dat

aSet

} );

999999

…to arrive at a new more complete view(interpreting the braces as a separate application)

// B

usin

ess

Requ

irem

ents

FNIH

, QIB

A, a

nd C

-Pat

h pa

rtici

pant

s do

n’t

have

a w

ay to

pro

vide

pre

cise

sp

ecifi

catio

n fo

r con

text

for u

se a

nd

appl

icab

le a

ssay

met

hods

(to

allo

w

sem

antic

labe

ling)

:Bi

omar

kerD

B =

Spec

ify (b

iom

arke

r do

mai

n ex

perti

se, o

ntol

ogy

for l

abel

ing)

;Re

sear

cher

s an

d co

nsor

tia d

on’t

have

an

abili

ty to

exp

loit

existi

ng d

ata

reso

urce

s w

ith h

igh

prec

isio

n an

d re

call:

Refe

renc

eDat

aSet

+ =

Form

ulat

e (B

iom

arke

rDB,

{Dat

aSer

vice

} );

Tech

nolo

gy d

evel

oper

s an

d co

ntra

ct re

sear

ch

orga

niza

tions

don

’t ha

ve a

way

to d

o la

rge-

scal

e qu

antit

ative

runs

:Re

fere

nceD

ataS

et .C

olle

cted

Valu

e+ =

Ex

ecut

e (R

efer

ence

Dat

aSet

.Raw

Dat

a);

The

com

mun

ity la

cks

way

to a

pply

defi

nitiv

e st

atisti

cal a

naly

ses

of a

nnot

ation

and

im

age

mar

kup

over

spe

cifie

d co

ntex

t for

us

e:Bi

omar

kerD

B.Su

mm

aryS

tatis

tic+

= An

alyz

e ( {

Ref

eren

ceD

ataS

et .C

olle

cted

Valu

e } )

;In

dust

ry la

cks

stan

dard

ized

way

s to

repo

rt

and

subm

it da

ta e

lect

roni

cally

:efi

ling

tran

sacti

ons+

= P

acka

ge

(Bio

mar

kerD

B, {R

efer

ence

Dat

aSet

} );

101010101010

Worked Example (starting from claim analysis we discussed in February 2011)

Measurements of tumor volume are more precise (reproducible) than uni-dimensional tumor measurements of tumor diameter. Longitudinal changes in whole tumor volume during therapy predict clinical outcomes (i.e., OS or PFS) earlier than corresponding uni-dimensional measurements. Therefore, tumor response or progression as determined by tumor volume will be able to serve as the primary endpoint in well-controlled Phase II and III efficacy studies of cytotoxic and selected targeted therapies (e.g., antiangiogenic agents, tyrosine kinase inhibitors, etc.) in several solid, measurable tumors (including both primary and metastatic cancers of, e.g., lung, liver, colorectal, gastric, head and neck cancer,) and lymphoma. Changes in tumor volume can serve as the endpoint for regulatory drug approval in registration trials.

Biomarker claim statements are information-rich and may be used to set up the needed analyses.

111111111111

The user enters information from claiminto the knowledgebase using SpecifyMeasurements of tumor volume are more precise (reproducible) than uni-dimensional tumor measurements of tumor diameter. Longitudinal changes in whole tumor volume during therapy predict clinical outcomes (i.e., OS or PFS) earlier than corresponding uni-dimensional measurements. Therefore, tumor response or progression as determined by tumor volume will be able to serve as the primary endpoint in well-controlled Phase II and III efficacy studies of cytotoxic and selected targeted therapies (e.g., antiangiogenic agents, tyrosine kinase inhibitors, etc.) in several solid, measurable tumors (including both primary and metastatic cancers of, e.g., lung, liver, colorectal, gastric, head and neck cancer,) and lymphoma. Changes in tumor volume can serve as the endpoint for regulatory drug approval in registration trials.

Subject Predicate Object

CT images Tumor

Volumetry analyzes CT

LongitudinalVolumetry

estimates TumorSizeChange

TumorSizeChange

predicts TreatmentResponse

Categoric

Continuous

Continuous

1212

…pulling various pieces of information,

Measurements of tumor volume are more precise (reproducible) than uni-dimensional tumor measurements of tumor diameter. Longitudinal changes in whole tumor volume during therapy predict clinical outcomes (i.e., OS or PFS) earlier than corresponding uni-dimensional measurements. Therefore, tumor response or progression as determined by tumor volume will be able to serve as the primary endpoint in well-controlled Phase II and III efficacy studies of cytotoxic and selected targeted therapies (e.g., antiangiogenic agents, tyrosine kinase inhibitors, etc.) in several solid, measurable tumors (including both primary and metastatic cancers of, e.g., lung, liver, colorectal, gastric, head and neck cancer,) and lymphoma. Changes in tumor volume can serve as the endpoint for regulatory drug approval in registration trials.

Subject Predicate Object

CT images Tumor

Volumetry analyzes CT

<compliant>LongitudinalVolumetry

estimates TumorSizeChange

TumorSizeChange predicts CytotoxicTreatmentResponse

TyrosineKinaseInhibitor

is CytotoxicTreatment

well-controlled Phase II and III efficacy studies

uses CytotoxicTreatmentResponse

CytotoxicTreatment

influences NonSmallCellLungCancer

CT images Thorax

Thorax contains NonSmallCellLungCancer

Intervention

Target Indication

1313

…to form the specification.

Measurements of tumor volume are more precise (reproducible) than uni-dimensional tumor measurements of tumor diameter. Longitudinal changes in whole tumor volume during therapy predict clinical outcomes (i.e., OS or PFS) earlier than corresponding uni-dimensional measurements. Therefore, tumor response or progression as determined by tumor volume will be able to serve as the primary endpoint in well-controlled Phase II and III efficacy studies of cytotoxic and selected targeted therapies (e.g., antiangiogenic agents, tyrosine kinase inhibitors, etc.) in several solid, measurable tumors (including both primary and metastatic cancers of, e.g., lung, liver, colorectal, gastric, head and neck cancer,) and lymphoma. Changes in tumor volume can serve as the endpoint for regulatory drug approval in registration trials.

To produce data for

registration

To substantiate quality of evidence

development

Subject Predicate Object

CT images Tumor

Volumetry analyzes CT

<compliant>LongitudinalVolumetry

estimates TumorSizeChange

TumorSizeChange predicts CytotoxicTreatmentResponse

TyrosineKinaseInhibitor is CytotoxicTreatment

well-controlled Phase II and III efficacy studies

uses CytotoxicTreatmentResponse

CytotoxicTreatment influences NonSmallCellLungCancer

CT images Thorax

Thorax contains NonSmallCellLungCancer

regulatory drug approval dependsOn PrimaryEndpoint

well-controlled Phase II and III efficacy studies

assess PrimaryEndpoint

CT Volumetry is <putative>SurrogateEndpoint

1414

Formulate interprets the specification as testable hypotheses,Measurements of tumor volume are more precise (reproducible) than uni-dimensional tumor measurements of tumor diameter. Longitudinal changes in whole tumor volume during therapy predict clinical outcomes (i.e., OS or PFS) earlier than corresponding uni-dimensional measurements. Therefore, tumor response or progression as determined by tumor volume will be able to serve as the primary endpoint in well-controlled Phase II and III efficacy studies of cytotoxic and selected targeted therapies (e.g., antiangiogenic agents, tyrosine kinase inhibitors, etc.) in several solid, measurable tumors (including both primary and metastatic cancers of, e.g., lung, liver, colorectal, gastric, head and neck cancer,) and lymphoma. Changes in tumor volume can serve as the endpoint for regulatory drug approval in registration trials.

Type of biomarker, in this case predictive (could have been

something else, e.g., prognostic), to establish the mathematical

formalism

Technical characteri

stic

Subject Predicate Object

CT images Tumor

Volumetry analyzes CT

<compliant>LongitudinalVolumetry

estimates TumorSizeChange

TumorSizeChange predicts CytotoxicTreatmentResponse

TyrosineKinaseInhibitor is CytotoxicTreatment

well-controlled Phase II and III efficacy studies

uses CytotoxicTreatmentResponse

CytotoxicTreatment influences NonSmallCellLungCancer

CT images Thorax

Thorax contains NonSmallCellLungCancer

regulatory drug approval dependsOn PrimaryEndpoint

well-controlled Phase II and III efficacy studies

assess PrimaryEndpoint

CT Volumetry is <proven>SurrogateEndpoint

1

3

2

1515

…setting up an investigation (I), study (S), assay (A) hierarchy…Subject Predicate Object

CT images Tumor

Volumetry analyzes CT

<compliant>LongitudinalVolumetry

estimates TumorSizeChange

TumorSizeChange predicts CytotoxicTreatmentResponse

TyrosineKinaseInhibitor is CytotoxicTreatment

well-controlled Phase II and III efficacy studies

uses CytotoxicTreatmentResponse

CytotoxicTreatment influences NonSmallCellLungCancer

CT images Thorax

Thorax contains NonSmallCellLungCancer

regulatory drug approval dependsOn PrimaryEndpoint

well-controlled Phase II and III efficacy studies

assess PrimaryEndpoint

CT Volumetry is <putative>SurrogateEndpoint

1

3

2

1616

Investigations to Prove the Hypotheses:1. Technical Performance = Biological

Target + Assay Method2. Clinical Validity = Indicated Biology

+ Technical Performance3. Clinical Utility = Biomarker Use +

Clinical Validity

Investigation-Study-Assay Hierarchy:• Investigation = {Summary Statistic} +

{Study}• Study = {Descriptive Statistic} +

Protocol + {Assay}• Assay = RawData + {AnnotationData}• AnnotationData = [AIM file|mesh|…]

…ADDING TRIPLES TO CAPTURE URIs:Subject Predicate Object

ClinicalUtility is Investigation URI

ClinicalValidity is Investigation URI

TechnicalPerformance is Investigation URI

Investigation has SummaryStatisticType

Investigation has Study URI

Study has DescriptiveStatisticType

Study has Protocol URI

Study has Assay URI

Assay has RawData URI

…and loading data into Execute (at least raw data, possibly annotations if they already exist)

Subject Predicate Object

A Is Patient

A isDiagnosedWith DiseaseA

DiseaseA Is NonSmallLCellLunCancer

Pazopanib Is TyrosoineKinaseInhibitor

A hasBaseline CT

A hasTP1 CT

A hasTP2 CT

B isDiagnosedWith DiseaseA

B hasBaseline CT

B hasTP1 CT

A hasOutcome Death

B hasOutcome Survival

1717

DISCOVERED DATA: …LOADING DATA INTO THE RDSM:

Reference Data Set Manager:

Heavyweight Storage with URIs

Knowledgebase:Lightweight

Storage linking to URIs

If no annotations, Execute creates them(in either case leaving Analyze with its data set up for it)

Subject Predicate Object

ClinicalUtility is Investigation URI

ClinicalValidity is Investigation URI

TechnicalPerformance is Investigation URI

Investigation has SummaryStatisticType

Investigation has Study URI

Study has DescriptiveStatisticType

Study has Protocol URI

Study has Assay URI

Assay has RawData URI

Assay has AnnotationData URI

AIM file is AnnotationData URI

Mesh is AnnotationData URI

1818

Either in batch or viaScripted reader studies

(using “Share” and “Duplicate” functions of RDSM to leverage cases across investigations)

(self-generating knowledgebase from RDSM hierarchy and ISA-TAB description files)

Reference Data Set Manager:

Heavyweight Storage with URIs

Knowledgebase:Lightweight

Storage linking to URIs

Analyze performs the statistical analyses…

Subject Predicate Object

A Is Patient

A isDiagnosedWith DiseaseA

DiseaseA Is NonSmallLCellLunCancer

A hasClinicalObservation

B

B Is TumorShrinkage

C Is Patient

C hasClinicalObservation

B

D hasClinicalObservation

B

Pazopanib Is TyrosoineKinaseInhibitor

A isTreatedWith Pazopanib

A hasOutcome Death

C hasOutcome Survival

Subject Predicate Object

CT images Tumor

Volumetry analyzes CT

<compliant>LongitudinalVolumetry

estimates TumorSizeChange

TumorSizeChange predicts CytotoxicTreatmentResponse

TyrosoineKinaseInhibitor is CytotoxicTreatment

well-controlled Phase II and III efficacy studies

uses CytotoxicTreatmentResponse

CytotoxicTreatment influences NonSmallCellLungCancer

CT images Thorax

Thorax contains NonSmallCellLungCancer

regulatory drug approval dependsOn PrimaryEndpoint

well-controlled Phase II and III efficacy studies

assess PrimaryEndpoint

CT Volumetry is SurrogateEndpoint for CytotoxicTreatment

1

3

2

1919

…and adds the results to the knowledgebase (using W3C “best practices” for “relation strength”).

Subject Predicate Object

45324 biasMethod <r script used>

45324 bias <summary statistic>

45324 variabilityMethod <r script used>

45324 variability <summary statistic>

9956 <correlation>Method <r script used>

9956 correlation <summary statistic>

9956 <ROC>Method <r script used>

9956 ROC <summary statistic>

98234 Effect of treatment on true endpoint <value>

98234 Effect of treatment on surrogate endpoint <value>

98234 Effect of surrogate on true endpoint <value>

98234 Effect of treatment on true endpoint relative to that on surrogate endpoint

<value>

Subject Predicate Object

CT images Tumor

Volumetry analyzes CT

<compliant>LongitudinalVolumetry

estimates TumorSizeChange

TumorSizeChange predicts CytotoxicTreatmentResponse

TyrosoineKinaseInhibitor is CytotoxicTreatment

well-controlled Phase II and III efficacy studies

uses CytotoxicTreatmentResponse

CytotoxicTreatment influences NonSmallCellLungCancer

CT images Thorax

Thorax contains NonSmallCellLungCancer

regulatory drug approval dependsOn PrimaryEndpoint

well-controlled Phase II and III efficacy studies

assess PrimaryEndpoint

CT Volumetry is SurrogateEndpoint for CytotoxicTreatment

1

3

2

URI=45324

URI=9956

URI=98234

2020

PackageStructure submissions according to eCTD, HL7 RCRIM, and SDTM

Section 2 Summaries2.1. Biomarker Qualification Overview2.1.1. Introduction2.1.2. Context of Use2.1.3. Summary of Methodology and Results2.1.4. Conclusion2.2. Nonclinical Technical Methods Data2.2.1. Summary of Technical Validation Studies and Analytical Methods2.2.2. Synopses of individual studies2.3. Clinical Biomarker Data2.3.1. Summary of Biomarker Efficacy Studies and Analytical Methods2.3.2. Summary of Clinical Efficacy [one for each clinical context]2.3.3. Synopses of individual studies

Section 3 Quality<used when individual sponsor qualifies marker in a specific NDA>

Section 4 Nonclinical Reports4.1. Study reports4.1.1. Technical Methods Development Reports4.1.2. Technical Methods Validation Reports4.1.3. Nonclinical Study Reports (in vivo)4.2. Literature references

Section 5 Clinical Reports5.1. Tabular listing of all clinical studies5.2. Clinical study reports and related information5.2.1. Technical Methods Development reports5.2.2. Technical Methods Validation reports5.2.3. Clinical Efficacy Study Reports [context for use]5.3. Literature references

21

Subject Predicate Object

45324 biasMethod <r script used>

45324 bias <summary statistic>

45324 variabilityMethod <r script used>

45324 variability <summary statistic>

9956 <correlation>Method <r script used>

9956 correlation <summary statistic>

9956 <ROC>Method <r script used>

9956 ROC <summary statistic>

98234 Effect of treatment on true endpoint <value>

98234 Effect of treatment on surrogate endpoint <value>

98234 Effect of surrogate on true endpoint <value>

98234 Effect of treatment on true endpoint relative to that on surrogate endpoint

<value>

Iterate: Reproducible Workflows with Documented Provenance (with illustration expansion of databases)

22222222222222

Knowledgebase Triples

N N+500 N+1000 N+1000 N+2000 -> eCTD

Reference Data Sets

M M M+1000 M+10,000 M+10,000 -> eCTD

2323

Value proposition of QI-Bench• Efficiently collect and exploit evidence establishing

standards for optimized quantitative imaging:– Users want confidence in the read-outs– Pharma wants to use them as endpoints– Device/SW companies want to market products that produce them

without huge costs– Public wants to trust the decisions that they contribute to

• By providing a verification framework to develop precompetitive specifications and support test harnesses to curate and utilize reference data

• Doing so as an accessible and open resource facilitates collaboration among diverse stakeholders

2424

Summary:QI-Bench Contributions• We make it practical to increase the magnitude of data for increased

statistical significance. • We provide practical means to grapple with massive data sets.• We address the problem of efficient use of resources to assess limits of

generalizability. • We make formal specification accessible to diverse groups of experts that are

not skilled or interested in knowledge engineering. • We map both medical as well as technical domain expertise into

representations well suited to emerging capabilities of the semantic web. • We enable a mechanism to assess compliance with standards or

requirements within specific contexts for use.• We take a “toolbox” approach to statistical analysis. • We provide the capability in a manner which is accessible to varying levels of

collaborative models, from individual companies or institutions to larger consortia or public-private partnerships to fully open public access.

2525

QI-BenchStructure / Acknowledgements• Prime: BBMSC (Andrew Buckler, Gary Wernsing, Mike Sperling, Matt Ouellette)

• Co-Investigators– Kitware (Rick Avila, Patrick Reynolds, Julien Jomier, Mike Grauer)– Stanford (David Paik)

• Financial support as well as technical content: NIST (Mary Brady, Alden Dima, John Lu)

• Collaborators / Colleagues / Idea Contributors– Georgetown (Baris Suzek)– FDA (Nick Petrick, Marios Gavrielides) – UMD (Eliot Siegel, Joe Chen, Ganesh Saiprasad, Yelena Yesha)– Northwestern (Pat Mongkolwat)– UCLA (Grace Kim)– VUmc (Otto Hoekstra)

• Industry– Pharma: Novartis (Stefan Baumann), Merck (Richard Baumgartner)– Device/Software: Definiens, Median, Intio, GE, Siemens, Mevis, Claron Technologies, …

• Coordinating Programs– RSNA QIBA (e.g., Dan Sullivan, Binsheng Zhao)– Under consideration: CTMM TraIT (Andre Dekker, Jeroen Belien)

2626