33
“WSI Meets Machine Learning: A Clinicians Perspective Michael Feldman, MD, PhD [email protected] @feldmanm30 Professor University of Pennsylvania Vice Chair Pathology & Laboratory Medicine
ldquoWSI Meets Machine Learning A Clinicians Perspective
Michael Feldman MD PhDfeldmanmmailmedupennedu
feldmanm30Professor University of Pennsylvania
Vice Chair Pathology amp Laboratory Medicine
Conflict Disclosure
SAB Philips Sponsored research SCOPIO
No ConflictshellipNo interest
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Digital Workflow and MLAI
Tissue Slide ScreeningRescreening
Predictive QuantitationIHC
Dx
ResFellowsAttending
MolecularIHC Manual TodayMicroscope
IBRIS- Breast- Lung- Prostate- ENT
Automated quant- Companies
Missed eventsNeg Bx - relook
RadioPathoGenomics
Rescreening
EHR
Digital fellow
Event finding- Ca in Nodes- Ca in tissue- Mitoses- Grading
HistoQC
Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology A Multicenter Blinded Randomized Noninferiority Study of 1992 Cases (Pivotal Study)
The American Journal of Surgical Pathology November 2017
MSK Whole slide imaging equivalency and efficiency study experience at a large academic centerMod Path Feb 2019 Sirintrapun et al
Transition to digital pathology workflowsndash Digital Quality Control is paramountndash Recut and rescan slides immediately before getting to a pathologistndash Cost and efficiency savings
Previously not insurmountablendash Increasingly too time consuming to do manuallyndash Non-reproducible
Unmet Need
We need better quality control of our slides
Slides taken from diagnostic cohort of TCGA-BRCA
HistoQC Properties Fast n=1143 in 466 minutes (24sslide) using 6 cores (11TB) Easy ldquoinstallrdquo (git clone) with minimum dependencies
ndash python-openslide matplotlib numpy scipy skimage sklearn
User interface is a single local html5 + JS file no hosting No specialized hardware No internet connection required Designed to be modular and easily extendible
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Conflict Disclosure
SAB Philips Sponsored research SCOPIO
No ConflictshellipNo interest
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Digital Workflow and MLAI
Tissue Slide ScreeningRescreening
Predictive QuantitationIHC
Dx
ResFellowsAttending
MolecularIHC Manual TodayMicroscope
IBRIS- Breast- Lung- Prostate- ENT
Automated quant- Companies
Missed eventsNeg Bx - relook
RadioPathoGenomics
Rescreening
EHR
Digital fellow
Event finding- Ca in Nodes- Ca in tissue- Mitoses- Grading
HistoQC
Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology A Multicenter Blinded Randomized Noninferiority Study of 1992 Cases (Pivotal Study)
The American Journal of Surgical Pathology November 2017
MSK Whole slide imaging equivalency and efficiency study experience at a large academic centerMod Path Feb 2019 Sirintrapun et al
Transition to digital pathology workflowsndash Digital Quality Control is paramountndash Recut and rescan slides immediately before getting to a pathologistndash Cost and efficiency savings
Previously not insurmountablendash Increasingly too time consuming to do manuallyndash Non-reproducible
Unmet Need
We need better quality control of our slides
Slides taken from diagnostic cohort of TCGA-BRCA
HistoQC Properties Fast n=1143 in 466 minutes (24sslide) using 6 cores (11TB) Easy ldquoinstallrdquo (git clone) with minimum dependencies
ndash python-openslide matplotlib numpy scipy skimage sklearn
User interface is a single local html5 + JS file no hosting No specialized hardware No internet connection required Designed to be modular and easily extendible
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Digital Workflow and MLAI
Tissue Slide ScreeningRescreening
Predictive QuantitationIHC
Dx
ResFellowsAttending
MolecularIHC Manual TodayMicroscope
IBRIS- Breast- Lung- Prostate- ENT
Automated quant- Companies
Missed eventsNeg Bx - relook
RadioPathoGenomics
Rescreening
EHR
Digital fellow
Event finding- Ca in Nodes- Ca in tissue- Mitoses- Grading
HistoQC
Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology A Multicenter Blinded Randomized Noninferiority Study of 1992 Cases (Pivotal Study)
The American Journal of Surgical Pathology November 2017
MSK Whole slide imaging equivalency and efficiency study experience at a large academic centerMod Path Feb 2019 Sirintrapun et al
Transition to digital pathology workflowsndash Digital Quality Control is paramountndash Recut and rescan slides immediately before getting to a pathologistndash Cost and efficiency savings
Previously not insurmountablendash Increasingly too time consuming to do manuallyndash Non-reproducible
Unmet Need
We need better quality control of our slides
Slides taken from diagnostic cohort of TCGA-BRCA
HistoQC Properties Fast n=1143 in 466 minutes (24sslide) using 6 cores (11TB) Easy ldquoinstallrdquo (git clone) with minimum dependencies
ndash python-openslide matplotlib numpy scipy skimage sklearn
User interface is a single local html5 + JS file no hosting No specialized hardware No internet connection required Designed to be modular and easily extendible
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Digital Workflow and MLAI
Tissue Slide ScreeningRescreening
Predictive QuantitationIHC
Dx
ResFellowsAttending
MolecularIHC Manual TodayMicroscope
IBRIS- Breast- Lung- Prostate- ENT
Automated quant- Companies
Missed eventsNeg Bx - relook
RadioPathoGenomics
Rescreening
EHR
Digital fellow
Event finding- Ca in Nodes- Ca in tissue- Mitoses- Grading
HistoQC
Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology A Multicenter Blinded Randomized Noninferiority Study of 1992 Cases (Pivotal Study)
The American Journal of Surgical Pathology November 2017
MSK Whole slide imaging equivalency and efficiency study experience at a large academic centerMod Path Feb 2019 Sirintrapun et al
Transition to digital pathology workflowsndash Digital Quality Control is paramountndash Recut and rescan slides immediately before getting to a pathologistndash Cost and efficiency savings
Previously not insurmountablendash Increasingly too time consuming to do manuallyndash Non-reproducible
Unmet Need
We need better quality control of our slides
Slides taken from diagnostic cohort of TCGA-BRCA
HistoQC Properties Fast n=1143 in 466 minutes (24sslide) using 6 cores (11TB) Easy ldquoinstallrdquo (git clone) with minimum dependencies
ndash python-openslide matplotlib numpy scipy skimage sklearn
User interface is a single local html5 + JS file no hosting No specialized hardware No internet connection required Designed to be modular and easily extendible
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology A Multicenter Blinded Randomized Noninferiority Study of 1992 Cases (Pivotal Study)
The American Journal of Surgical Pathology November 2017
MSK Whole slide imaging equivalency and efficiency study experience at a large academic centerMod Path Feb 2019 Sirintrapun et al
Transition to digital pathology workflowsndash Digital Quality Control is paramountndash Recut and rescan slides immediately before getting to a pathologistndash Cost and efficiency savings
Previously not insurmountablendash Increasingly too time consuming to do manuallyndash Non-reproducible
Unmet Need
We need better quality control of our slides
Slides taken from diagnostic cohort of TCGA-BRCA
HistoQC Properties Fast n=1143 in 466 minutes (24sslide) using 6 cores (11TB) Easy ldquoinstallrdquo (git clone) with minimum dependencies
ndash python-openslide matplotlib numpy scipy skimage sklearn
User interface is a single local html5 + JS file no hosting No specialized hardware No internet connection required Designed to be modular and easily extendible
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
MSK Whole slide imaging equivalency and efficiency study experience at a large academic centerMod Path Feb 2019 Sirintrapun et al
Transition to digital pathology workflowsndash Digital Quality Control is paramountndash Recut and rescan slides immediately before getting to a pathologistndash Cost and efficiency savings
Previously not insurmountablendash Increasingly too time consuming to do manuallyndash Non-reproducible
Unmet Need
We need better quality control of our slides
Slides taken from diagnostic cohort of TCGA-BRCA
HistoQC Properties Fast n=1143 in 466 minutes (24sslide) using 6 cores (11TB) Easy ldquoinstallrdquo (git clone) with minimum dependencies
ndash python-openslide matplotlib numpy scipy skimage sklearn
User interface is a single local html5 + JS file no hosting No specialized hardware No internet connection required Designed to be modular and easily extendible
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Transition to digital pathology workflowsndash Digital Quality Control is paramountndash Recut and rescan slides immediately before getting to a pathologistndash Cost and efficiency savings
Previously not insurmountablendash Increasingly too time consuming to do manuallyndash Non-reproducible
Unmet Need
We need better quality control of our slides
Slides taken from diagnostic cohort of TCGA-BRCA
HistoQC Properties Fast n=1143 in 466 minutes (24sslide) using 6 cores (11TB) Easy ldquoinstallrdquo (git clone) with minimum dependencies
ndash python-openslide matplotlib numpy scipy skimage sklearn
User interface is a single local html5 + JS file no hosting No specialized hardware No internet connection required Designed to be modular and easily extendible
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
HistoQC Properties Fast n=1143 in 466 minutes (24sslide) using 6 cores (11TB) Easy ldquoinstallrdquo (git clone) with minimum dependencies
ndash python-openslide matplotlib numpy scipy skimage sklearn
User interface is a single local html5 + JS file no hosting No specialized hardware No internet connection required Designed to be modular and easily extendible
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
HistoQChellipYour Pixels Matter
HistoQC reproducible slide quality metrics with artifact localization
githubcomchoosehappyHistoQC
andrewjanowczykcaseeduandrewjanowczykcom
HistoQCRepocom
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Gold in the hillshellipRole of Tumor Morphology ER+ Breast Ca Modified Bloom-Richardson (mBR) grading (Elston and Ellis1991)
ndash Tubule formation nuclear pleomorphism mitotic activity
mBR identifies tumors as low intermediate and high grade Correlation between tumor Grade and outcome Visually determined qualitative High inter- and intra-observer variability
Among 7 pathologists k = 050 ndash 059 (Meyer et al 2005)Between pathology departments k = 051 to 054 (Boisen et al 2000)
Suboptimal treatment can result from incorrect grading (Dalton et al 2000)
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
IbRiS Comparing against Oncotype Dx RS
0 01 02 03 04 05 06 07 08 09 1Normalized Distance
θ1 θ2
te ed ate outco e
Poor outcome
Good outcomeIntermediate outcomePoor outcome
Journal Path Informatics 2011 Madabhushi Feldman et al
~450 feature data spaceHand crafted featuresInspired by Pathology expertise
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Ibris and outcomes ECOG 2197
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Beck et al Science Trans Med 3 (108) 2011
Predicting Breast Cancer Survival using Computational Morphology C-Path
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Beck et al Science Trans Med 3 (108) 2011
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Finding Lymph node mets ISBI 2016CAMELYON 201617
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
JAMA December 12 2017 Volume 318 Number 22 and arXiv 170302442v2 March 2017
Machine vs person Performance (ROC)
Pathologist 0966 (34 misses) WOTC
Harvard (Best algorithm) 0925 (75 misses)
Combination Man + Machine 0994 (06 misses)
Dual Neural net 0994
FROC=sensitivity at various FP rates8FP is FN rate at 8FP per slide
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Impact of Deep Learning Assistance on theHistopathologic Review of Lymph Nodes for MetastaticBreast CancerAJSP 42(12) 2018 Stumpe et al
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Terabyte-scale Deep Multiple Instance Learningfor Classication and Localization in Prostate Pathology
12000 WSI
Fuchs et al Nature Med 2019
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Rescreening for prostate cancerIbex inc Maccabi Healthcare Services is a large healthcare provider with a centralized
pathology institute - 120000 histology accessions per year- ~700 prostate core needle biopsies (PCNBs)- Roughly 40 of the PCNBs are diagnosed with cancer
IBEX Medical Analytics whole slide images of PCNBs including cancerous glands (of Gleason patterns 3 4 and 5) high-grade PIN and inflammation The algorithm utilizes state-of-the-art Deep learning CNN trained on many thousands of image samples taken from hundreds of PCNBs from multiple institutes and manually annotated by senior pathologists
Small study shown at ECDP 2018 in Helsinki ndash 100 retrospective cases that had been diagnosed as benign and found two three errors - In two cases the algorithm identified small foci of Gleason 3 Placed into watchful waiting groups Two years later both patients were diagnosed with higher grade cancer and underwent radical prostatectomy - Third case was a larger focus of pseudo-hyperplastic CAP resection showed a CAP(4+3) confined to prostate
System now used to rescreen all negative core prostate biopsies- New workflow AI has 30-40 false positive rate ndash pathologist then reviews specific cores identified by hotspots to decide if any lesion needs further workup or staining
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs How to get big data with outcomes Federation Practice models
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Current Status Diagnostic Silos without clear integration strategy
RadiologicImaging
Lab MolecularDiagnostics
Histology
Other ldquoomicsrdquo
MetabolomicsMicrobi ldquoomerdquoSocial Media ldquoomerdquo
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Path Rads integration
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Enterprise imaging team -shared Enterprise storage ndash shared PACS Software development will thrive in
Radiology $$$ Structured reporting off the Image Integration with EMR to gather clinical data App store API access to pixel pipeline
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Outline
Digital Pathology Fellow Silo or Enterprise view (RadPath)
So why arenrsquot we there yethellip Scanner costs Itrsquos not just the pixels Big data with outcomes Practice and business models
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Industry timeline digital pathology
199860 min scan$$$
2018lt60 s scanZ axis coming$$$$
2017FDA V10
V15ComputationalPhotonics5 minZ axis$
Gen1 Gen4-5 Gen6
V20Slidefree imaging2-3 minexperimental
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
New imaging modalities coming
075NA 12 NA
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Digital Pathology is not just images
Treanor et al 2014 time and motion studies 14-12 of time is looking through medical record for data How do we make data gathering more precise more focused
and faster Center healthcare Innovation EHR extensions Yevgeniy Gitelman Katherine Choi Oncology Pathology RadiologyOncology Surgery
Hum Path Vol45(10) Oct2014 2101-6
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Important disease-specific patient context is highlighted
1
Full reports for drilldownEliminate non-clinical 50
Notes Imaging and Surgical Pathology on one timeline chronologyFocus on the key 30
Smart previewsshow ldquoImpressionrdquo for quick scanningFocus on the key 5
2
3
4
Scope + Data Feasibility
Prototype + Design
Contextual Inquiry
Scope + Data Feasibility
Prototype + Design
Build +IterateExploring EHR Extensions
Custom filters by type or smarter groupers
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Penn DataStore II Foundation
Penn DataStore
Many source databases consolidated into a massive Data Warehouse (over 6 billion rows of data)
bull Standard metricsmeasures (data domain committees)
bull Curated Data Stores
bull Self-Service tools
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Multi-Institutional Collaborations
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Current Paradigm Data-Sharing Pooled Datasets
DOES NOT SCALE DUE TO
LEGAL PRIVACY amp DATA-OWNERSHIP
CONCERNS
The HopeMulti-Institutional Collaborations
Spyridon Bakas UO1 with Intel partnership
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Feasibility study on brain tumor segmentationbull Data from 10 institutions
bull Patients per institution 70 27 17 12 11 9 6 6 4 3
Multi-Institutional Collaboration
Sheller et al MICCAI BrainLes Springer LNCS 2019 (arXiv181004304)
Proposed Paradigm Federated LearningWithout sharing patient data
Current Paradigm Data-SharingLEGAL PRIVACY DATA-OWNERSHIP CONCERNS
Data-sharing performance 862 Federated performance 852 (987 of Data-sharing performance)
Spyridon Bakas UO1 with Intel partnership
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Conclusions FDA clearance is only a beginning Machine learning will accelerate Targeted review Rare event detection Tumor finding Feature classification Grading ScreeningRescreening
Outcome prediction
qIHC qMultiplex
Large well curated and annotated datasets are platinum Federation May help us to get to big data
Data Science is our future ndash we are hiring Business ndash new models of practice How do we position these tools for new opportunities to care
for patients
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
Case Western Lab Director Anant Madabhushi PhDPostdocs James Monaco PhD Gaoyu Xiao PhD Jun Xu PhD Andrew Janowczyk PhDGraduate Students Jonathan Chappelow Scott Doyle Satish Viswanath Pallavi Tiwari George Lee Shannon Agner Ajay Basavanhally Rob Toth Andrew JanowczykUndergraduate Students Jay Naik Hussain Fatakdawala Amod Jog
Penn Clinical Collaborators Spyridon Bakas PhD David Roth MD PhD Mitch Schnall MD PhD David Roth MD PhD John E Tomaszewski MD William Lee Natalie Shih MD
Clinical Collaborators Shridar Ganesan MD PhD
Penn Center Clinical Innovation Roy Rosin Yevgeniy Gitelman MD Katherine Choi
