Explainable Prediction of Medical Codes from Clinical Text
James Mullenbach
Sarah Wiegreffe
Jon Duke
Jimeng Sun
Jacob Eisenstein
The Clinical Coding Problem● Electronic Health Records have seen increasing
adoption in the last 5 years
2
The Clinical Coding Problem● Electronic Health Records have seen increasing
adoption in the last 5 years● They abound with written physician notes
3
The Clinical Coding Problem● Electronic Health Records have seen increasing
adoption in the last 5 years● They abound with written physician notes● ICD: taxonomy of diagnoses and procedures
4
The Clinical Coding Problem● Electronic Health Records have seen increasing
adoption in the last 5 years● They abound with written physician notes● ICD: taxonomy of diagnoses and procedures● Human coding laborious, error-prone [Birman-Deych et
al., 2005]
5
The Clinical Coding Problem, as an NLPer● Highly multi-label classification
○ Of nearly 9,000 labels, predict a subset
6
The Clinical Coding Problem, as an NLPer● Highly multi-label classification
○ Of nearly 9,000 labels, predict a subset● Testbed for document representations
7
The Clinical Coding Problem, as an NLPer● Highly multi-label classification
○ Of nearly 9,000 labels, predict a subset● Testbed for document representations● Documents are long and loosely structured
8
The dataset
● Open-access, de-identified
9
The dataset
● Open-access, de-identified● 47k admissions -> 47k documents for training
10
The dataset
● Open-access, de-identified● 47k admissions -> 47k documents for training● Loosely structured:
11
Admission Date: [**2118-6-2**] Discharge Date: [**2118-6-14**]
Date of Birth: Sex: F
Service: MICU and then to [**Doctor Last Name **] Medicine
HISTORY OF PRESENT ILLNESS: This is an 81-year-old femalewith a history of emphysema (not on home O2), who presents...
The dataset
● Open-access, de-identified● 47k admissions -> 47k documents for training● Loosely structured:
12
Admission Date: [**2118-6-2**] Discharge Date: [**2118-6-14**]
Date of Birth: Sex: F
Service: MICU and then to [**Doctor Last Name **] Medicine
HISTORY OF PRESENT ILLNESS: This is an 81-year-old femalewith a history of emphysema (not on home O2), who presents...
Long: Median post-processed document length: 1,341
The dataset
● Open-access, de-identified● 47k admissions -> 47k documents for training● Loosely structured:
13
Admission Date: [**2118-6-2**] Discharge Date: [**2118-6-14**]
Date of Birth: Sex: F
Service: MICU and then to [**Doctor Last Name **] Medicine
HISTORY OF PRESENT ILLNESS: This is an 81-year-old femalewith a history of emphysema (not on home O2), who presents...
Long: Median post-processed document length: 1,341
Many labels: 519.1: ‘Other disease…’491.21: ‘Obstructive …’518.81: ‘Acute respir…’486: ‘Pneumonia, orga…’276.1: ‘Hyposmolality…’244.9: ‘Unspecified h…’31.99: ‘Other operati…’...
Median # labels: 14
Prior approaches● Predict from a subset of labels
14
This work● Predict from all labels
Prior approaches● Predict from a subset of labels● Sub-domain focus, e.g.
radiology
15
This work● Predict from all labels● General ICU setting
Prior approaches● Predict from a subset of labels● Sub-domain focus, e.g.
radiology● Private dataset
16
This work● Predict from all labels● General ICU setting● Open-access data
Modeling desiderata
...who sustained a fall at home she was found to have a large acute on chronic subdural hematoma with extensive midline shift...
17
● Focus on the parts that matter
Modeling desiderata
...who sustained a fall at home she was found to have a large acute on chronic subdural hematoma with extensive midline shift...
18
E849.0: Home accidents
801.26: ...subdural, and extradural hemorrhage...
● Focus on the parts that matter● Treat labels individually
Convolution
19
h1
h2
h3
patient
was
initially
placed
on
embed19
convolve
Convolution + Attention
20
U
α11
α21
α31
vh1
h2
h3
patient
was
initially
placed
on
parameter
variable
embed convolve attend (separately for each label)20
Convolution + Attention = the CAML model
21
U
α11
α21
α31
v
lossh1
h2
h3
patient
was
initially
placed
on
parameter
variable
classifyembed convolve attend21
β
Dealing with the long tail● Huge label space (nearly 9,000 total)
2222
Dealing with the long tail● Huge label space (nearly 9,000 total)● Many labels are similar250.00: “Diabetes mellitus without mention of complication, type II or unspecified type, not stated as uncontrolled”
250.02: “Diabetes mellitus without mention of complication, type II or unspecified type, uncontrolled”
2323
Dealing with the long tail: DR-CAML
24
U
α11
α21
α31
lossh1
h2
h3
patient
was
initially
placed
on
parameter
variable
“Atrial fibrillation” Max-pool CNN Label embedding
L2 similarity
24
v
β
MIMIC results● Few works use the whole
label space
2525
*[Scheurwegs et al., 2017]
MIMIC results● Few works use the whole
label space● 20%, 8% relative
improvement over prior SotA
26
*[Scheurwegs et al., 2017]
26
More MIMIC metrics
27
● Enable future comparison● Precision @ k: decision support use-case
27
But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain
2828
But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain● Physician evaluation: select all text snippet(s) that explain the code
2929
But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain● Physician evaluation: select all text snippet(s) that explain the code● Baselines: LogReg, CNN, cosine similarity
3030
But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain● Physician evaluation: select all text snippet(s) that explain the code● Baselines: LogReg, CNN, cosine similarity● 100 random label-document samples
3131
But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain● Physician evaluation: select all text snippet(s) that explain the code● Baselines: LogReg, CNN, cosine similarity● 100 random label-document samples● “Informative” and “Highly Informative”
3232
Physician evaluation example*Code: 575.4
Full descriptions: Perforation of gallbladder
“. . in the setting of gallbladder perforation secondary to acute acalculous cholecystitis after . . . . . . . . inhalation hospital1 times a day metronidazole mg tablet sig one tablet po tid times . . . . . . . to have an infection in your gallbladder requiring iv antibiotics and tube placement for . . .”
33
*not exact format used33
Physician evaluation example*Code: 575.4
Full descriptions: Perforation of gallbladder
“. . in the setting of gallbladder perforation secondary to acute acalculous cholecystitis after . . . . . . . . inhalation hospital1 times a day metronidazole mg tablet sig one tablet po tid times . . . . . . . to have an infection in your gallbladder requiring iv antibiotics and tube placement for . . .”
34
*not exact format used
attention
34
Physician evaluation example*Code: 575.4
Full descriptions: Perforation of gallbladder
“. . in the setting of gallbladder perforation secondary to acute acalculous cholecystitis after . . . . . . . . inhalation hospital1 times a day metronidazole mg tablet sig one tablet po tid times . . . . . . . to have an infection in your gallbladder requiring iv antibiotics and tube placement for . . .”
35
*not exact format used
attentionCNN
Physician evaluation example*Code: 575.4
Full descriptions: Perforation of gallbladder
“. . in the setting of gallbladder perforation secondary to acute acalculous cholecystitis after . . . . . . . . inhalation hospital1 times a day metronidazole mg tablet sig one tablet po tid times . . . . . . . to have an infection in your gallbladder requiring iv antibiotics and tube placement for . . .”
36
*not exact format used
attention cosine sim.CNN
Physician evaluation example*Code: 575.4
Full descriptions: Perforation of gallbladder
“. . in the setting of gallbladder perforation secondary to acute acalculous cholecystitis after . . . . . . . . inhalation hospital1 times a day metronidazole mg tablet sig one tablet po tid times . . . . . . . to have an infection in your gallbladder requiring iv antibiotics and tube placement for . . .”
37
*not exact format used
attention cosine sim.CNN
LogReg
Physician evaluation results● Improves upon CNN,
LogReg
3838
Physician evaluation results● Improves upon CNN,
LogReg● More experts needed!
3939
Future work● Exploit the loose structure of discharge summaries
○ Some have already done this [Shi et al., 2017]
4040
Future work● Exploit the loose structure of discharge summaries
○ Some have already done this [Shi et al., 2017]● Sparsify or structure attention to improve interpretability, accuracy
○ “fusedmax” [Niculae & Blondel, 2017]
4141
Future work● Exploit the loose structure of discharge summaries
○ Some have already done this [Shi et al., 2017]● Sparsify or structure attention to improve interpretability, accuracy
○ “fusedmax” [Niculae & Blondel, 2017]● Develop methods for few-shot or zero-shot classification
4242
Future work● Exploit the loose structure of discharge summaries
○ Some have already done this [Shi et al., 2017]● Sparsify or structure attention to improve interpretability, accuracy
○ “fusedmax” [Niculae & Blondel, 2017]● Develop methods for few-shot or zero-shot classification● Exploit hierarchy of codes
○ GRAM [Choi et al., 2017]
43
250.0 / \250.00 250.01
DM type 2 DM type 1
43
Summary● ICD coding is valuable and challenging
4444
Summary● ICD coding is valuable and challenging● Convolution + attention works well
4545
Summary● ICD coding is valuable and challenging● Convolution + attention works well● Attention can explain the predictions
4646
Summary● ICD coding is valuable and challenging● Convolution + attention works well● Attention can explain the predictions
4747
Thank you!Questions?
Code, data splits, pretrained models: github.com/jamesmullenbach/caml-mimic