Date post: | 11-Nov-2014 |
Category: |
Health & Medicine |
Upload: | brad-doebbeling |
View: | 1,168 times |
Download: | 4 times |
VA HSR&D Salt LakeConsortium forHealthcareInformaticsResearch
CHIR MRSA
Brad Doebbeling, MD, MSc; Jennifer Garvin, PhD; Mahesh Merchant, PhD; Mike Rubin, MD; Rick Martinello, MD; Mary Goldstein, MD; Phil Foulis, MD; Steve Luther, PhD; Pradeep Mutalik, PhD; Matt Scotch, PhD, Samson Tu, MS; Susana Martins, MD; Jeff Friedlin, DO; Katalina Gullans, MA
Indianapolis, IN; Salt Lake City, UT; Palo Alto, CA; West Haven, CT; Tampa, FL.
Project Origin
Healthcare-associated infections (HAIs) leading cause of preventable death.
VA Pittsburgh Reduced MRSA Significantly!
VHA National MRSA Reduction Initiative
HA Surveillance Time Intensive
Staff need to Focus on Problems, Intervene Early
Need for Automated Classification of MRSA Status
Interdisciplinary, multisite collaborative extracting structured and unstructured data from electronic health records (EHR)
(VHA, METHICILLIN-RESISTANT STAPHYLOCOCCUS AUREUS (MRSA) INITIATIVE, http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=1525. 2007.)
CHIR MRSA Project Specific Aims
Aim 1: Develop, review and refine an ontology for clinically- and epidemiologically-relevant concepts to enable detection of MRSA for reporting purposes and gather requirements for developing a MRSA surveillance and reporting tool
Aim 2: Index MRSA-related Concepts in Clinical Narrative
Aim 3: Clinical Inference and Analysis of MRSA-Related Information Contained in the Medical Record
Aim 4: Develop and evaluate a prototype surveillance application that uses automatically processed VA electronic health record data
Translational Aim: Evaluate algorithms for making multi-type predictions based on heterogeneous data, using MRSA as a clinical domain
Research Questions/Hypotheses
Developing an ontology for clinically- & epidemiologically-relevant concepts enables detection of MRSA for reporting purposes
Concepts from practice guidelines, interviews, clinical narrative, and machine learning help operationalize use cases
Developing an automated MRSA surveillance system is as reliable and valid as human reviewers
Methods
1. Identify relevant documents; 2. De-identify documents; 3. Determine document quality; 4. Build a clinical and population-based ontology;5. Annotate documents; 6. Extract and define MRSA relevant concepts; and 7. Determine clinical relevance and perform inference
modeling to identify linguistic elements assoc. with MRSA.
Microbiology completes a microbiology test for all patients that tested NEGATIVE
at admit.
Microbiology completes a microbiology test for all patients that tested NEGATIVE
at admit.
ALL positive MRSA are reported to Diana and sent to BUGS (printer)
ALL positive MRSA are reported to Diana and sent to BUGS (printer)
ADMITS ADMITS
DISCHARGEDISCHARGE
Ordered for ALL patients admitted to the hospital.
Ordered for ALL patients admitted to the hospital.
Nasal Nares DNA Swab (2 swabs taken for each patient)
Nasal Nares DNA Swab (2 swabs taken for each patient)
Swabs are run by molecular tech once daily. Swabs are processed at 6am – and results are available to unit by 2:30pm.
Swabs are run by molecular tech once daily. Swabs are processed at 6am – and results are available to unit by 2:30pm.
Patients that are swabbed after 8am on a given-day, their tests are not run until the following
day.
Patients that are swabbed after 8am on a given-day, their tests are not run until the following
day.
Capacity to run 96 swabs.Indy averages 40-50 day
Capacity to run 96 swabs.Indy averages 40-50 day
Positive MRSA phone call to MRSA coordinator and to unit. Results inputted in to DHCP. (if positive, 2nd swab is placed in salt broth and saved for growth. Saved
for files)
Positive MRSA phone call to MRSA coordinator and to unit. Results inputted in to DHCP. (if positive, 2nd swab is placed in salt broth and saved for growth. Saved
for files)
Tests only detect “colonization.” There are no tests run for “infection” or
determinations made regarding colonization v infection in molecular
pathology.
Tests only detect “colonization.” There are no tests run for “infection” or
determinations made regarding colonization v infection in molecular
pathology.
MRSA swabs at discharge are to capture “conversions” only (what patients
contracted MRSA during the hospital stay).
MRSA swabs at discharge are to capture “conversions” only (what patients
contracted MRSA during the hospital stay).
Discharge swab is tied to physician
orders. Physicians could use a tool that is “physician
timed discharge” for
ordering swab – this
not used frequently.
Discharge swab is tied to physician
orders. Physicians could use a tool that is “physician
timed discharge” for
ordering swab – this
not used frequently.
TRANSFERSTRANSFERS
Only patients that tested NEGATIVE at admit are swabbed at transfer
Only patients that tested NEGATIVE at admit are swabbed at transfer
Swabs are ordered by the physician in the transferring unit. Receiving unit ‘should’
also order a swab. Indy working to automate this process.
Swabs are ordered by the physician in the transferring unit. Receiving unit ‘should’
also order a swab. Indy working to automate this process.
Molecular swab testing process completed in same fashion as admit process.
Molecular swab testing process completed in same fashion as admit process.
If a swab is ordered at transfer for a patient that tested
POSITIVE at admit, the molecular tech will cancel the order.
If a swab is ordered at transfer for a patient that tested
POSITIVE at admit, the molecular tech will cancel the order.
“swabbing technique” molecular tech monitors the occurrence of PCR inconclusive
results. Inconclusive results are run again the next day. If still inconclusive, a resubmit order
is issued.
“swabbing technique” molecular tech monitors the occurrence of PCR inconclusive
results. Inconclusive results are run again the next day. If still inconclusive, a resubmit order
is issued.
Units that have a high rate of PCR inconclusive results are given training on swabbing
techniques.
Units that have a high rate of PCR inconclusive results are given training on swabbing
techniques.
ER and Psych do NOT do nasal swabs.
ER and Psych do NOT do nasal swabs.
Patients go to their unit/ward while awaiting swab results.
Patients go to their unit/ward while awaiting swab results.
Patients with a MRSA warning/flag within the last YEAR are placed in isolation pending swab
results.
Patients with a MRSA warning/flag within the last YEAR are placed in isolation pending swab
results.
Positive MRSA patient kept in isolation until cleared (negative swab).
Positive MRSA patient kept in isolation until cleared (negative swab).
Molecular biology might
have two swabs for
same patient at transfer.
Molecular biology might
have two swabs for
same patient at transfer.
Conversion count is
attributed to the last unit
that swabbed.
Conversion count is
attributed to the last unit
that swabbed.
Determination of
Colonization v.
Infection??
Machine Learning
MRSA Pipeline Diagram
Use Cases
Clinical Relevance &
Inference Modeling
Information Extraction
Quality of Documents
De-Identification
Building Ontology
Identify Relevant
DocumentsData
Extraction
*Workflow Data
Collected(Observation
&Interview)
Annotation of Documents
Relevant Publications
Ontology
Natural Language Processing
Feedback provided at each step to previous and all other steps affectedby changes
Documents Ready for NLP
*Selected Workflow Data Used
MRSA Ontology Annotation-inform schema and link free text
to ontology concepts A source of terms (including synonyms) for
natural language processing (NLP)
…Patient 123456789… MRSR…
mouth..
MRSR ontologyin Protégé
Protégé API
MRSA terms &
relationships for NLP
Patient 123456789MRSA has anatomical locationOral cavity structure
NLP engine
Goldstein, Tu, Martin @ Palo Alto
Uses of MRSA Ontology
A vocabulary for queries and inference
…Patient 123456789… MRSR…
mouth..
NLP engine
MRSA-related
dataVISTA data
Inference engines
Rules/Influence
Diagrams…
MRSA surveillance application
Queries
Ontology
Results MRSA Pipeline Model Ontology Development Sampling Plan & Approvals Use Cases Draft Annotation Schema Development of system-Name: Automatic Classification
of MRSA (ACOM): This NLP system will classify patients who have a positive
MRSA culture into those who have an infection and those who are colonized using data from the VHA EMR system.
This will provide assistance to the Infection Preventionist (actor).
The trigger for this system will be the positive culture or PCR for MRSA (note that this will trigger isolation procedures).
Interim Conclusions Communication & FTF meetings key to effective
progress & collaboration. First MRSA ontology developed with a clinical,
healthcare or population emphasis. Ontology will be used for clinical decision support
combining concepts identified in NLP process with other structured data to enhance MRSA surveillance.
Will help differentiate between patients colonized from those infected and provide timely information for multiple clinical and decision support purposes.
Plan create a state-of-the-art surveillance tool that will help reduce MRSA infections in VHA.
Leadership Plans for FY2011 Foster stronger communication between CHIR MRSA
subgroups and CHIR projects VA CHIR-MRSA Use Case Interview conducted at six VA
facilities Development of annotation timeline Publication and manuscript development Work with document quality on CHIR MRSA research aims Connecting the CHIR MRSA pipeline – how the subgroups
inter-relate Provide leadership and direction for subgroups to attain
‘next steps’ and receive deliverables from other teams to accomplish goals and research aims
Test annotation schema with data uploaded to VINCI Analysis of use case interview data – that will inform the
use case
Goals/Expectations for Impact
Milestones Expected Date
Ontology of clinically, epidemiologically relevant concepts that enhance MRSA reporting and surveillance
9/30/10
Development of use cases, incorporation of machine learning
12/30/10
MRSA documents for annotation, sharing, modeling on VINCI
6/30/11
NLP System to extract MRSA related concepts from VA clinical documents
6/30/12
Prototype MRSA surveillance application that uses structured and unstructured data sources
9/30/2013
Backup Slides
Key Workflows Details of Accomplishments Detailed Plans for Subgroups for
FY2011
Ontology Plans for FY2011
Work with annotation team to apply ontology to annotation schema (iterative process).
Incorporation of Drug Terms into the ontology. Mapping of terms to NDF-RT.
Ontology Team works with NLP group to facilitate integration. (iterative process).
Through domain expert meetings, explore possibilities of collaboration with the Clinical Inference and Modeling project.
Preparation of manuscript describing the MRSA ontology for a clinical journal.
Machine Learning Plans for FY2011
Create shared work space on VINCI virtual environment to interact with Tampa team
Enhance versions of GATE, cTakes, and YTEX pipelines to support MRSA use cases
Explore methods to identify relevant documents in positive cases and develop methods to use them in identifying specific infections
Extract a set of age-matched control documents and use them to hone the NLP rules for classification
NLP Plans for FY2011
Create shared work space on VINCI virtual environment to interact with Tampa team
Enhance versions of GATE, cTakes, and YTEX pipelines to support MRSA use cases
Explore methods to identify relevant documents in positive cases and develop methods to use them in identifying specific infections
Extract a set of age-matched control documents and use them to hone the NLP rules for classification
Project Management Received approval for NDS data access for 51 CHIR MRSA team members Received Indianapolis IRB approval for data extraction of local patient data that
will be uploaded to VINCI. Additionally, received local IRB to cover all CHIR MRSA members to access and analyze the Indianapolis patient data that will be stored on VINCI.
Led development and refinement of the Use Case Questionnaire. Received Indianapolis IRB approval to conduct all interviews from Indianapolis. Conducted initial Use Case Questionnaire interviews with 4 key informants.
Scheduled interviews with MICU physician and SICU nurse. Planned Face-to-Face Meeting held in Salt Lake City, April 18 – 20th, 2010 Participated in the development of the MRSA Working Use Case and Workflow
Analysis AMIA Abstracts; review and edits of all abstracts submitted from CHIR MRSA
subgroups Lead weekly CHIR MRSA team calls; responsible for agenda development,
minutes, and follow-up Interface with CHIR MRSA Annotation subgroup, Ontology subgroup, Machine
Learning subgroup, and NLP subgroup Development of patient use cases utilizing de-identified Indianapolis data for
annotation schema testing Developed CHIR MRSA pipeline Presenter in the CHIR MRSA WIP
Ontology Accomplishments Scientific Literature and MRSA Surveillance Guidelines Identified. Literature and
Guidelines discussed with MRSA team and domain experts. Selected Literature posted to Sharepoint.
Identification of Formal MRSA Terms from Literature and Guidelines. Terms added to the ontology in Protégé Environment.
Ontology team identified standards to use for concepts in ontology in collaboration with standards team.
Addition of SNOMED code to ontology concepts. Terms in Sample MRSA reports from Indianapolis integrated into MRSA ontology. Synonyms added to concepts in ontology using ULMS. Meetings with MRSA clinical domain experts to review and refine MRSA Ontology
Terms. Experts are provided Protégé Access to MRSA ontology. MRSA Ontology Version 1* handed off to MRSA NLP team. Ontology team provides
guidance to NLP team to integrate Protégé output with GATE. Ontology refined to reflect the latest concepts/terms/synonyms as ontology team is
informed. Version 2* to Annotation Team for examination and annotation strategy
development. Ontology sent to domain experts, refined to include only MRSA relevant concepts
(current version has terms related to other health associated infections). Potential MRSA relevant drug list (antibiotics, etc) sent to domain experts for review.
Explore possibilities of integration with Clinical Inference and Modeling project.
Machine Learning Accomplishments
Domain Expertise: Pathology and Laboratory Scientific literature and MRSA surveillance guidelines identified for MRSA Ontology sub-group.
Discussed literature and guidelines with other domain experts and MRSA ontology subgroup team members. Emailed literature to group. Sent Tampa VA MRSA surveillance documents to both Ontology sub-group and larger MRSA project group. Documents posted to SharePoint.
Identification of formal MRSA terms from literature and guidelines. Meetings held with MRSA Ontology Subgroup to review and refine MRSA Ontology Terms. Reviewed ontology from MRSA Ontology Subgroup to refine scope of ontology. Provided ongoing consultation to MRSA Ontology sub-group on Pathology and Laboratory testing and
reporting related to MRSA. MRSA Staff Interviews Assisted with development of interview questionnaire for infection control staff. Piloted MRSA interview questionnaire. Consulted with project staff regarding feasibility and logistics of conducting face-to-face or telephone
interviews with infection control staff. Machine Learning Defined use cases for machine learning of Tampa VA MRSA surveillance data. Presented to MRSA Project
Team. Identified and selected 1,200 cases from Tampa VA MRSA surveillance database for data analysis. Identified and extracted 855,244 progress notes from selected cases for use in machine learning
analysis. Used lab test dates and administrative data to reduce number of progress notes for analysis to 98,889. Identified progress note types likely to contain MRSA-related information; Presented note types to MRSA
Project Team. Conducted post-processing of machine learning results to facilitate interpretability. Provided a preliminary report with results of machine learning analyses to MRSA Project Team and MRSA
Ontology Sub-group. Shared case selection and data extraction methodologies to assist with local data extraction efforts at
other project sites. Emailed to Indianapolis project management staff. Presented to MRSA project team.
NLP Accomplishments Successfully imported the MRSA Ontology into GATE NLP. Created a preliminary NLP pipeline for information extraction including the use of
processing resources such as logic rules and gazetteers. Published preliminary pipeline as an executable on MRSA Sharepoint Site. Developed UIMA/cTakes pipeline for NLP of MRSA documents. Created YTEX, a version of cTakes, for machine learning classification of free text
documents. Identified and selected 200 cases from West Haven VA MRSA surveillance
database for data analysis. Identified and extracted 17,000 progress notes from selected cases for use in
machine learning analysis. These notes are within a specified time window of a positive culture.
Domain Expertise: Infection Prevention Scientific literature and MRSA surveillance guidelines identified for MRSA
Ontology sub-group. Discussed literature and guidelines with other domain experts and MRSA ontology subgroup team members.
Identification of formal MRSA terms from literature and guidelines. Meetings held with MRSA Ontology Subgroup to review and refine MRSA Ontology
Terms. Reviewed ontology from MRSA Ontology Subgroup to refine scope of ontology. Provided ongoing consultation to MRSA Ontology sub-group on Infection
Prevention