Leveraging EHR Data to Evaluate Sepsis Guidelines
Bonnie L. Westra, PhD, RN, FAAN, FACMI
Beverly Christie, DNP, RN; Connie W. Delaney, PhD, RN, FAAN, FACMI; Grace Gao, DNP, RN; Steven G. Johnson, MS; Anne LaFlamme, DNP, RN; Jung In Park, PhD-C, RN; Lisiane Pruinelli, PhD-C, RN; Suzan Sherman, PhD, RN;
Piper Svensson-Ranallo PhD; Stuart Speedie, PhD
Acknowledgment
This was supported by Grant Number 1UL1RR033183 from the National Center for Research Resources (NCRR) of the National Institutes of Health (NIH) to the University of Minnesota Clinical and Translational Science Institute (CTSI).
Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the CTSI or the NIH.
The University of Minnesota CTSI is part of a national Clinical and Translational Science Award (CTSA) consortium created to accelerate laboratory discoveries into treatments for patients.“
Purpose
• Demonstrate feasibility of creating a hierarchical flowsheet ontology in i2b2 using data‐derived information models
• Determine the underlying informatics and technical issues
• Demonstrate the applicability of flowsheet data to Surviving Sepsis Campaign (SSC) guidelines on patient outcomes
Requirements for Useful Data
• Common data models• Standardized coding of data• Standardized queries
http://www.pcornet.org/resource-center/pcornet-common-data-model/
6
Vision – Inclusion Nursing and Other Interprofessional Data
Clinical DataNMDS
Management Data
NMMDS
Other Data Sets
Continuum of Care
Example Flowsheet
Flowsheet Data Challenges
• Volume of data• There are multiple measures for the same concepts• Different people building screens• Software upgrades• Discipline/ practice specific needs
• No information models exist• Data driven information modeling required
Information Model Development Process
Identify Clinical Data Model Topic
Identify Concepts
Map Flowsheets
to Concepts
Present Validate
UMN – Academic Health Center CDR
Flowsheets constitute 34% of all data
• 14,564 measure types• 2,972 groups• 562 templates• 1.2 billion observations
• 2,000 measures cover 95% of observations
Sample Data Source ‐ Clinical Data Models
T562
Groups2,696
Flowsheet Measures14,550
Data Points153,049,704
• Flowsheet Data from 10/20/2010 - 12/27/2013
• 66,660 patients• 199,665 encounters
Development Process Details• Identify clinical topic important to researchers/ operations• Develop a list of concepts from research questions, clinical
guidelines and literature• Search for concepts in templates/groups/measures
• Search associated groups for additional concepts• Add matched concepts to running list• Categorize into assessment and interventions• Organize into hierarchy• Combine similar concepts that have similar value sets• Validated by a second researcher
Pain Neuromusculoskeletal System
Falls/ Safety Respiratory system
Peripheral Neurovascular (VTE) Vital Signs, Height & Weight
Genitourinary System/ CAUTI
Aggression and InterpersonalViolence
Pressure Ulcers Psychiatric Mental Status Exam
Cardiovascular System Substance Abuse
Gastrointestinal System Suicide and Self Harm
Flowsheet Information Models
Example Information Model
57 Unique Flowsheet Measures
What is i2b2?• Informatics for Integrating Biology and the Bedside (i2b2)• Framework for research cohort discovery
Example Vital SignsTraditional i2b2 ModelExtended Flowsheet
i2b2 Model
Informatics Issues Encountered• Redundancy – flowsheet and value sets
• 7 blood pressure and 10 heart rate measures• Mapped multiple flowsheet measures to same concept
• Variations in value sets • Created a unique list of all for same concept
• Measures with similar names represented different concept –i.e. search “display name” – Urine Output• R IP URINE FOLEY• URINE OUTPUT• URINE OUTPUT.MODIFIED ALDRETE • R NEPROSTOMY URINE OUTPUT• URINE OUTPUT (ML) 0‐unable to void and uncomfortable
1‐unable to void but comfortable2‐has voided, adequate urine output per device, or not applicable
Technical Issues Encountered• Free text response
• Included name of measure, no data included in i2b2• Multi‐response items
• Created a separate row OBSERVATION_FACT table• Choice list ‐ comment or “other” option
• Created a row for each type of comment• Numeric response measures ‐ units of measure not clearly identifiable• Modified name to include unit of measure
• Mapping issues• Changed names to exclude “* | / \ “ < > ? %”• Constructed synthetic value item id’s
• Names must be unique within first 32 characters • Changed from fully specified names to multiple levels
Discussion/ Conclusion• Flowsheet data represent the largest portion of CDR, rich source of nursing and interprofessional clinical data
• Created 14 information models, 81M observations• Transformed models for flowsheet measures into i2b2• Identified a number of informatics and technical issues and developed processes for managing these issues
• Continue to clean up information models• External validation initiated• Flowsheet data can extend knowledge of interprofessional evidence‐based practice to improve health outcomes
Next Steps• External validation of information models with additional organizations• http://www.fhims.org/press_ulcer.html• Adding conceptual definitions• Mapping to standardized terminology – LOINC/ SNOMED CT
• Demonstrate comparative effectiveness research across organizations
• Collaborate with other common data model efforts to expand CDMs to include assessments and additional interventions in IM’s derived from flowsheet data
A Data Mining Approach to Determine Sepsis Guideline Impact on Inpatient
Mortality and ComplicationsMichael Steinbach, PhD; Bonnie L. Westra, PhD, RN, FAAN, FACMI; György J. Simon, PhD
Lisiane Pruinelli, MSN, RN, PhD‐C; Pranjul Yadav, PhD‐C; Andrew Hangsleben; Jakob Johnson; Sanjoy Dey, PhD;
Maribet McCarty, PhD, RN; Vipin Kumar, PhD; Connie W. Delaney, PhD, RN, FAAN, FACMI
Acknowledgments• Support for this study is provided by
• NSF grant IIS‐1344135 • National Center for Research Resources of the NIH 1UL1RR033183.
• Contents of this document are the sole responsibility of the authors and do not necessarily represent official views of the NSF, CTSI, or NIH
Introduction• Sepsis or septicemia has doubled from 2000 to 2008• Hospitalizations increased 70%• Severe sepsis and septic shock have higher mortality – 18%‐40%
• Patients are sicker, have longer length of stay, more expensive• EBP guidelines (SSC) could lead to earlier diagnosis and treatment
• Guidelines are not fully implemented in clinical practice• The effectiveness of these guidelines are unclear
AimThe overall aim is to evaluate and extend evidence‐based guidelines for patients with health disparities for the prevention and management of sepsis complications1. Map EHRs data to SSC guideline recommendations2. Estimate the compliance with the SSC guideline
recommendations; and3. Estimate the effect of the SSC individual
recommendations on the prevention of in‐hospital mortality and sepsis‐related complications
Data Source• De‐identified EHR data obtained, after Institutional Review Board approval • Data obtained from a Midwest hospital• All data from patients hospitalized between 1/1/09 ‐ 12/31/11 (including all encounters through 12/31/13)
• Billing diagnosis code of sepsis (ICD‐9: 785.5*, 038.*, 998.*, 599.*, 995.9*)
• 1,993 patients (1,270 with little missing data)• 189 (177) Severe sepsis/ septic shock (995.92 and 785.5*)• 1,804 (1,093) Other sepsis diagnoses
• Exclusion criteria:• Patients with cardiogenic shock• Patients with no antibiotic therapy
Study Sequence
Baseline and Comorbidities
Propensity Score
Matching
“TimeZero”Onset of Sepsis
Start SSC Recommenda‐
tions
Mortality and complications
Baseline• Sociodemographics
• Age• Gender• Race/ ethnicity• Payer (Medicaid for low income)
• Vital signs • Heart rate (HR)• Respiratory Rate (RR)• Temperature (Temp)• Mean arterial pressure (MAP)
• Laboratory results• Lactate• White Blood Cell Count (WBC)
Sepsis Time Zero• At least 2 of the following criteria:
• MAP < 65• HR > 100• RR > 20• Temp < 95 or > 100.94 F• WBC < 4 or > 12• Lactate > 2.0
Baseline/ Outcomes• 5 outcome variables• In‐hospital mortality• New complication (in hospital and up to 30 days after discharge)
• Cerebrovascular• Respiratory • Cardiovascular• Kidney
SSC guideline ‐ Interventions
Data Preparation• Matching SSC guidelines to data elements• Data quality assessment based on literature and domain knowledge• Missing values (lactate – 7.7%, temp – 3%, WBC – 3%)• Out of range values (CVP, > 50 for 133 patients, some negative values
• Excluded negative values and those > 30
• For each data element, we evaluated range and created rules for suitable range
• Compared with other values i.e. MAP and SBP/ DBP
• Determine use of one or more flowsheet measures for vital signs
Flowsheet Data NeededConcept (number uniquemeasures)
Concept (number uniquemeasures)
Heart rate (8) IV (11)Respirations (3) Weight (5)Temperature (1) Urine output (15)CVP (3) Dialysis (3)MAP/ BP (15) Ventilator (3)
Baseline CharacteristicsCharacteristics Patient Count
n=177
Characteristics Patient Count
n=177
Mean (IQR) Mean (IQR)
Age (years) 61 (51‐71) Temperature 98.4 (97.3‐99.5)
Gender (Male) 102 Heart rate 101.3 (87.4‐200.4)
Race (Caucasian) 97 Respiratory rate 20.6 (17.1‐22.8)
Ethnicity (Latino) 11 Cardiovascular 100
Payer (Medicaid) 102 Cerebrovascular 66
White blood cell 15.8 (9.1‐18.6)) Respiratory 69
Lactate 2.8 (1.6‐2.8) Kidney 62Mean blood pressure 73.9 (40.7)
Compliance with SSC GuidelinesRules Description Patient Count / %
Y N % Compl N/A
1. Was Blood Culture done? (BCulture) 126 51 71 0
2. Was Antibiotic given after Blood Culture? (Antibiotic) 99 27 79 513. Was Lactate checked? (Lactate) 127 50 72 0
4. Was Fluid Resuscitation done if Lactate > 4? (LactateFluid) 36 0 100 1415. Was Blood Glucose checked? (BGlucose) 132 45 75 0
6. Was Insulin given if two Blood Glucose measures were > 180? (GlucoseInsulin)
38 8 83 131
7. Was MAP checked? (MAP) 177 0 100 08. Was Fluid Resuscitation give if MAP < 65? (MAPFluids) 160 6 96 11
9. Was Vasopressor given if MAP < 65 after Fluid Resuscitation? (Vasopressor)
26 140 16 11
10. Was CVP checked? (CVP) 121 56 68 011. Was Fluid Resuscitation done if CVP < 2? (CVPFluids) 15 162 9 0
12. Was Albumin given if CVP < 2 after Fluid Resuscitation? (Albumin)
4 11 27 162
13. Was a Diuretic given if CVP above 12? (Diuretic) 10 71 12 9614. Was there Respiratory Distress*? (RespDistress) 167 10 94 0
15. Was a ventilator given if there was Respiratory Distress? (Ventilator)
92 75 55 10
Results ‐MortalityCI= (0.03, 0.20)
Results ‐ Complications
Cardiovascular Respiratory Kidney Cerebrovascular DeathBCulture (‐0.11, 0.15) (‐0.16, 0.12) (‐0.15, 0.11) (‐0.09, 0.20) (‐0.14, 0.09)Antibiotic (‐0.16, 0.10) (‐0.23, 0.13) (‐0.08, 0.26) (‐0.09, 0.28) (‐0.21, 0.10)Lactose (‐0.05, 0.19) (‐0.20, 0.07) (‐0.08, 0.18) (‐0.04, 0.21) (‐0.12, 0.10)BGlucose (‐0.02, 0.25) (‐0.02, 0.28) (‐0.16, 0.14) (‐0.06, 0.18) (‐0.19, 0.09)Vasopressor (‐0.11, 0.27) (0.04, 0.35) (‐0.20, 0.17) (‐0.32, ‐0.07) (‐0.10, 0.21)CVP (‐0.03, 0.16) (‐0.06, 0.17) (‐0.10, 0.14) (‐0.08, 0.16) (‐0.08, 0.13)RespDistress (‐0.25, 0.36) (‐0.36, 0.37) (‐0.14, 0.40) (‐0.30, 0.37) (‐0.25, 0.14)Ventilator (0.04, 0.19) (0.08, 0.32) (‐0.11, 0.09) (‐0.08, 0.11) (0.03, 0.20)
CI (0.04, 0.35)
CI (0.04, 0.19)
CI (0.08, 0.32)
CI (‐0.32, ‐0.07)
Limitations• Small sample size• No attempt to add guideline timing of 3 or 6 hours• Guidelines as a whole may affect outcome vs single recommendations within guideline – no comparison group
• Timing of data used (2009 – 2013) – may not reflect current practice• SSC Guidelines may not have been thoroughly implemented at health organization
Conclusions• Flowsheet data are useful for research• EHR data can be used to estimate compliance with individual guideline recommendations
• EHR can be used to estimate the effect of the guideline adherence on sepsis‐related complications
• Some guideline recommendations are protective for patients for certain outcomes
• Other variables may be needed to control for variation in severity of illness or variation in practice