Reality Check: Challenges with Reporting Data from Various Sources

Dr. Nisha Thampi, MD, MSc

Medical Director, Infection Prevention and Control Program

Children’s Hospital of Eastern Ontario, Ottawa

May 26, 2019

Disclosure Statement

• I have no affiliation (financial or otherwise) with a pharmaceutical, medical device or communications organization.

Measurement is the first step that leads to control and eventually to improvement. - H. James Harrington

Measure understand control improve

But…

Pressure on a program to measure inaccurately so the problem remains poorly understood

Individuals may perceive and therefore measure healthcare-associated infections differently

And…

Critical to follow standard definitions and ensure consistency

• Ascertainment of infection

• Attribution to healthcare setting

• Reporting internally and externally

HAI data under scrutiny by public payer as quality metrics(Reimbursement in the US)

Objectives

• Review process and outcome measures in surveillance of healthcare-associated infections

• Review impact of reportable HAI surveillance indicators

• Highlight pitfalls in accurately reporting data from various sources

Objectives

• Review process and outcome measures in surveillance of healthcare-associated infections

• Review impact of reportable HAI surveillance indicators

• Highlight pitfalls in accurately reporting data from various sources

Value of surveillance

• Systematic, ongoing collection and analysis of data during a defined period of time using standardized definitions• May also involve laboratory confirmation and chart review

• Timely dissemination of information to those who require it in order to take action = improve patient safety

• Why?• Provides measure of burden of illness• Establishes benchmark rates for internal and external comparison

• May motivate hospitals to adhere more closely to best practices in infection prevention

• Identifies potential risk factors• Allows for assessment of specific interventions

PIDAC-IPC 2014: Best Practices for Surveillance of Health Care-associated Infections in Patient and Resident Populations

HAI surveillance indicators: outcomes

• Communicable diseases and events

• Body site-specific• Central line-associated bloodstream infections• Surgical site infections• Infection-related ventilator-associated events• Catheter-associated urinary tract infections

• Organism-specific healthcare-associated infections• Methicillin-resistant Staphylococcus aureus• Vancomycin-resistant Enterococci• Carbapenemase-producing organisms• C.difficile

HAI surveillance indicators: processes

• Central line-associated bloodstream infections• Hand hygiene prior to accessing central line• Bundle adherence (5-7 elements)

• Surgical site infections• Adherence to pre-operative MRSA screening protocol• Appropriateness of surgical antimicrobial prophylaxis

• Infection-related ventilator-associated events• Adherence to practices that reduce infection risk associated with ventilator

use

• Catheter-associated urinary tract infections• Adherence to practices to limit urinary catheter use

Hand hygiene!

Objectives

• Review process and outcome measures in infection prevention and control

• Review impact of reportable HAI surveillance indicators

• Highlight pitfalls in accurately reporting data from various sources

“hospital-wide surveillance of hospital-acquired infections provides appropriate targets for interventions tailored to the specific needs of the hospital”

Surveillance associated with 30% decrease in rate of pneumonias and SSI, 20% decrease in BSI

Journal of the Pediatric Infectious Diseases Society 2018;7(1):18–24

Prospective surveillance for HA-VRI found higher incidence rates compared to hospital-associated bloodstream infections

Organized, intensive surveillance and control activities associated with 32% decrease in HAI

Following introduction of public reporting, C. difficile infections declined by 26% across Ontario = >1,900 cases averted per year

Collaboration in surveillance + bundle implementation= 21% reduction in SSI rate of pediatric cardiothoracic, neurosurgical ventricular shunt, and spinal fusion surgeries

Canadian Nosocomial Infection Surveillance Program

• Established in 1994

• Epidemiology and molecular characteristics of healthcare-associated infections in Canadian hospitals

• Partnerships: PHAC, NML, AMMI

• 73 sentinel hospitals in 10 provinces• 9 freestanding pediatric hospitals

• ~ 78% of Canadian population lives within 100km of CNISP site

15

CNISP

• Provides Canadian hospitals with “benchmark” data • Standardized HAI surveillance case definitions - reviewed annually

• National, regional and site-specific HAI rates, strain types and antimicrobial resistance and utilization data

• Provides evidence-based data:• Antimicrobial surveillance program (CARSS)

• Canadian infection prevention and control guideline preparation (National Advisory Committee for Infection Prevention and Control)

• Raises public awareness of important infection control issues relating to AROs and HAIs

https://www.canada.ca/en/public-health/services/infectious-diseases/nosocomial-occupational-infections.htmlhttps://ipac-canada.org/cnisp-publications.php

Objectives

• Review process and outcome measures in infection prevention and control

• Review impact of reportable HAI surveillance indicators

• Highlight pitfalls in accurately reporting data from various sources• Case study

Variability in reporting

• What: clinical diagnosis vs standard definition

• Who: infection control professional, coders, clinicians in admin roles

• When: continuously or periodically

• Where: internal and external reporting authorities

• How are data reported: numerators and denominators

What is reported

• Sources of surveillance data:• Patient chart/records (e.g. pharmacy, medical imaging)

• Laboratory reports

• Safety reports

• Clinical rounds

• Communication with caregivers

• Decision Support (coders)

• Surveillance-specific forms (e.g. NICU patients with central lines by weight category)

• Surgical Information System

• Critical Care Information System

What is reported

• Clinical diagnosis vs surveillance definitions

• Critically important to have uniform application of standardized and validated case definitions• For both outcome and process measures

• Unique challenges to pediatric surveillance:• Surgical site infection

• ICD-10 codes do not have specific codes for pediatric procedures e.g. Tetralogy of Fallot

• Contentious diagnosis (“purulent drainage” vs “incision clean and dry” by MDs)

• C.difficile infection

Case: Clostridium difficile reporting

• Colonization is common in children under 2 yo• Higher rates of colonization, lower rates of complications compared to adults

• Benchmarked against adult teaching and community hospitals in provincial reporting

• IDSA “strongly discourages” public reporting of cases in children <2 yo• CNISP and most provincial ministries of health report rates among 1-18 yo

• Alberta moving to 2-18 yo

= What gets measured as a quality metric?

• Case definition for CNISP reporting:• 3 x loose, watery stools or fever, abdominal pain and/or ileus• “without reasonable evidence of another cause of diarrhea”

• >70% cases found to have alternate pathogen present

= Requires individual chart review + lab confirmation

Clinical Practice Guidelines for Clostridium difficile infection: IDSA 2018AMMI Canada treatment practice guidelines for Clostridium difficile: 2018

When are HAI data reported?

• Continuously or concurrently

• Periodically (retrospective)

• Challenges• Human resources for case finding and review

• Different case definitions among reporting bodies

• May not be able to determine “truth” when retrospectively reviewing data due to numerous sources

Who does the reporting?

• Infection control professional

• Coder from Decision Support

• Clinical team: notifies IPAC to review if clinical suspicion of HAI

• Clinical nurse: temporary administrative role to support QI initiative

Where do HAI data go?

• Internal committee: IPAC, Quality and Safety; Executive, Board

• Provincial mandatory reporting

• Quality improvement / surveillance programs• Solutions for Patient Safety (US and Canadian network of pediatric hospitals)

• National Surgical Quality Improvement Program

• Canadian Nosocomial Infection Surveillance Program

How are HAI reported?

• Numerators: • Based on case definition; can differ between surveillance programs

• e.g.: CSF shunt infection for CNISP = up to 1 year post-op • SPS = up to 90 days post-op

• We can create workflow process, but which rate to report internally?

• Denominators:• Not consistent between surveillance programs

Denominator NSQIP SPS CNISP

Cardiovascular surgery surveillance No Yes Yes

General surgery surveillance Yes No No

Pericardial window procedures No Yes No

Case Study:

• Hospital IPAC program prospectively identifies HAIs• Submits data to CNISP, Ministry of Health, Critical Care Services Ontario

• 2016: Hospital joined 2 surveillance networks:• Solutions for Patient Safety to reduce risk of CLABSI, SSI

• CV surgery requiring bypass, spinal fusion, neurosurgical shunt infections

• IPAC program central to surveillance strategy and development of process audits

• National Surgical Quality Improvement Program-Pediatrics network• Data abstractor hired by NSQIP team to prospectively collect preoperative,

intraoperative and postoperative data, with outcomes at 30 days after index surgery• Surveillance every 8 days x 35 cases, no dental or CV surgery

• Data submitted directly to NSQIP and anonymized in network publications

• Hospital benchmarked against similar institutions, info sent to hospital leadership

Case Study:

• 2017: CIHI approached hospital in advance of launching “In-hospital infections indicators”• 2 publicly reportable indicators: MRSA, C.difficile

• Meant to complement Patient Safety indicators (e.g. in-hospital sepsis) with site-specific data

• Infection data to be collected by coders

• Coders review clinical chart and code for HAI if documented in physician’s note

Case study: Current state

• CNISP• ICP collects data

• Most of CNISP data reported to Solutions for Patient Safety, MOH

• NSQIP: • Reported directly to NSQIP and internally by clinical administrator

• ICP may receive data to validate, with short turnaround time

• Solutions for Patient Safety• ICP collects data, sends to clinical administrator to report internally and to SPS

• CIHI• No published HAI indicator

Challenges = human resource-intensive; various denominators for same measureWhich one reflects the truth? Which one to follow in quality improvement?

• Case studies published in AJIC with link to online survey• Multiple-choice questions based on standard surveillance criteria and protocols

• Assessed accuracy and consistency in applying standardized surveillance definitions

• 62.5% responses correct • ICPs responded correctly (62%) significantly more often than physicians (55%)

• Highlights need for continuing education, competency development, auditing

How to overcome challenges in data quality?

• One source of information internally and externally• All data collected by IPAC

• Workload? Opportunity costs?

• All data reviewed by IPAC• Forum (if no IPAC Committee)? Frequency? Responsibility if inaccurate data collection?

• HAI case finding and reporting• How to assess competency in applying case definitions within the

organization?

• How to measure accuracy of reporting

Acknowledgements

• Pat Bedard, RN, CIC

• Allyson Shephard, RN, CIC