Assessing Patient Safety through Assessing Patient Safety through Administrative Data:Administrative Data:

Adapting and Improving Existing SystemsAdapting and Improving Existing Systems

Patrick S. Romano, MD MPHPatrick S. Romano, MD MPHProfessor of Medicine and PediatricsProfessor of Medicine and Pediatrics

UC Davis School of Medicine UC Davis School of Medicine Sacramento CA, USASacramento CA, USA

June 29, 2006June 29, 2006

AcknowledgmentsAcknowledgments

Support for Quality Indicators II (Contract No. 290-04-0020) Mamatha Pancholi, AHRQ Project Officer Marybeth Farquhar, AHRQ QI Senior Advisor Mark Gritz and Jeffrey Geppert, Project Directors,

Battelle Health and Life Sciences Kathryn McDonald (PI) and Sheryl Davies (project Kathryn McDonald (PI) and Sheryl Davies (project

manager), Stanford Universitymanager), Stanford University Other clinical team members: Douglas Payne Other clinical team members: Douglas Payne

(medicine), Garth Utter (surgery), Shagufta Yasmeen (medicine), Garth Utter (surgery), Shagufta Yasmeen (obstetrics & gynecology), Corinna Haberland (obstetrics & gynecology), Corinna Haberland (pediatrics), Banafsheh Sadeghi (research assistant)(pediatrics), Banafsheh Sadeghi (research assistant)

OverviewOverview

General approaches to assessing inpatient safetyGeneral approaches to assessing inpatient safety Rationale for using administrative data: strengths Rationale for using administrative data: strengths

and limitationsand limitations Background about the AHRQ Quality Indicators Background about the AHRQ Quality Indicators

programprogram Development and maintenance of the AHRQ Development and maintenance of the AHRQ

Patient Safety Indicators (PSIs)Patient Safety Indicators (PSIs) OECD international expert panel reviewOECD international expert panel review International interest in the AHRQ PSIsInternational interest in the AHRQ PSIs Practical issues associated with international Practical issues associated with international

application of the AHRQ PSIsapplication of the AHRQ PSIs

Taxonomy of patient safety Taxonomy of patient safety measuresmeasures

Donabedian’sDonabedian’sclassificationclassification ExamplesExamplesStructural measuresStructural measures Hospital designHospital design

Staffing (intensity, training)Staffing (intensity, training)Decision support systemsDecision support systemsSafety cultureSafety culture

Process measuresProcess measures Medication errors (incorrect Medication errors (incorrect dosing, inappropriate use)dosing, inappropriate use)Medical errorsMedical errorsNear missesNear misses

Outcome measuresOutcome measures Adverse events (potentially Adverse events (potentially preventable complications, preventable complications, medical injuries)medical injuries)

Zhan et al., Med Care 2005;43:I42-I47

General approaches to General approaches to assessing inpatient safetyassessing inpatient safety

Analyze administrative data (adverse events, Analyze administrative data (adverse events, selected types of medical errors)selected types of medical errors)

Review medical records (adverse events, selected Review medical records (adverse events, selected types of medical errors)types of medical errors)

Collect confidential provider reports of “incidents” Collect confidential provider reports of “incidents” or “safety events” (passive surveillance of medical or “safety events” (passive surveillance of medical errors or near misses)errors or near misses)

Conduct active surveillance or real-time observation Conduct active surveillance or real-time observation (same)(same)

Survey patientsSurvey patients Survey employees or managers on organizational Survey employees or managers on organizational

capabilities or climate (“culture of safety”)capabilities or climate (“culture of safety”)

Ethnographic observation to Ethnographic observation to identify adverse events and errorsidentify adverse events and errors

Andrews LB, et al. Lancet 1997;349:309-13.“Ethnographers trained in qualitative observational research attended regularly scheduled attending rounds, resident work rounds, nursing shift changes, case conferences, and other scheduled meetings” (e.g., M&M conferences, QA meetings) on 3 units at one teaching hospital.

480 of 1047 patients (46%) experienced a mean of 4.5 events

Oakley, E. et al. Pediatrics 2006;117:658-664

Video recording to identify errors in pediatric trauma resuscitation

Mean of 5.9 errors per resuscitation, with 93% agree-ment between 2 reviewers.

Mean of 2.2 errors in each seriously injured child, with 20% capture on medical records

Rationale for using Rationale for using administrative dataadministrative data

LimitationsLimitations– Limited/no information on Limited/no information on

processes of care and processes of care and physiologic measures of physiologic measures of severity severity

– Limited/no information on Limited/no information on timing (comorbidities vs. timing (comorbidities vs. adverse events) adverse events)

– Heterogeneous severity Heterogeneous severity within some ICD codeswithin some ICD codes

– Accuracy depends on Accuracy depends on documentation and codingdocumentation and coding

– Data are used for other Data are used for other purposes, subject to gamingpurposes, subject to gaming

– Time lag limits usefulnessTime lag limits usefulness

OpportunitiesOpportunities– Data availability improvingData availability improving

– Coding systems and Coding systems and practices improvingpractices improving

– Large data sets optimize Large data sets optimize precisionprecision

– Comprehensiveness (all Comprehensiveness (all hospitals, all payers) avoids hospitals, all payers) avoids sampling/selection bias sampling/selection bias

– Data are used for other Data are used for other purposes, subject to purposes, subject to auditing and monitoringauditing and monitoring

AHRQ Quality Indicators (QIs)AHRQ Quality Indicators (QIs)

Developed through contracts with UC-Stanford Developed through contracts with UC-Stanford Evidence-based Practice CenterEvidence-based Practice Center

Use existing hospital discharge data, based on Use existing hospital discharge data, based on readily available data elementsreadily available data elements

Incorporate severity adjustment methods (APR-Incorporate severity adjustment methods (APR-DRGs, comorbidity groupings) when possibleDRGs, comorbidity groupings) when possible

Offer free, downloadable software (SAS, Offer free, downloadable software (SAS, Windows) with documentation, biennial updates, Windows) with documentation, biennial updates, and user support through listserve, newsletters, and user support through listserve, newsletters, national meetings, web seminars, e-mail systemnational meetings, web seminars, e-mail system

User feedback drives continuous improvementUser feedback drives continuous improvement

Inpatient QIsInpatient QIs

MortalityMortalityUtilizationUtilization

VolumeVolume

AHRQ Quality IndicatorsAHRQ Quality Indicators

Prevention QIsPrevention QIs

(Area Level)(Area Level)Avoidable HospitalizationsAvoidable HospitalizationsOther Avoidable ConditionsOther Avoidable Conditions

Patient Safety IndicatorsPatient Safety Indicators

ComplicationsComplicationsFailure-to-rescueFailure-to-rescueUnexpected deathUnexpected death

Pediatric Pediatric QIsQIs

Structure of indicatorsStructure of indicators

Definitions based onDefinitions based on– ICD-9-CM diagnosis and procedure codes ICD-9-CM diagnosis and procedure codes – Inclusion/exclusion criteria based upon DRGs, sex, Inclusion/exclusion criteria based upon DRGs, sex,

age, procedure dates, admission typeage, procedure dates, admission type Numerator = number of cases “flagged” with Numerator = number of cases “flagged” with

the complication or situation of interest the complication or situation of interest – e.g., postoperative sepsis, avoidable hospitalization e.g., postoperative sepsis, avoidable hospitalization

for asthma, deathfor asthma, death Denominator = number of patients considered Denominator = number of patients considered

to be at risk for that complication or situationto be at risk for that complication or situation– e.g. elective surgical patients, county population e.g. elective surgical patients, county population

from census datafrom census data Indicator “rate” = numerator/denominatorIndicator “rate” = numerator/denominator

Literature review (all)Literature review (all)– To identify quality concepts and potential indicators To identify quality concepts and potential indicators

– To find previous work on indicator validityTo find previous work on indicator validity

ICD-9-CM coding review (all)ICD-9-CM coding review (all)– To ensure correspondence between clinical concept and coding To ensure correspondence between clinical concept and coding

practicepractice

Clinical panel reviews (PSI’s, pediatric QIs)Clinical panel reviews (PSI’s, pediatric QIs)– To refine indicator definition and risk groupingsTo refine indicator definition and risk groupings

– To establish face validity when minimal literature To establish face validity when minimal literature

Empirical analyses (all)Empirical analyses (all)– To explore alternative definitionsTo explore alternative definitions

– To assess nationwide rates, hospital variation, relationships among To assess nationwide rates, hospital variation, relationships among indicatorsindicators

– To develop methods to account for differences in riskTo develop methods to account for differences in risk

AHRQ QI development: AHRQ QI development: General processGeneral process

Literature review to find Literature review to find candidate PSI indicatorscandidate PSI indicators

MEDLINE/EMBASE search guided by medical MEDLINE/EMBASE search guided by medical librarians at Stanford and NCPCRD (UK)librarians at Stanford and NCPCRD (UK)– Few examples described in peer reviewed journalsFew examples described in peer reviewed journals

Iezzoni et al.’s Complications Screening Program Iezzoni et al.’s Complications Screening Program (CSP)(CSP)

Miller et al.’s Patient Safety IndicatorsMiller et al.’s Patient Safety Indicators Review of ICD-9-CM code bookReview of ICD-9-CM code book Codes from above sources were grouped into Codes from above sources were grouped into

clinically coherent indicators with appropriate clinically coherent indicators with appropriate denominatorsdenominators

Coding (criterion) validity based on Coding (criterion) validity based on literature review (MEDLINE/EMBASE)literature review (MEDLINE/EMBASE)

Validation studies of Iezzoni et al.’s CSPValidation studies of Iezzoni et al.’s CSP– At least one of three validation studies (coders, At least one of three validation studies (coders,

nurses, or physicians) confirmed PPV at least nurses, or physicians) confirmed PPV at least 70% among flagged cases70% among flagged cases

– Nurse-identified process-of-care failures were Nurse-identified process-of-care failures were more prevalent among flagged cases than more prevalent among flagged cases than among unflagged controlsamong unflagged controls

Other studies of coding validityOther studies of coding validity– Very few in peer-reviewed journals, some in Very few in peer-reviewed journals, some in

“gray literature”“gray literature”

Construct validity based on literature Construct validity based on literature review (MEDLINE/EMBASE)review (MEDLINE/EMBASE)

Approaches to assessing construct validityApproaches to assessing construct validity– Is the outcome indicator associated with explicit Is the outcome indicator associated with explicit

processes of care (e.g., appropriate use of processes of care (e.g., appropriate use of medications)?medications)?

– Is the outcome indicator associated with implicit Is the outcome indicator associated with implicit process of care (e.g., global ratings of quality)?process of care (e.g., global ratings of quality)?

– Is the outcome indicator associated with nurse Is the outcome indicator associated with nurse staffing or skill mix, physician skill mix, or other staffing or skill mix, physician skill mix, or other aspects of hospital structure?aspects of hospital structure?

ICD-9-CM coding consultant reviewICD-9-CM coding consultant review

All definitions were reviewed by an expert All definitions were reviewed by an expert coding consultant from the American Health coding consultant from the American Health Information Management Association, with Information Management Association, with special attention to prior coding guidelinesspecial attention to prior coding guidelines

Central staff of ICD-9-CM were queried as Central staff of ICD-9-CM were queried as necessarynecessary

Definitions were refined as appropriateDefinitions were refined as appropriate

Face validity: Face validity: Clinical panel reviewClinical panel review

Intended to establish consensual validityIntended to establish consensual validity Modified RAND/UCLA Appropriateness Modified RAND/UCLA Appropriateness

MethodMethod Physicians of various specialties and Physicians of various specialties and

subspecialties, nurses, other specialized subspecialties, nurses, other specialized professionals (e.g., midwife, pharmacist)professionals (e.g., midwife, pharmacist)

Potential indicators were rated by 8 Potential indicators were rated by 8 multispecialty panels; surgical indicators were multispecialty panels; surgical indicators were also rated by 3 surgical panelsalso rated by 3 surgical panels

Face validity: Face validity: Clinical panel review (cont’d)Clinical panel review (cont’d)

All panelists rated all assigned indicators on: All panelists rated all assigned indicators on: – Overall usefulnessOverall usefulness– Likelihood of identifying the occurrence of an Likelihood of identifying the occurrence of an

adverse event or complication (i.e., not present at adverse event or complication (i.e., not present at admission)admission)

– Likelihood of being preventable (i.e., not an Likelihood of being preventable (i.e., not an expected result of underlying conditions) expected result of underlying conditions)

– Likelihood of being due to medical error or Likelihood of being due to medical error or negligence (i.e., not just lack of ideal or perfect negligence (i.e., not just lack of ideal or perfect care)care)

– Likelihood of being clearly charted Likelihood of being clearly charted – Extent to which indicator is subject to case mix Extent to which indicator is subject to case mix

biasbias

Medical error and complications continuum

Evaluation frameworkEvaluation framework

Pre-conference ratings and comments/suggestionsPre-conference ratings and comments/suggestions Individual ratings returned to panelists with Individual ratings returned to panelists with

distribution of ratings and other panelists’ distribution of ratings and other panelists’ comments/suggestionscomments/suggestions

Telephone conference call moderated by PI and Telephone conference call moderated by PI and attended by note-taker, focusing on high-variability attended by note-taker, focusing on high-variability items and panelists’ suggestions (90-120 mins)items and panelists’ suggestions (90-120 mins)

Suggestions adopted only by consensusSuggestions adopted only by consensus Post-conference ratings and comments/ suggestionsPost-conference ratings and comments/ suggestions

Medical error NonpreventableComplications

Example reviewsExample reviewsMultispecialty panelsMultispecialty panels

Overall ratingOverall rating

Not present on Not present on admissionadmission

Preventability (4)Preventability (4)

Due to medical error Due to medical error (2)(2)

Charting by Charting by physicians (6)physicians (6)

Not biased (3)Not biased (3)

(5)

(7)

(4)

(2)

(6)

(3)

(8)

(8)

(8)

(8)

(7)

(7)

Postop Pneumonia Decubitus Ulcer

Final selection of indicatorsFinal selection of indicators

Retained indicators for which “overall usefulness” rating Retained indicators for which “overall usefulness” rating was “Acceptable” or “Acceptable-” was “Acceptable” or “Acceptable-” : : – Median score 7-9Median score 7-9– Definite or indeterminate agreementDefinite or indeterminate agreement

Excluded indicators rated “Unclear,” “Unclear-,” or Excluded indicators rated “Unclear,” “Unclear-,” or “Unacceptable”“Unacceptable”: : – Median score <7, ORMedian score <7, OR– At least 2 panelists rated the indicator in each of the extreme 3-point At least 2 panelists rated the indicator in each of the extreme 3-point

rangesranges

Candidate PSIs reviewedCandidate PSIs reviewed

48 indicators reviewed in total48 indicators reviewed in total– 37 reviewed by multispecialty panel37 reviewed by multispecialty panel– 15 of those reviewed by surgical panel15 of those reviewed by surgical panel

20 “accepted” based on face validity20 “accepted” based on face validity– 2 dropped due to operational concerns2 dropped due to operational concerns

17 “experimental” or promising indicators17 “experimental” or promising indicators 11 rejected11 rejected

““Accepted” PSIsAccepted” PSIsSelected postop complicationsSelected postop complications Postoperative thromboembolism Postoperative respiratory failure Postoperative sepsis Postoperative physiologic and

metabolic derangements Postoperative abdominopelvic

wound dehiscence Postoperative hip fracture Postoperative hemorrhage or

hematomaSelected technical adverse eventsSelected technical adverse events Decubitus ulcer Selected infections due to medical

careTechnical difficulty with proceduresTechnical difficulty with procedures Iatrogenic pneumothorax Accidental puncture or laceration Foreign body left in during procedure

Other Other Complications of anesthesia Death in low mortality DRGs Failure to rescue Transfusion reaction (ABO/Rh)

Obstetric trauma and birth Obstetric trauma and birth traumatrauma

Birth trauma – injury to neonate Obstetric trauma – vaginal

delivery with instrument Obstetric trauma – vaginal

delivery without instrument Obstetric trauma – cesarean

section delivery

Pediatric Quality IndicatorsPediatric Quality Indicators

Inpatient IndicatorsInpatient Indicators– Accidental puncture and lacerationAccidental puncture and laceration

– Decubitus ulcerDecubitus ulcer

– Foreign body left in after procedureForeign body left in after procedure

– Iatrogenic pneumothorax in neonates at riskIatrogenic pneumothorax in neonates at risk

– Iatrogenic pneumothorax in non-neonatesIatrogenic pneumothorax in non-neonates

– Pediatric heart surgery mortalityPediatric heart surgery mortality

– Pediatric heart surgery volumePediatric heart surgery volume

– Postoperative hemorrhage or hematomaPostoperative hemorrhage or hematoma

– Postoperative respiratory failurePostoperative respiratory failure

– Postoperative sepsisPostoperative sepsis

– Postoperative wound dehiscence due to medical carePostoperative wound dehiscence due to medical care

– Transfusion reactionTransfusion reaction

PSI risk adjustment methodsPSI risk adjustment methods

Must use only administrative dataMust use only administrative data APR-DRGs and other canned packages APR-DRGs and other canned packages

may adjust for complicationsmay adjust for complications Final model Final model

– DRGs (complication DRGs aggregated) DRGs (complication DRGs aggregated)

– Modified Comorbidity Index based on list Modified Comorbidity Index based on list developed by Elixhauser et al. (completely developed by Elixhauser et al. (completely redesigned for Pediatric QIs)redesigned for Pediatric QIs)

– Age, Sex, Age-Sex interactions Age, Sex, Age-Sex interactions

Pediatric QI Risk AdjustmentPediatric QI Risk Adjustment

Reason for admission/type of procedureReason for admission/type of procedure– DRGs (with/without CC collapsed)DRGs (with/without CC collapsed)– Other (e.g., diagnostic/therapeutic procedure Other (e.g., diagnostic/therapeutic procedure

categories for accidental injury)categories for accidental injury) ComorbidityComorbidity

– Special pediatric-oriented comorbidity listSpecial pediatric-oriented comorbidity list Gender, age groupsGender, age groups

– <29 d, 29-60 d, 61-90 d, 91-365 d, 1-2 yrs, 3-5 <29 d, 29-60 d, 61-90 d, 91-365 d, 1-2 yrs, 3-5 yrs, 6-12 yrs, 13-17 yrsyrs, 6-12 yrs, 13-17 yrs

Low birth weight categories (neonates)Low birth weight categories (neonates)– 500 gram categories (500-2500 g)500 gram categories (500-2500 g)

OECD Health Care Quality OECD Health Care Quality Indicators ProjectIndicators Project

Includes 21 countries, WHO, European Includes 21 countries, WHO, European Commission, World Bank, ISQua, etc.Commission, World Bank, ISQua, etc.

Five priority areasFive priority areas– Cardiac careCardiac care

– Diabetes mellitusDiabetes mellitus

– Mental healthMental health

– Patient safetyPatient safety

– Prevention/health promotion and primary Prevention/health promotion and primary carecare

OECD Indicator Selection CriteriaOECD Indicator Selection Criteria ImportanceImportance

– Impact on healthImpact on health

– Policy importance (concern for policymakers and Policy importance (concern for policymakers and consumers)consumers)

– Susceptibility to being influenced by the health care Susceptibility to being influenced by the health care systemsystem

Scientific soundnessScientific soundness

– Face validity (clinical rationale and past usage)Face validity (clinical rationale and past usage)

– Content validityContent validity FeasibilityFeasibility

– Data availability on the international levelData availability on the international level

– Reporting burdenReporting burden

OECD Review ProcessOECD Review Process

Patient safety panel constituted with 5 members Patient safety panel constituted with 5 members (Dr. John Millar, Chair)(Dr. John Millar, Chair)

59 indicators from 7 sources submitted for review 59 indicators from 7 sources submitted for review (US, Canada, Australia)(US, Canada, Australia)

Modified RAND/UCLA Appropriateness MethodModified RAND/UCLA Appropriateness Method Panelists rated each indicator on importance and Panelists rated each indicator on importance and

scientific soundness (2 rounds with intervening scientific soundness (2 rounds with intervening discussion)discussion)

Retained 21 indicators with median score >7 Retained 21 indicators with median score >7 (scale 1-9) on both domains; rejected indicators (scale 1-9) on both domains; rejected indicators with median score with median score ≤≤5 on either domain5 on either domain

OECD expert panel ratings of PSIsOECD expert panel ratings of PSIs

PSIs recommended

PSIs not recommended

Experimental or rejected PSIs recommended

Selected infections due to medical care

Death in low mortality DRG Postop wound infection

Decubitus ulcer Postop hemorhage/ hematoma In-hospital hip fracture or fall Complications of anesthesia Iatrogenic pneumothorax Postop PE or DVT Postop abdominopelvic wound

dehiscence

Postop sepsis Failure to rescue Technical difficulty with procedure

Postop physiologic/ metabolic derangement

Transfusion reaction Postop respiratory failure Foreign body left in Postop hip fracture Birth trauma Obstetric trauma (all delivery types)

AHRQ panel ratings of PSI “preventability”AHRQ panel ratings of PSI “preventability”very similar to OECD ratingsvery similar to OECD ratings

Acceptable Acceptable (-) Unclear Unclear (-) Decubitus ulcer Complications of

anesthesia Death in low mortality DRG

Failure to rescue

Foreign body left in Selected infections due to medical care

Postop hemorhage/ hematoma

Postop physiologic/ metabolic derangement

Iatrogenic pneumothoraxa

Postop PE or DVTb Postop respiratory failure

Postop hip fracturea Transfusion reaction Postop abdominopelvic wound dehiscence

Technical difficulty with procedure

Birth trauma Postop sepsis

Obstetric trauma (all delivery types)

a Panel ratings were based on definitions different than final definitions. For “Iatrogenic pneumothorax,” the rated denominator was restricted to patients receiving thoracentesis or central lines; the final definition expands the denominator to all patients (with same exclusions). For “In-hospital fracture” panelists rated the broader Experimental indicator, which was replaced in the Accepted set by “Postoperative hip fracture” due to operational concerns. b Vascular complications were rated as Unclear (-) by surgical panel; multispecialty panel rating is shown here.

US rates of OECD-endorsed PSIsUS rates of OECD-endorsed PSIs

Patient Safety IndicatorPatient Safety Indicator

2003 2003 eventsevents

2003 rate 2003 rate per 1,000per 1,000

COMPLICATIONS OF ANESTHESIA COMPLICATIONS OF ANESTHESIA 7,4067,406 0.7750.775

DECUBITUS ULCER DECUBITUS ULCER 198,752198,752 23.36523.365

FOREIGN BODY LEFT IN DURING PROC FOREIGN BODY LEFT IN DURING PROC 2,7412,741 0.0860.086

INFECTION DUE TO MEDICAL CARE INFECTION DUE TO MEDICAL CARE 43,59143,591 2.0522.052

POSTOPERATIVE HIP FRACTURE POSTOPERATIVE HIP FRACTURE 1,5111,511 0.2790.279

POSTOPERATIVE PE OR DVT POSTOPERATIVE PE OR DVT 80,47780,477 9.8839.883

POSTOPERATIVE SEPSIS POSTOPERATIVE SEPSIS 10,43510,435 10.46310.463

ACCIDENTAL PUNCTURE/LACERATIONACCIDENTAL PUNCTURE/LACERATION 97,05897,058 3.5743.574

TRANSFUSION REACTION TRANSFUSION REACTION 151151 0.0050.005

BIRTH TRAUMA -INJURY TO NEONATEBIRTH TRAUMA -INJURY TO NEONATE 22,06122,061 5.4125.412

OB TRAUMA - VAGINAL W INSTRUMENT OB TRAUMA - VAGINAL W INSTRUMENT 55,50255,502 189.576189.576

OB TRAUMA - VAGINAL W/O INSTRUMENT OB TRAUMA - VAGINAL W/O INSTRUMENT 116,707116,707 45.21945.219

Primary uses of the AHRQ PSIsPrimary uses of the AHRQ PSIs

Internal hospital quality improvementInternal hospital quality improvement– Individual hospitals and health care systems, Individual hospitals and health care systems,

hospital associationshospital associations– Trigger case finding, root cause analyses, Trigger case finding, root cause analyses,

identification of clusters identification of clusters – Evaluate impact of local interventionsEvaluate impact of local interventions– Monitor performance over timeMonitor performance over time

External hospital accountability to the External hospital accountability to the communitycommunity

National, State and regional analysesNational, State and regional analyses– National Healthcare Quality/Disparities ReportsNational Healthcare Quality/Disparities Reports– Surveillance of trends over timeSurveillance of trends over time– Disparities across areas, SES strata, ethnicitiesDisparities across areas, SES strata, ethnicities

Relative change from 1999-2000 to 2002-2003 in Relative change from 1999-2000 to 2002-2003 in observed and risk-adjusted AHRQ PSI ratesobserved and risk-adjusted AHRQ PSI rates

Patient Safety IndicatorPatient Safety Indicator % change% changeObservedObserved

% change% changeRisk-adjustedRisk-adjusted

COMPLICATIONS OF ANESTHESIA COMPLICATIONS OF ANESTHESIA 14.7%14.7% 13.7%13.7%

DECUBITUS ULCER DECUBITUS ULCER 12.1%12.1% 11.7%11.7%

FOREIGN BODY LEFT IN DURING PROC FOREIGN BODY LEFT IN DURING PROC 4.5%4.5%

INFECTION DUE TO MEDICAL CARE INFECTION DUE TO MEDICAL CARE 13.8%13.8% 11.0%11.0%

POSTOPERATIVE HIP FRACTURE POSTOPERATIVE HIP FRACTURE -8.4%-8.4% -12.2%-12.2%

POSTOPERATIVE PE OR DVT POSTOPERATIVE PE OR DVT 25.3%25.3% 26.6%26.6%

POSTOPERATIVE SEPSIS POSTOPERATIVE SEPSIS 15.6%15.6% 14.7%14.7%

ACCIDENTAL PUNCTURE/LACERATIONACCIDENTAL PUNCTURE/LACERATION 3.1%3.1% 3.9%3.9%

TRANSFUSION REACTIONTRANSFUSION REACTION 13.2%13.2%

BIRTH TRAUMA -INJURY TO NEONATEBIRTH TRAUMA -INJURY TO NEONATE -8.3%-8.3% -8.3%-8.3%

OB TRAUMA - VAGINAL W INSTRUMENT OB TRAUMA - VAGINAL W INSTRUMENT -10.1%-10.1% -9.4%-9.4%

OB TRAUMA - VAGINAL W/O INSTRUMENT OB TRAUMA - VAGINAL W/O INSTRUMENT -15.3%-15.3% -14.9%-14.9%

Newer uses of the AHRQ PSIsNewer uses of the AHRQ PSIs

Testing research hypotheses related to patient Testing research hypotheses related to patient safetysafety– Housestaff work hours reformHousestaff work hours reform– Nurse staffing regulationNurse staffing regulation

Public reporting by hospitalPublic reporting by hospital– Texas, New York, Colorado, Oregon, Texas, New York, Colorado, Oregon,

Massachusetts, Wisconsin, Florida, UtahMassachusetts, Wisconsin, Florida, Utah Pay-for-performance by hospitalPay-for-performance by hospital

– CMS/Premier Demonstration (278 hospitals, focus CMS/Premier Demonstration (278 hospitals, focus on 2 postop events after THA/TKA)on 2 postop events after THA/TKA)

– Anthem of Virginia (focus on monitoring any two)Anthem of Virginia (focus on monitoring any two) Hospital profilingHospital profiling

– Blue Cross/Blue Shield of IllinoisBlue Cross/Blue Shield of Illinois

International inquiries International inquiries regarding the AHRQ QIsregarding the AHRQ QIs

Canada 58

Spain 3

Italy 15

Australia 7

Belgium 5

South Africa 1

Philippines 1

Slovenia 1

Taiwan 3

Switzerland 1

Romania 3

New Zeland 4

Argentina 2

Portugal 1

United Kingdom 1

Japan 3

Germany 7

France 1

Indonesia 2

Saudi Arabia 2

Guyana 1

International inquiries International inquiries regarding the AHRQ QIsregarding the AHRQ QIs

Quality Indicator ModuleQuality Indicator Module NumberNumberPrevention Quality IndicatorsPrevention Quality Indicators 1515Inpatient Quality IndicatorsInpatient Quality Indicators 4646Patient Safety IndicatorsPatient Safety Indicators 7474Pediatric Quality IndicatorsPediatric Quality Indicators 11No specific moduleNo specific module 5151

Practical issues in international Practical issues in international implementation of AHRQ PSIsimplementation of AHRQ PSIs

ICD-9-CM to ICD-10 conversionICD-9-CM to ICD-10 conversion– Entirely different coding structureEntirely different coding structure– Three new chaptersThree new chapters– 12,420 codes versus 6,96912,420 codes versus 6,969– Nation-specific versions (CA, AU, GM)Nation-specific versions (CA, AU, GM)

No internationally accepted coding No internationally accepted coding system for proceduressystem for procedures

Practical issues in international Practical issues in international implementation of AHRQ PSIsimplementation of AHRQ PSIs

Variation in documentation and coding practicesVariation in documentation and coding practices Variation in other data definitionsVariation in other data definitions

– Principal versus primary diagnosisPrincipal versus primary diagnosis– Number of diagnosis codesNumber of diagnosis codes– Procedure datesProcedure dates– External cause of injury codesExternal cause of injury codes– Type of admission (elective, urgent, emergency)Type of admission (elective, urgent, emergency)

Variation in how administrative data are Variation in how administrative data are collected and usedcollected and used– DRG-based payment versus global budgeting versus DRG-based payment versus global budgeting versus

service-based paymentservice-based payment

Coding of secondary Coding of secondary diagnoses in the USAdiagnoses in the USA

For reporting purposes the definition for "other diagnoses" For reporting purposes the definition for "other diagnoses" is interpreted as additional conditions that affect patient is interpreted as additional conditions that affect patient care in terms of requiring:care in terms of requiring:clinical evaluation; orclinical evaluation; ortherapeutic treatment; ortherapeutic treatment; ordiagnostic procedures; ordiagnostic procedures; orextended length of hospital stay; orextended length of hospital stay; orincreased nursing care and/or monitoring.increased nursing care and/or monitoring.

““All conditions that occur following surgery…are not All conditions that occur following surgery…are not complications… there must be more than a routinely complications… there must be more than a routinely expected condition or occurrence… there must be a expected condition or occurrence… there must be a cause-and-effect relationship between the care provided cause-and-effect relationship between the care provided and the condition…”and the condition…”

ICD-9-CM Coding: ICD-9-CM Coding: ProceduresProcedures

Coding of proceduresCoding of procedures

““The UHDDS requires all significant procedures to be The UHDDS requires all significant procedures to be reported… A significant procedure is defined as one reported… A significant procedure is defined as one that meets any of the following conditions:that meets any of the following conditions:Is surgical in natureIs surgical in natureCarries an anesthetic riskCarries an anesthetic riskCarries a procedural riskCarries a procedural riskRequires specialized training.”Requires specialized training.”

What about central venous catheters?What about central venous catheters?

International initiativesInternational initiatives

Conversion efforts are underway, but need to Conversion efforts are underway, but need to be coordinated internationallybe coordinated internationally

Undertake detailed meta-analysis of national Undertake detailed meta-analysis of national data systems data systems

Review international variation in coding rules Review international variation in coding rules and procedures and procedures

Improve data systems (e.g., “present at Improve data systems (e.g., “present at admission” coding in USA) and develop data admission” coding in USA) and develop data on accuracyon accuracy

Prioritize indicators based on likelihood of Prioritize indicators based on likelihood of international comparabilityinternational comparability

International collaborative meeting of health International collaborative meeting of health services researchers using administrative dataservices researchers using administrative data

Project Mean score Number of raters

Priority Rank

Meta-Data documentation 8.4 10 1 Patient safety indicators translation 8.0 10 2 Internal consistency algorithms 7.0 10 3 Interventional studies to enhance coding quality

6.9 10 4

“True” gold standard 6.8 10 5 Travelling coders for comparative recoding 6.8 9 5 Training standards for health records coders 6.4 10 7 International comparisons of predictive model performance (C-statistic)

6.2 10 8

Value of diagnosis-type coding 6.0 9 9 Chart-Database comparison studies 5.9 10 10 International cross-validation of ICD-10 coding algorithms

5.8 10 11

International scan of privacy considerations 5.6 10 12 Learning curve in ICD-10 uptake 5.0 10 13 Calgary, Alberta, June 2005; supported by CIHR; forthcoming in BMC HSR

Conversion of Elixhauser comorbidity list Conversion of Elixhauser comorbidity list from ICD-9-CM to ICD-10, ICD-10-CAfrom ICD-9-CM to ICD-10, ICD-10-CA

0.00

0.20

0.40

0.60

0.80

1.00

32 Conditions

ICD-9-CM vs. Chart

ICD-10 vs. Chart

Quan H, et al., reported at AcademyHealth 2006

German mapping of PSIs from German mapping of PSIs from ICD-9-CM to ICD-10-GMICD-9-CM to ICD-10-GM

Saskia E. Droesler and Juergen Stausberg

PSI incidence comparisonPSI incidence comparisonGermany vs. USAGermany vs. USA

US population rate (log) 2002

German population rate (log) 2004

Developing data on accuracy and relevance: Developing data on accuracy and relevance: AHRQ PSIs in Children’s HospitalsAHRQ PSIs in Children’s Hospitals

Sedman A, et al. Sedman A, et al. PediatricsPediatrics 2005;115(1):135-145 2005;115(1):135-145

PSIPSI No. reviewedNo. reviewed(total events)(total events)

PreventablePreventable(PPV %)(PPV %)

NonpreventableNonpreventable UnclearUnclear

Complications of anesthesia 74 (503) 11 (15%) 37 25

Death in low-mortality DRG 121 (1282) 16 (13%) 89 16

Decubitus ulcer 130 (2300) 71 (55%) 47 10

Failure to rescue 187 (5271) 15 (8%) 148 11

Foreign body left in 49 (235) 25 (51%) 14 10

Postop hemorrhage or hematoma 114 (1571) 40 (35%) 51 23

Iatrogenic pneumothorax 114 (1113) 51 (45%) 42 21

Selected infection 2° to med care 152 (7291) 63 (41%) 45 39

Postop DVT/PE 126 (1956) 36 (29%) 61 29

Postop wound dehiscence 41 (232) 19 (46%) 16 6

Accidental puncture or laceration 133 (4020) 86 (65%) 19 26