Johan de Koning, Bratislava, 2005 Hospital quality indicators Johan de Koning, MPH, PhD Senior...

Johan de Koning, Bratislava, 2005

Hospital quality indicatorsHospital quality indicators

Johan de Koning, MPH, PhDSenior researcher

Department of Social MedicineAcademic Medical CentreUniversity of Amsterdam


Outline presentationOutline presentation

1.1. The Slovak Hospital Indicator Set The Slovak Hospital Indicator Set

2.2. Quality check….!Quality check….!

3.3. Risks and Challenges in indicator use Risks and Challenges in indicator use

3.3. Presenting and Interpreting performance dataPresenting and Interpreting performance data

4.4. Q & AQ & A


Purpose and UsersPurpose and Users

Internal quality improvement

- for management information to monitor, evaluate or improve hospital functioning (long or short term)

External accountability and transparency

- for accountability requested by stakeholders: - National or regional government- Health Insurances companies- Health care surveillance authority- Public?


Criteria for selection of indicatorsCriteria for selection of indicators

The overall importance of the aspects of quality being The overall importance of the aspects of quality being measuredmeasured

- Expected burden of disease- Expected burden of disease

- Effectiveness of the intervention- Effectiveness of the intervention

The scientific soundness of the measuresThe scientific soundness of the measures

The feasibility of collecting data on the indicatorsThe feasibility of collecting data on the indicators

Interest for hospitals and external stakeholders Interest for hospitals and external stakeholders (government, HICs, HCSA).(government, HICs, HCSA).


The Slovak Hospital Indicator SetThe Slovak Hospital Indicator Set

- Partly based on PATH indicators (pre-tested)

- Workshops, hospital pilot's, other meetings

- Visits of project staff to the Netherlands

- Other indicator sets (e.g MOH, international sets)

29 indicators29 indicators - Clinical effectiveness and safety

- Efficiency

- Staff orientation and staff safety

- Patient centeredness


Indicators (hospitals / day surgery centres)Indicators (hospitals / day surgery centres)

Acute Myocardial Infarction Indicators:Acute Myocardial Infarction Indicators:

- Aspirin on Arrival- Aspirin on Arrival

- Aspirin prescribed on discharge- Aspirin prescribed on discharge

- ACEI or ARB for LVSD- ACEI or ARB for LVSD

- Adult smoking cessation advice- Adult smoking cessation advice

- Beta Blockers prescribed on discharge- Beta Blockers prescribed on discharge

- Beta Blockers on arrival- Beta Blockers on arrival

- Mean time to thrombolysis- Mean time to thrombolysis

Heart Failure Indicators:Heart Failure Indicators:

- Discharge instructions- Discharge instructions

- LVF assessment- LVF assessment

- ACEI or ARB for LVSD- ACEI or ARB for LVSD

- Adult smoking cessation advice- Adult smoking cessation advice

Preventing pneumonia in ventilated patientsPreventing pneumonia in ventilated patients

Hip fracture mortality rateHip fracture mortality rate

Acute stroke mortality rateAcute stroke mortality rate

Total hip replacement mortality rateTotal hip replacement mortality rate

AMI mortality rateAMI mortality rate

Primary Caesarean section ratePrimary Caesarean section rate

Cataract day surgery rateCataract day surgery rate

Inguinal hernia day surgery rateInguinal hernia day surgery rate

Readmission for AMIReadmission for AMI

Readmission for pneumoniaReadmission for pneumonia

Cancelled surgeryCancelled surgery

Complaint resolutionComplaint resolution

Infection due to medical careInfection due to medical care

Wound infectionsWound infections

Transfusion reactionsTransfusion reactions

Inpatient hip fractureInpatient hip fracture

Unanticipated admission for cataract day Unanticipated admission for cataract day surgerysurgery

Unanticipated admission for inguinal hernia Unanticipated admission for inguinal hernia day surgeryday surgery


Indicator specificationsIndicator specificationsAMI mortality rateAMI mortality rate

Indicator: ____________ Indicator: ____________

Indicator ID number: ____________Indicator ID number: ____________

Indicator Name: ____________Indicator Name: ____________

Description: ____________Description: ____________

Rationale: ____________Rationale: ____________

Type of measure: ____________Type of measure: ____________

Improvement: ____________Improvement: ____________

Numerator / Denominator: ____________Numerator / Denominator: ____________

Data elements: ____________ Data elements: ____________

Risk adjustment: ____________Risk adjustment: ____________

Source of data: ____________Source of data: ____________

Data reported as: ____________Data reported as: ____________

Upper and lower triggers: ____________Upper and lower triggers: ____________


Appraisal of indicators through Research and Appraisal of indicators through Research and Evaluation (AIRE-instrument)Evaluation (AIRE-instrument) De Koning JS, Klazinga NS, 2005De Koning JS, Klazinga NS, 2005


Appraisal of indicators through Research and Appraisal of indicators through Research and Evaluation (AIRE-instrument)Evaluation (AIRE-instrument) De Koning JS, Klazinga NS, 2005De Koning JS, Klazinga NS, 2005

Indicators evaluated against

- 20 quality items: Statements

- 4 quality domains: 1. Purpose, relevance, focus (5)

2. Involvement of stakeholders (3)

3. Evidence-based (3)

4. Rigour of development, clarity and applicability (9)

- 4-point Likert Scale: (strongly agree – strongly disagree)


AIRE-instrument: quality check…!AIRE-instrument: quality check…!

Are the quality indicator described in unambiguous terms and comprehensible language?

Are all the terms used in the description clearly described?

Are the reasons for applying the indicators explicitly stated?

Are the target groups (e.g. patient groups) of the quality indicators identified?

Is (are) the patient(s) receiving the care to be measured clearly defined

(e.g. as to sex, age, stage of the disease)?

Are the factors influencing the indicator value clearly described?

Are corrective adjustment procedures described that may help to improve inter-performer

comparability? +/-

Are the performance thresholds for the indicators explicitly stated? +/-

Are the sources used to collect the data for the indicators and influencing factors

clearly described?


Field-testingField-testing

Give answers to questions about “feasibility”, “usefulness” Give answers to questions about “feasibility”, “usefulness” and “interpretability” of individual indicators.and “interpretability” of individual indicators.

Reliability / Quality of reporting: consistent identification Reliability / Quality of reporting: consistent identification and reporting of data..?and reporting of data..?

Whether changes of indicators are needed.Whether changes of indicators are needed.


Way forward: risks and Way forward: risks and challenges…!challenges…!


Unintended effects of indictors (1)Unintended effects of indictors (1)

Important lesson learned:

The more direct and serious the consequences of high or

low scores, the more manipulation and perverse reactions…!

i.e. systems ‘less’ punitive, ‘less’ manipulation and perverse

reactions… Induced by high pressure to perform (financial

incentives / punishment).



- - Tunnel visionTunnel vision: strong focus on a few indicators, while other : strong focus on a few indicators, while other aspects get less attention aspects get less attention (risk: (risk: measured performance may measured performance may improve, overall quality may fall). improve, overall quality may fall).

- - Focus on easily measurable indicatorsFocus on easily measurable indicators (e.g. waiting lists for (e.g. waiting lists for non-acute surgery) while more important indicators of unmet non-acute surgery) while more important indicators of unmet need being ignored. need being ignored.

- Creating a - Creating a “short term culture”“short term culture” (focus on this year good (focus on this year good performance replaces more strategic thinking on fundamental performance replaces more strategic thinking on fundamental long term improvements). long term improvements).



ExamplesExamples::

- Waiting times: GPs reduce referral rate / specialists increase - Waiting times: GPs reduce referral rate / specialists increase their thresholds for treatment. their thresholds for treatment.

- Hospital reluctance to deal with high-risk patients or - Hospital reluctance to deal with high-risk patients or procedures: damage of performance profiles. procedures: damage of performance profiles.

- Purposefully underachieve in one year to show steady - Purposefully underachieve in one year to show steady improvement over time (organisations at a high level to start improvement over time (organisations at a high level to start with may look worse). with may look worse).


Case reportCase report

““I was just at a theatre meeting today [where] we were I was just at a theatre meeting today [where] we were discussing that cancellations on day of operation is a bad mark, discussing that cancellations on day of operation is a bad mark,

a cross in the box there. So what do we do? We cancel them a cross in the box there. So what do we do? We cancel them the night before so they are not recorded as cancellations on the night before so they are not recorded as cancellations on

the same day.”the same day.”


Methodological issues (variation & noise)Methodological issues (variation & noise)


Gepubliceerd | zaterdag 8 oktober 2005. Gewijzigd | zaterdag 8 oktober 08:52 uur.

Radboud: verbod deel hartoperatiesVan onze verslaggeefster

AMSTERDAM - Het St. Radboudziekenhuis in Nijmegen moet op last van de Inspectie voor de Gezondheidszorg (IGZ) stoppen met hartklepoperaties bij patiënten met een hoog risico. Een team van externe deskundigen gaat eerst onderzoeken hoe het komt dat het sterftecijfer in het ziekenhuis veel hoger ligt dan in andere hartcentra van Nederland. Het ziekenhuis erkent dat er al langer problemen zijn in de hartkliniek.


Analyzing en interpreting Analyzing en interpreting performance data…!performance data…!


League tablesLeague tables

Technique for displaying comparative rankings of performance Technique for displaying comparative rankings of performance indicators scores for similar providers (e.g. hospitals)indicators scores for similar providers (e.g. hospitals)

Purpose:Purpose:

1.1. Identify (few) providers with scores greater or lower than Identify (few) providers with scores greater or lower than expected expected

2. Show range of variation between providers2. Show range of variation between providers


League tablesLeague tables

Adab, P. et al. BMJ 2002;324:95-98

Ranking

Mean score

Ranking


Advantages and drawbacksAdvantages and drawbacks

Advantage:Advantage:

-Stimulate competition Stimulate competition among hospitals among hospitals

- Encourage managers and Encourage managers and health care insurers to health care insurers to prioritize quality instead of prioritize quality instead of costs onlycosts only

- Identifies poor performing Identifies poor performing clinicians or hospitals, start clinicians or hospitals, start inquiry…! inquiry…!

Drawbacks:Drawbacks:

-Depend on quality of dataDepend on quality of data

- Stimulate “creative Stimulate “creative reporting”reporting”

- Enhance unintended Enhance unintended adverse consequences: adverse consequences: “gaming”“gaming”

- Stimulates a culture of Stimulates a culture of naming and shamingnaming and shaming


Control chartsControl charts

Requirement indicator system: Requirement indicator system:

Sort out “signals” from background “noise”…!Sort out “signals” from background “noise”…!

Special causes thatSpecial causes that Variation explained by Variation explained by need further investigation need further investigation common causes common causes


Common “noise” / special cause “signal”Common “noise” / special cause “signal”

Surgeon specific mortality rates after colorectal cancer surgery HR= adjusted hazard ratios



Adab, P. et al. BMJ 2002;324:95-98



Mean score

Ranking


(A) A ‘‘forest’’ plot showing

95% confidence intervals

compared with the ‘‘target’’

overall average rate.

30 day mortality following coronary artery 30 day mortality following coronary artery bypass grafts in 25 English hospitals bypass grafts in 25 English hospitals

(B) A ‘‘funnel’’ plot of observed

rate against number of

operations (3SD from the target).

Warning!

Alarming!


(A) A ‘‘forest’’ plot showing

95% confidence intervals

compared with the ‘‘target’’

overall average rate.

Emergency (within 30 days) readmission rates Emergency (within 30 days) readmission rates following dischargefollowing discharge





Emergency (within 30 days) readmission rates Emergency (within 30 days) readmission rates following dischargefollowing discharge




Action:

Do not use the indicator…? Improve risk stratification…?

Analysis by clustering…?


ConclusionConclusion

Managed to develop a good quality “first version” hospital indictor set.

Further field testing required.

Next to external accountability / transparency indicators should stimulate

‘internal’ quality improvement.

Development of plan for data-analysis and presentation: no simple

calculation of numbers or percentages.

Last but not least: treat indicators as indicators…! Additional quality

information essential to understand performance (e.g. quality reports).

Date post:	12-Jan-2016
Category:	Documents
Upload:	alicia-elliott
View:	230 times
Download:	7 times

Johan de Koning, Bratislava, 2005 Hospital quality indicators Johan de Koning, MPH, PhD Senior...

Documents