Quality of care from the patient’s perspective

Background and introduction of the CAHPS® questionnaires into

the Dutch health and social insurance system

• Diana Delnoij (NIVEL)• Herman Sixma (NIVEL)

Structure of the presentation

Introduction + developments in society Developments in QoC research;

perspectives + ‘state-of-the-art’ Past, present and future of the CAHPS-

approach Concluding remarks + discussion

Developments macro level

technological developments rising expectations + expenditures

in the health care sector health care reforms tendency toward democratization

Developments meso level

professionalization of health care

services rise of organizational structures burocratization of services development of category-specific

patient organizations

Developments micro level

changes in doctor-patient

relationshipconsequences ‘ageing society’changes the family structures the ‘professional’ patient

Developments in society: a summary

Macro level :

- from ‘leadership’ to ‘stewardship’

  Meso level :

- from ‘supply’ to ‘demand centered’

  Micro level :

- from ‘patient’ to ‘active participant’

insurerinsurerinsurer

consumerconsume

rprovider

insurance market

purchasing market

provider market

The new health care ‘market’

Part 2 :

Quality of care (QoC) from the patient’s perspective; history and developments

“ The key to successful doctor-patient partnerships is recognize that patients are experts too ….. The new emphasis is on shared information, shared evaluation, shared decision making, and shared responsibilities ..…”

(Angela Coulter, BMJ 1999; 319: 719-720)

Different QoC perspectives

health insurers > cost efficiency providers > guidelines + protocols patients > satisfaction + QoC ratings government > legal framework

Four generations QoC research

Patient satisfaction studies (see Linder-

Pelz, 1982 a.o.) Service Quality models (Parasuraman et

al, 1985, 1988) Extended SERVQUAL models (CAHPS,

QUOTE, Picker Instruments a.o.) Future: Performance indicator models

 

Main characteristics ‘state-of-the-art’ QoC questionnaires

patients involvementqualitative + quantitative methodsshift from ‘satisfaction’ to ‘reports’ QoC = multi-dimensional conceptapplicable in QA and QI studies

“ Questions asking for reports tend to reflect better the quality of care and ar more interpretable and actionable for quality improvement purposes than ratings of satisfaction or excellence ..…”

(Paul Cleary & Susan Edgman-Levitan, JAMA 1997, 278 (19): 1608-12)

Examples of some ‘state-of-the-art’ QoC questionnaires

Responsiveness measures WHO Instruments Picker-Europe (UK)Family of QUOTE instruments (NL)CAHPS Instruments (USA)

What is CAHPS?Consumer Assessment of Health Plan Survey

Questionnaires containing items on patients’:experiences with health care providersexperiences with their insurance companygeneral rating of health care and health

plan

Commissioned by: Agency for Healthcare Research and Quality (AHQR).

Developed by: Harvard Medical School, the RAND corporation, and the American Institutes for Research

Introduction of CAHPS in the Netherlands

• Initiative: Agis (regional public health insurer)

• Translation and validation of three questionnaires:

1. CAHPS Adult Commercial Questionnaire

2. Hospital CAHPS

3. Diabetes questionnaire

CAHPS instruments (example 1)Please answer the questions in this survey about this stay at [FACILITYNAME]. Do not include any other hospital stay in your answers.

 YOUR CARE FROM NURSES

1. During this hospital stay, how often did nurses treat you with courtesy and respect?1 Never2 Sometimes 3 Usually4 Always

 19. Before giving you the medicine, did hospital staff

describe possible side effects in a way you could understand?1 Yes2 No

CAHPS instruments (example 2)

5. Using any number from 0 to 10 where 0 is the worst possible care and 10 is the best possible care, what number would you give the care you got from all the nurses who treated you?

0 0 Worst possible nursing care 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9

10 10 Best possible nursing care

Translating the CAHPS Adult Commercial Questionnaire

Translation: forward - backward by 2 x 2 translators

Tailored to the Dutch health care system:Absence of nurse practitioners or physician

assistantsdistinction between ER in hospital and out-of-

hours services of GPs

Adaptation to social-cultural values in NL:Question about educational system Use of ethnicity instead of race

Fielding the CAHPS Adult Commercial Questionnaire

Representative sample of 1,000 Agis clients who were insured 6 months prior to sample selection

In November 2003, 977 insured received a mailed questionnaire; 545 responded (56%)

Respondents are significantly older than non-respondents

Respondents are also older than the general Dutch population and in worse health

Validating the CAHPS Adult Commercial Questionnaire

1. Check consistency of the data o.k.

2. Check frequencies and missing values o.k.

3. Check factor structure and reliability o.k.

4. Compare results with other Dutch studies and with the American National CAHPS Benchmarking Database 2003:

This presentation

% of respondents experiencing problems:

0%

20%

40%

60%

80%

100%

Finding apersonaldoctor or

nurse

Getting areferral for a

specialist

Getting careor treatmentyou needed

Waiting forconsent from

theinsurancecompany

NL

USA

Comparison of Dutch data (NL) with American benchmark data (USA): access to care

% of respondents with positive experiences:

0%

20%

40%

60%

80%

100%

staffcourtesy

helpfulstaff

doctorsspendenough

time

doctorslisten

carefully

doctorsexplain

things well

NL

USA

Comparison of Dutch data (NL) with American benchmark data (USA): patient-centered care

Ratings of providers and health plan

0%

20%

40%

60%

80%

100%

NL USA NL USA NL USA NL USA

personaldoctor

specialist health care health plan

9 or 10

7 or 8

0 to 6

Comparison of Dutch data (NL) with American benchmark data (USA): general rating

Translation into consumer information: % consumers who always experience that:

0,00% 20,00% 40,00% 60,00% 80,00% 100,00%

doctors spendenough time

doctors listencarefully

doctors explainthings well

national Agis

Conclusion

Dutch respondents are able to fill out the questionnaire

Results are internally consistent and reliable

Patients’ evaluation of the Dutch and American primary process is strikingly comparable

More research is needed on the external validity in the Dutch context

Major uncertainties

Will insurers really engage in strategic purchasing?If so, is CAHPS-information useful for

them?

Will consumers choose rationally between different insurers?If so, is CAHPS-information useful for

them?

Looking at the future:

results of a national study on quality of home care from the clients perspective;

MDB – model (home care NL)

Analysis of environmental factors that cannot be influenced by a home care institution

Identification of points that can be used to improve efficiency and quality by analysing the structure and nature of the operation

of the best-practice institutions.

Assessment of the quality of care and services by clients

Assessment of the quality of work by staff

Assess efficiency by comparing expenditure

and production

Building Block 1: Efficiency

Building Block 2: Quality of care and services

Assessment of performance by interested parties

Building Block 4:Social evaluation

Building Block 3:Quality of work

Identification of best-practice institutions

Conceptual model for client survey

Communication

Artificial aids

Professional competence

Continuity

Accessibility

Courtesy

Process

Structure

Quality of home care

Autonomy

Data collection

Questionnaires Number %

Gross random survey 56,094 100.0

- could not be delivered 2,675 4.8

- do not receive home care 500 0.9

Net random survey 52,919 100.0

Gross nr. of respondents 25,973 49.1

- declined to participate 920 1.7

- nothing filled in 494 0.9

Net nr. of respondents 24,579 46.5

Process quality on 1 -10 scale

0

0 .5

1

1.5

2

2 .5

3

3 .5

4

4 .5

5

A verag e = 8 .24 Stan d ard d evia tio n = 1 .3

Num

ber o

f Clie

nts

(in th

ousa

nds)

1 -1.5 1.5 -2 2 -2 .5 2 .5 -3 3 -3 .5 3 .5 -4 4 -4 .5 4 .5 -5 5 -5 .5 5 .5 -6 6 -6 .5 6 .5 -7 7 -7.5 7.5 -8 8 -8 .5 8 .5 -9 9 -9 .5 9 .5 -10

N = 24 .4 4 3

Structure quality, 1 – 10 scale

A verag e = 6 .9 7 Stand ard d evia tio n = 1.6 3

1 -1.5 1.5 -2 2 -2 .5 2 .5 -3 3 -3 .5 3 .5 -4 4 -4 .5 4 .5 -5 5 -5 .5 5 .5 -6 6 -6 .5 6 .5 -7 7 -7.5 7.5 -8 8 -8 .5 8 .5 -9 9 -9 .5 9 .5 -100

0 .5

1

1.5

2

2 .5

3

3 .5

4

4 .5

5

Num

ber o

f Clie

nts

(in th

ousa

nds)

N = 24 .5 5 0

QoC ratings 106 home care organizations

6 .5

7.0

7.5

8 .0

8 .5

9 .0

1 6 11 16 21 2 6 31 3 6 41 4 6 51 5 6 61 6 6 71 76 81 8 6 91 9 6 101 10 6 111

R anked b y C lien t Eva lu atio n

A verag e(7.7 3 )

N = 10 6

Overall QoC scores, broken down by size of organization

1

2

3

0

2

4

6

8

10

12

14

N = 10 6

C lien t evalu atio n categ o ry

C ateg o ry o f Size

Num

ber o

f Ins

titut

ions

1 – ab ove-averag e

2 – a ro u n d averag e (7.7 3 )

3 – b elo w average

Main conclusions (1)

Patient views on QoC should be an important topic

for policy makers + health care providers + health

insurers + patients themselves

Standardization and internationalization of

QoC measuring instruments, based on the

CAHPS approach offers new possibilities for QA,

QI and to derive information to facilitate patient choice

Main conclusions (2) Follow-up research on validity, reliability,

‘sensitivity to change’ and discriminating power

of new measuring instruments (such as the

CAHPS-NL instruments) is still necessary

Mesuring QoC from the patients’ perspective

should be a part of an integrated approach (TQM,

MDB, BSC)

Contact details

Herman J. Sixma, PhDNIVELP.O. Box 15683500 BN UtrechtThe Netherlands

Tel.: ++ 31 30 2 729 710Fax.: ++ 31 30 2 729 729E-mail: h.sixma@nivel.nlwww.nivel.nl