Use of intelligence by EPSO member statesDr Alex Mears

Care Quality Commission

2

The Care Quality Commission

• The Care Quality Commission is the regulator for England

• Created 1st April 2009 (just over a year old)

• Remit covers all aspects of health (NHS and privately owned) and adult social care

• Does not include medical or clinical personnel (other bodies do that)

• Registration-based model- initial registration then ongoing monitoring of compliance

• Information is delivered to field staff through statistical risk model into the Quality and Risk Profile (QRP), indicating risk level for a number of outcomes for each service/ provider

• QRP used by field staff to prioritise regulatory activity

• QRP uses information from many sources including users of services

3

The project- background

•Different organisations within EPSO undertake different roles with different objectives.

• This project questionnaire looks at how our organisations use intelligence, information and/or data as part of regulatory/ supervisory activity.

•Some organisations will not use information very much, relying on a full-coverage model (where all providers of care are inspected on a rotational basis); others will use information to select or prioritise inspection activity.

•This questionnaire aims to understand how supervisory bodies in EPSO use information, and how this is linked to their aims and goals and other factors such as how they interact with supervised organisations

4

Methods

• This questionnaire was sent out by the EPSO central secretariat to representatives of all nations active within EPSO, regardless of membership status, comprising 18 countries in total.

•Two reminders were sent out to participants, each with a fresh copy of the questionnaire attached.

• The questionnaire was developed to enable the researchers to understand more about a number of areas of regulation.

•Sections gathered data on

• Demographics

• regulatory approach

• availability of information

• analysis capability

• regulatory model

• use of information.

•Analyses run were descriptive (univariate) and cross-tabulation (bi- or multi-variate).

5

Results: structure

• Results are presented using a structured approach (this was a long questionnaire and there are lots to show)]

• Viable responses were received from 12 nations of the 18 that received it, an adjusted response rate of 67% (some countries were not included in the original mail out and were added later).

• Analyses are presented as descriptives (univariate) and crosstabulations (bi- and multi-variate) to show relationships.

• The results presented do not represent the totality of data collected, but the highlights of analyses so far.

•V. small dataset means findings are only illustrative.

6

Demographics

7

Organisation status

- Most organisations have no relationship with local government

outside govt structure but funded by govt

Part of cent govt but not within a govt dept

Part of central govt

organisation status within central gov

8

6

4

2

0

Fre

qu

en

cyorganisation status within central gov

8

Entry to and exit from the market

little or no influencehigh level of influencefull control

extent of control over entry to market

6

5

4

3

2

1

0

Fre

qu

ency

extent of control over entry to market

little or no influencesome influencehigh level of influencefull control

extent of control over exit from market

4

3

2

1

0

Fre

qu

ency

extent of control over exit from market

9

Service failure and economic factors

not consideredwithin supervision but peripheralone aspect amongst others

how far regulate economic factors

6

5

4

3

2

1

0

Fre

qu

encyhow far regulate economic factors

- Most supervisors investigate service failure (10/12)

10

Relations with supervised bodies

can be reluctantshare goals and co-operate

view of the supervised

10

8

6

4

2

0

Fre

qu

ency

view of the supervised

All respondents reported a good relationship with supervised organisations

11

Reporting

0

2

4

6

8

10

12

reports:supervised

bodies

reports:personnel

reports:patients/ users

reports: public reports:government

reports: other

12

Team members & recruitment

0

2

4

6

8

10

12

traine

d insp

ectors

med

ics

nurse

s

othe

r hea

lthca

re p

rofes

siona

ls

audit

ors

admin

perso

nnel

rese

ach/ a

cacd

emic

studen

tsot

her

0

2

4

6

8

10

12

inspectedorganisations

clinicians civil servants academics private sector other

13

Research, mergers, change trends and finance

about the sameless moneymore money

financial resource trend

6

5

4

3

2

1

0

Fre

qu

ency

financial resource trend

•Some organisations commission independent research from third parties (7/12)•Most respondents feel that changes in their work follow a direction of travel (9/12)•Half of respondents have been involved in mergers

14

Breadth of supervision

0

2

4

6

8

10

12

emer

genc

y ca

re

elec

tive

care

men

tal h

ealth

inc

depr

ivat

ion

men

tal h

ealth

no

depr

ivat

ion

prim

ary

med

ical

car

e

prim

ary

dent

al c

are

publ

ic h

ealth

child

care

mat

erna

l scr

eeni

ng

radi

atio

n

drug

and

pha

rmac

y

addi

cito

n se

rvic

es

lear

ning

dis

abili

ty

mid

wife

ry0

2

4

6

8

10

12

mili

tary

lega

l

resi

dent

ial c

are

nurs

ing

hom

es

assi

sted

livi

ngfo

r LD

med

ical

sta

ff

nurs

ing

staf

f

othe

rpr

ofes

sion

als

med

ical

tech

nolo

gy

clin

ical

educ

atio

n

med

ical

educ

atio

n

15

Powers

0

2

4

6

8

10

12po

wer

s: in

form

alac

tion

pow

ers:

for

mal

actio

n

pow

ers:

inve

stig

atio

n

pow

ers:

fin

e

pow

ers:

rest

rictio

n

pow

ers:

wih

draw

licen

ce

pow

ers:

with

draw

accr

edita

tion

pow

ers:

civ

illit

igat

ion

pow

ers:

crim

inal

litig

atio

n

16

Data available for supervision 1

unspecified databoth numeric & qualitative

numericno data

data: from licensing

5

4

3

2

1

0

Fre

qu

ency

data: from licensing

unspecified databoth numeric & qualitative

numericno data

data: from registration

5

4

3

2

1

0

Fre

qu

ency

data: from registration

unspecified databoth numeric & qualitative

qualitativeno data

data: from accreditation

6

4

2

0

Fre

qu

ency

data: from accreditation

unspecified datanumericno data

data: episode statistics

6

4

2

0

Fre

qu

ency

data: episode statistics

17

Data available for supervision 2

unspecified databoth numeric & qualitative

numericno data

data: from government/ standards etc

4

3

2

1

0

Fre

qu

ency

data: from government/ standards etc

unspecified databoth numeric & qualitative

qualitativenumericno data

data: from clinical best practice

4

3

2

1

0

Fre

qu

ency

data: from clinical best practice

unspecified databoth numeric & qualitative

qualitativenumericno data

data: patient survey

5

4

3

2

1

0

Fre

qu

ency

data: patient survey

unspecified databoth numeric & qualitative

qualitativenumericno data

data: staff survey

5

4

3

2

1

0

Fre

qu

ency

data: staff survey

18

Data available for supervision 3

unspecified databoth numeric & qualitative

qualitativenumericno data

data: other patient

5

4

3

2

1

0

Fre

qu

ency

data: other patient

unspecified databoth numeric & qualitative

qualitativenumericno data

data: other govt agencies

5

4

3

2

1

0

Fre

qu

ency

data: other govt agencies

unspecified databoth numeric & qualitative

qualitativenumericno data

data: self-report

4

3

2

1

0

Fre

qu

ency

data: self-report

unspecified databoth numeric & qualitative

qualitativenumericno data

data: directly collected

5

4

3

2

1

0

Fre

qu

en

cy

data: directly collected

19

Data available for supervision 4

unspecified databoth numeric & qualitative

qualitativenumericno data

data: feedback from supervision

5

4

3

2

1

0

Fre

qu

ency

data: feedback from supervision

20

How information is used

0

2

4

6

8

10

12

basicdescriptives

benchmark/comparative

report cards sector specific frameworks risk framework

21

How data drives inspection

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5re

gist

ratio

nst

anda

rds

accr

edita

tion

stan

dard

s

epis

ode

stat

istic

s

govt

sta

ndar

ds

clin

cial

bes

tpr

actic

e

patie

nt s

urve

y

staf

f su

rvey

othe

r pa

tient

info

othe

r go

vtag

ency

dat

a

self-

repo

rt

dire

ctly

colle

cted

supe

rvis

ion

feed

back

22

Crosstabulations

• Looked at relationships between different variables

• created categories to explore the data

• Countries: Eastern Europe, Western Europe, Nordic

• Breadth of supervision: high, low

• Powers available: many, few

• Data use: basic to complex including risk-based

• Inspection model: driven by defined period to risk based

• Risk-model: hybrid of data use and inspection model

23

Breadth of supervision activity by geographical area

NordicEastern EuropeWestern Europe

country category

4

3

2

1

0

Co

un

t

Bar Chart

high

low

breadth of supervision dichotomous

-Eastern European supervisors’ scope is relatively smaller-the Nordic scope is relatively wider-Western Europe is between

24

Powers available by geographical region

NordicEastern EuropeWestern Europe

country category

4

3

2

1

0

Co

un

t

Bar Chart

high

low

total powers dichotomous

- Western Europe has comparatively more powers- Nordic countries have less powers- Eastern European countries are between

25

Availability of data by geographical area

NordicEastern EuropeWestern Europe

15.00

10.00

5.00

0.00

Mea

n d

ata

ava

ilab

le t

ota

l

Availability of data by country category

26

Cluster chart of data available, powers and geographical region

- More data is associated with more powers-This effect is most pronounced for Eastern European countries

27

Data sources available by powers available

28

Breadth of supervision by powers available

29

Inspection frequency by most sophisticated form of analyses

- There appears to be a loose relationship between responsiveness of model and sophistication of analysis- These two variables can be combined to form a proxy measure for how risk-based a regulator is

30

Risk based model by geographical area

31

Risk based model by analyses used

- There appears to be a loose relationship between risk-based models of regulation and total use of analyses

32

Risk based model by data available

- There appears to be a loose relationship between risk-based models of regulation and availability of data

33

Risk based model by breadth of supervision

- risk-based regulation is loosely associated with a smaller regulatory scope

34

Much more data to analyse!

• I’ll stop there, as we will have no time for discussion

• There are plans to complete the analyses, look for more associations and unpack the more qualitative information

• Plan to write and submit a journal article, probably to the International Journal of Quality in Health Care

• Open to the floor: invite comments, thoughts, opinion from delegates

• Questions for discussion:

• Does what the data show make sense?

• Are there relationships that are expected?

• Are there relationships that are surprising?

• Are there discernable patterns in the data?

• Are there other questions we should be asking?

• What other analyses should we be doing?