How reliable are value judgements about health inequality aversion? Results of two experimental...

Shehzad Ali

Centre for Health Economics

University of York

Value judgements about health

inequality aversion: results of

two experimental studies

1

Research Team: Richard Cookson (York); Shehzad Ali (York);

Miqdad Asaria (York); Aki Tsuchiya (Sheffield)

With thanks to: Ruth Helstrip, James Koh, Matthew Robson,

Paul Toner and participants of the piloting session

Conference papers:

Ali, Cookson, Tsuchiya and Asaria (2014). Eliciting value

judgements about health inequality aversion: testing for framing

effects. Paper presented in HESG Sheffield in Jan 2014.

Cookson, Ali, Tsuchiya and Asaria (2015). Value judging, fast

and slow: an experimental study of the effects of slow thinking

interventions on expressed health inequality aversion. Paper to

be presented in HESG Leeds in Jan 2015.

Equity vs efficiency

• Value judgements are important for making

policy decisions

• Equity relates to fairness in the the

distribution of health and health care as

opposed to maximising the total sum

(efficiency)

• Potential trade-offs between efficiency and

equity

Public concern for equity, beyond “a

QALY is a QALY”

1. Severity of illness

2. Children vs. adults

3. Socioeconomic inequality in health

• Evidence suggests public concern for all three

issues

Dolan, P, Shaw, R, Tsuchiya, A and Williams, A. (2005). QALY

maximisation and people's preferences: a methodological

review of the literature Health Economics 14(2): 197-208.

Empirical social choice

• In recent years, the inter-

disciplinary field of

“empirical social choice”

has emerged to investigate

social norms about fairness

(Gaertner and Schokkaert,

2012; Konow, 2003)

• Not economic lab

experiments but

psychological experiments

to investigate people’s

views about fairness

Quantifying equity concerns

• Typically use questionnaire methods to quantify

the magnitude of inequality aversion in different

contexts (Shaw et al 2001, Abásolo, Tsuchiya,

2004, 2013; Dolan, Tsuchiya, 2011)

• The concern for inequality can be explicitly

incorporated in decision analysis using

methods such as Distributional Cost-

effectiveness Analysis (Asaria et al 2014) or

other approaches (Johri and Norheim 2012) in

the literature

Potential cognitive biases

• Estimates of inequality aversion are likely to be

influenced by cognitive biases

• We conducted two experimental studies to

assess reliability of value judgements about

health inequality aversion obtained from a

standard questionnaire instrument

Experimental Study 1

Reliability of a standard questionnaire

instrument for eliciting value judgements

Questionnaires

• A standard questionnaire is presented in four

formats

Large gains (in years): Individual level TO1

Small gains (in hours): Individual level TO2

Small gains ABSTRACT (in years): population level TO3

Small gains CONCRETE (in years): population level TO4

0 10 20 30 40 50 60 70 80

Richest Fifth

2nd Richest

Middle Fifth

2nd Poorest

Poorest Fifth

Expected Years of Life in Full Health

England and Wales

Quality adjusted life expectancy at birth

74

62 12

Source: Asaria, M, Griffin, S, Cookson, R, Whyte, S, Tappenden, P. (2012). Cost-equality analysis of health care programmes – a methodological case study of the UK Bowel Cancer Screening Programme. Paper presented to Health Economists Study Group in Exeter, January 2013.

Expected Years of Life in Full Health

England and Wales

Shaw, Dolan, Tsuchiya, Williams, Smith and Burrows, 2001.

"Development of a questionnaire to elicit public preferences

regarding health inequalities," Working Papers 040cheop,

Centre for Health Economics, University of York

Final choice

Programme A “dominates” Programme B: more health for the rich and same health for the poor.

... But Programme B reduces health inequality.

Five views about health inequality

1. Pro-rich (AAAAA)

2. Health maximisers (EAAAA)

3. Weighted prioritarians (BXXXA)

4. Maximin (BBBBA)

5. Strict egalitarians (BBBBB)

TO2 Small Individual Questions

TO3 Small Population Questions: Abstract

Scenario for small population-level

“concrete” question

19

-0.001

0.000

0.001

0.002

0.003

0.004

0.005

Most Deprived IMD 4 IMD 3 IMD 2 Least DeprivedIncr

emen

tal P

er

Pe

rso

n Q

ALY

s C

om

par

ed t

o N

o In

terv

enti

on

targeted universal

Bowel Cancer screening: Impact of Redesign on Health

TO4 Small Population Questions: Concrete

(1) Small versus

unrealistically large health

inequality reductions

(2) Population-level versus

individual-level descriptions

of health inequality

reductions

(3) Concrete versus

abstract intervention

scenarios

(4) Online versus face-to-

face mode of administration

(5) “Academic versus non-

academic” background

Large Individual

Questions TO1

Small Individual

Questions TO2

Small Population

Questions TO3

(abstract)

Small Population

Questions TO4

(concrete)

(1) Small versus






reductions

(3) Concrete versus


scenarios





Large Individual

Questions TO1

Small Individual

Questions TO2

Small Population

Questions TO3

(abstract)

Small Population

Questions TO4

(concrete)

(1) Small versus






reductions

(3) Concrete versus


scenarios





Large Individual

Questions TO1

Small Individual

Questions TO2

Small Population

Questions TO3

(abstract)

Small Population

Questions TO4

(concrete)

(1) Small versus






reductions

(3) Concrete versus


scenarios





Large Individual

Questions TO1

Small Individual

Questions TO2

Small Population

Questions

(abstract) TO3

Small Population

Questions TO4

(concrete)

(1) Small versus






reductions

(3) Concrete versus


scenarios





Large Individual

Questions TO1

Small Individual

Questions TO2

Small Population

Questions TO3

(abstract)

Small Population

Questions TO4

(concrete)

Pro-rich >7

Health maximiser 7

Weighted prioritarian 6.5

Weighted prioritarian 6






Maximin 3

Strict egalitarian <3

AAAAA

=AAAA

BAAAA

B=AAA

BBAAA

BB=AA

BBBAA

BBB=A

BBBBA

BBBB=

BBBBB

Non-

Egalitarian

Strong

Egalitarian

{

{

Response classification

B

A

A

A

A

Recruitment and Administration

Administration:

5-hour Saturday session in York city centre

•facilitated discussions in groups of five or six;

•individual completion of the questionnaire

Recruitment of face-to-face sample:

Advertisements in a monthly Your Local Link and 810 leaflets distributed

door-to-door in 10 of the most deprived streets in York.

Payment = £70

On-Line

Recruitment: 1) Website of the Centre for Health Economics at the University of York,

2) Social media,

3)York Local Link magazine

4) Jiscmail mailing list for health economists,

Payment = £0 Respondents divided into Non-academic (n = 83) & Academic (n = 46)

(n=129)

Face-to-face (n=52)

Results

Table 1: Descriptive statistics of the discussion group and on-line survey respondents

Discussion group

(N = 52)

Online group:

non-academic

(N = 83)

Online group:

academic

(N = 46)

Baseline Statistic n Statistic n Statistic n

Male (%) 40.4% 21 32.5% 27 32.6% 15

Age (%)

Under 18 0.0% 0 0.0% 0 2.2% 1

18-34 21.2% 11 18.1% 15 39.1% 18

35-49 13.5% 7 15.7% 13 39.1% 18

50-64 38.5% 20 42.2% 35 17.4% 8

65+ 26.9% 14 24.1% 20 2.2% 1

Mean deprivation quintile (mean)

(1 = most deprived; 5 = most affluent) 3.71 51 3.17 83 3.39 33

Social attitude statements* (mean)

(1= strongly agree; 5= strongly disagree)

The creation of the welfare state is one

of Britain's proudest achievements. 1.42 52 1.36 82 1.37 46

Government should redistribute income

from the better-off to those who are less

well off.

2.86 51 2.05 82 2.07 46

*1 suggests most egalitarian and 5 suggests most non-egalitarian

Descriptive statistics of the discussion group and on-line survey respondents

Pro-rich (AAAAA);

Health maximiser (=AAAA);

Weighted prioritarian (BXXXA);

Maximiner (BBBB=);

Strict egalitarian (BBBBB)

** The vertical line indicates the

location of the median

respondent

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

O3: Small-population…

O2: Small-averagequestion

O1: Large-averagequestion

Percentage of respondents

Online mode: academic

Pro-rich

Health maximiser

WeightedprioritarianMaximiner

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

D4: Concretequestion

D3: Small-population…

D2: Small-averagequestion

D1: Large-averagequestion


Discussion mode

Pro-rich

Health maximiser


Strict egalitarian

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

O3: Small-population…

O2: Small-averagequestion

O1: Large-averagequestion


Online mode: non-academic

Pro-rich

Health maximiser


Distribution of responses across principles of health justice

Results (cont.)

Table: Statistical tests (D = Discussion group; O = Online group)

First hypothesis Difference in % that are

strong egalitarian*

Difference in % that are

non-egalitarians*

Wilcoxon signed-rank equality test

on matched dataǁ

Large-average vs small-population question (D1 – D3) -0.8% (p = 0.937) -0.2% (p = 0.937) z = 0.458; p = 0.647

Large-average vs small-average question (D1 – D2) +9.2% (p = 0.377) -5.4% (p = 0.225) z = 1.964; p = 0.050

Large-average vs small-population question (O1 – O3) +2.6% (p =0.765) -3.8% (p =0.281) z = 0.979; p = 0.328

Large-average vs small-average question (O1 – O2) +2.46% (p =0.773) -5.1% (p =0.167) z = 1.915; p = 0.056

Second hypothesis Difference in % that are

strong egalitarian*


non-egalitarians*

Wilcoxon signed-rank equality test

on matched dataǁ

Small-average vs small-population question (D2 – D3) -9.9% (p = 0.350) +5.2% (p = 0.271) z = -1.313; p = 0.189

Small-average vs small-population question (O2 – O3) +0.1% (p =0.991) +1.4% (p =0.767) z = 0.311; p = 0.756

Third hypothesis Difference in % that are

strong egalitarian*


non-egalitarians*

Wilcoxon rank-sum equality test

on unmatched dataǂ

Small-population vs concrete question (D3 – D4) +18.6% (p = 0.080) -11.6% (p = 0.049) z = 3.244; p = 0.001

Fourth hypothesis Difference in % that are

strong egalitarian*


non-egalitarians*


on unmatched dataǂ

Large-average question (D1 – O1) +17.3% (p =0.059) -0.6% (p = 0.838) z =1.338; p = 0.181

Small-average question (D2 – O2) +10.6% (p =0.292) -0.3% (p = 0.954) z = 0.987; p = 0.324

Small-population question (D3 – O3) +20.7% (p =0.036) -4.1% (p = 0.329) z = 2.022; p = 0.043

Fifth hypothesis Difference in % that are

strong egalitarian*


non-egalitarians*


on unmatched dataǂ

Large-average question (Na1 – A1) +23.8% (p = 0.012) -6.6% (p = 0.114) z = 2.930; p = 0.003

Small-average question (Na2 – A2) +29.2% (p = 0.004) -6.9% (p = 0.283) z = 2.506; p = 0.012

Small-population question (Na3 – A3) +26.8% (p = 0.006) -1.05% (p = 0.838) z = 2.489; p = 0.013

Results (cont.)

* Strong egalitarians = maximiner or strict egalitarian; Non-egalitarians = pro-rich or health maximisers.

Conclusion (study 1)

• No evidence of effects of small versus unrealistically

large health inequality reduction scenarios (1) or

population-level descriptions (2)

• Evidence of an anti-egalitarian concrete scenario

effect (3)

• Weak evidence of an anti-egalitarian online mode of

administration effect (4): “socially desirable” face-to-

face responses?

• Clear evidence of an anti-egalitarian academic

sample selection effect (5): academics may be more

comfortable with cognitively demanding tasks?

Conclusion (cont.)

• Reassuring that no clearly significant effects of using

small rather than unrealistically large, or using

population-level rather than individual-level

presentations health gains

• The other effects are potential cause for concern

• Weakness:

– Gain egalitarianism over outcome egalitarianism

(Tsuchiya, Dolan, 2009). However, identical framing with a

fixed ratio of gains has been maintained

– Order of questions was not randomised

Experimental Study 2

Effects of slow thinking interventions on

expressed health inequality aversion

Thinking, fast and slow

• Questionnaire methods are

vulnerable to “fast thinking”

cognitive biases

• Kahneman defines two

systems:

– System 1: Fast, automatic,

emotional, subconscious

– System 2: Slow, effortful,

calculating, conscious

• Respondents may use simple

“like-dislike” approach rather

than carefully weighing the

competing values

“Slow thinking” interventions

• Video animation

– exposing subjects to rival points of view

• Interactive computer-based version of the

questionnaire

– Providing feedback on implied trade-offs between

health inequality and sum total health

Study design

Paper questionnaire

Paper group

Video animation

Paper questionnaire

Interactive questionnaire

Video animation

Interactive questionnaire

Interactive group

Years

fo

r th

e r

ich

Years for the poor

Indifference curves representing

different views on equity

With thanks to Matthew Robson

Video animation

The interactive slider

Recruitment and Administration

Administration:

5-hour Saturday session in Heslington East

Campus

Individual completion of the questionnaire

Recruitment:

N = 60 (two sessions with 30 participants each)

Advertisements in a monthly Your Local Link.

Payment = £50

Results Sample characteristics

Paper group

(N = 29)

Interactive group

(N = 30)

Characteristic Statistic n Statistic n

Male (%) 38% 11 47% 14

Age (%)

18-34 31% 9 20% 6

35-49 7% 2 27% 8

50-64 38% 11 20% 6

65+ 24% 7 33% 10

Deprivation quintile group (mean)

(1 = most deprived; 5 = most affluent) 3.41 29 3.7 30

Social attitude statements(1) (mean)

(1= strongly agree; 5= strongly disagree)

The creation of the welfare state is one of Britain's

proudest achievements. 1.79 29 1.77 30

Government should redistribute income from the

better-off to those who are less well off. 3.03 29 3.10 30

Note: (1) 1 suggests most egalitarian and 5 suggests least egalitarian

Results (cont.)

Figure 1: Inferred principles of health justice by question and sample design* ** ***

* Complete case analysis, n = 30 in the interactive group, n = 29 in the paper group

** See table 2 for the response classification system corresponding to the five principles of health justice

*** The vertical line indicates the location of the median respondent

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Post-video interactive

Pre-video interactive

Post-video paper

Pre-video paper


Pro-rich

Health maximiser

Weighted prioritarian

Maximin

Strict egalitarian

Figure 2: Cumulative distribution of responses (1), (2)

Notes (1)

Complete case analysis, n = 30 in the slider group and n=29 in the paper group (2)

The trade off point represents the point of indifference in terms of the gain to the poorest fifth in programme

A, as shown in the response classification system in table 2.

Wilcoxon rank sum test[p = 0.000]

0.0

00.2

00.4

00.6

00.8

01.0

0

Cu

mu

lativ

e r

esp

onse

pro

po

rtio

n

>8 8 7.5 7 6.5 6 5.5 5 4.5 4 3.5 3 2.5 2 <2

Trade off point (number of years)

group = paper group = interactive

Wilcoxon rank sum test[p = 0.004]

0.0

00.2

00.4

00.6

00.8

01.0

0

Cu

mu

lativ

e r

esp

onse

pro

po

rtio

n

>8 8 7.5 7 6.5 6 5.5 5 4.5 4 3.5 3 2.5 2 <2


group = paper group = interactive

Wilcoxon sign rank test[p = 0.000]

0.0

00.2

00.4

00.6

00.8

01.0

0

Cu

mu

lativ

e r

esp

onse

pro

po

rtio

n

>8 8 7.5 7 6.5 6 5.5 5 4.5 4 3.5 3 2.5 2 <2


Pre-video, paper Post-video, paper

Wilcoxon sign rank test[p = 0.945]

0.0

00.2

00.4

00.6

00.8

01.0

0

Cu

mu

lativ

e r

esp

onse

pro

po

rtio

n

>8 8 7.5 7 6.5 6 5.5 5 4.5 4 3.5 3 2.5 2 <2


Pre-video, interactive Post-video, interactive

Pre-video vs. post-video (interactive) Paper vs. interactive (post-video)

Pre-video vs. post-video (paper) Paper vs. interactive (pre-video)

Results (cont.)

Figure: Paper group responses, pre- and post-video

>8

87.5

76.5

65.5

54.5

43.5

32.5

2<

2

Po

st-

vid

eo

sw

itch

ing p

oin

t

>8 8 7.5 7 6.5 6 5.5 5 4.5 4 3.5 3 2.5 2 <2

Pre-video switching point

Results (cont.)

Table 3: Random effects ordered probit models of the five ordered response categories

Variables Without

respondent

covariates

With

respondent

covariates

Interactive (𝛽1) -2.32*** -2.18***

(0.417) (0.407)

Post-video (𝛽2) -1.49*** -1.50***

(0.344) (0.343)

Interactive*post-video (𝛽3) 1.70*** 1.70***

(0.451) (0.451)

Joint test of (𝛽2 + 𝛽3):

Video effect on interactive

0.21

(0.28)

0.20

(0.28)

Joint test of (𝛽1 + 𝛽3):

Interactive post-video vs. paper post-video

-0.63*

(0.34)

-0.49

(0.34)

Intercept 1 (strict egalitarian) -3.39***

(0.459)

-4.09***

(0.766)

Intercept 2 (maximin) -2.70*** -3.43***

(0.399) 0.727)

Intercept 3 (weighted prioritarian) -0.92*** -1.67**

(0.283) (0.649)

Intercept 4 (health maximiser) -0.23 -0.96

(0.264) (0.631)

Observations 118 118

Number of individuals 59 59

Notes:

(1) A positive coefficient indicates a difference in a more egalitarian direction

(2) Standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1

(3) The respondent covariates were four age groups, sex and five deprivation quintile groups; coefficients on

the covariates are suppressed as none were significant.

Results (cont.)

Notes:

(1) A positive coefficient indicates a difference in a more egalitarian direction

(2) Standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1

(3) The respondent covariates were four age groups, sex and five deprivation quintile groups; coefficients on

the covariates are suppressed as none were significant.

-.4

-.3

-.2

-.1

0.1

.2.3

.4

Ch

an

ge

in p

redic

ted p

rob

ab

ility

Pro-rich Health max Trader Maximin Strict Egal

Inferred principles of health justice

(post-video paper minus pre-video paper)

Change in predicted probabilities: ordered probit model

Figure: Marginal effects on probabilities, from ordered probit model

Results (cont.)

Conclusion (study 2)

• Both “slow thinking” interventions produced

significantly less egalitarian responses

• Paper group (before vs after): strong egalitarian

response reduced from 75% to 21%

– Due to strong egalitarians switching to weighted

prioritarians

• Interactive vs paper: strong egalitarian responses

were 23% vs 75%

• Fast thinking effect: treating equality as a “sacred

value”

• Potential “Social desirability bias”?

Conclusion (overall)

• Standard methods of eliciting value

judgements about inequality aversion are

vulnerable to cognitive biases

• “Slow thinking” interventions may reduce pro-

egalitarian bias

• Expressed inequality aversion is vulnerable to

scenario effect, sample selection and, to some

extent, on mode of administration

Thank you.

Date post:	16-Jul-2015
Category:	Health & Medicine
Upload:	cheweb1
View:	59 times
Download:	1 times

How reliable are value judgements about health inequality aversion? Results of two experimental...

Health & Medicine