Saturation effect on over-surveyed people versus willingness to participate in online

panels: experimental results

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

Prof. Silvia BiffignandiDr. Roberto ArtazUniversity of Bergamo (Italy)

Panel system PAADEL

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

PAADEL (Agro-food and Demographic Panel in Lombardy) Project & Task: building up two online panels • businesses panel (PAADEL producers agro-food sector firms) • households panel (PAADEL consumers)

Characteristics: representative panels

Pilot – experiment: panel with whole sub-population (PAADEL BgPA)

Theoretical approach: the research strand based on the Norm of Reciprocity (Gouldner, 1960), framework that enables to explain the effectiveness of inducements.

…in offline panels and in those online completing a given wave seems to be an indicator for responding to next wave (Presser,

1989, Göritz and Wolff, 2007; Göritz, Wolff and Goldstein, 2008).

…role of incentive in motivating survey participation (Göritz, 2006; 2008; Bandilla and Haas, 2009).

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

People becoming over-surveyed by the too many invitations received and so turn less interested in response (Jarret, 2012)

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

Hypothesis tested:

participation in previous wave sets a ceiling for the response in the next wave.

Findings: Hypothesis not supported in the last wave of a 4 waves experiment

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

3-wave experiment conducted online in September, October and November 2011

and a 4th Wave in November 2012.

E-mail invitations to a sample of 502 individuals at first (500 at last)

(members of local governments in province of Bergamo since in Italy publishing officials’ online e-mail address and C.V. is required by law).

Mean age of the individuals: 50 years, and 43,02% (n = 216) were female.

Members of the sample randomly assigned to one of two conditions:control condition (n = 250 at the start – 249 at the end) and experimental condition (n = 252 at the start – 251 at the end)

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

TREND OF THE RESPONSE RATE

TABLE 1 Units in

mailing

list

Units in

experiment

al group

Units

in

control

group

Total

amount of

respondents

Respondents

in

experimental

group

Respondents

in control

group

Response

rate

aggregate

Response

rate in

experimental

group

Response

rate in

control group

Wave 1

14.09.2011

Reminder

21.09.2011

502

=286M

+216F

252

=145M

+107F

250

=141M

+109F

154

=95M

+59F

70

=41M

+29F

84

=54M

+30F

30,67%

=33,21%M

+27,31%F

27,72%

=28,27%M

+27,10%F

33,60%

=38,29%M

+27,52%F

Wave 2

28.09.2011

Reminder

01.10.2011

501*

=286M

+215F

251

=145M

+106F

250

=141M

+109F

95

=48M

+47F

46

=21M

+25F

49

=27M

+22F

18,96%

=16,78%M

+21,86%F

18,32%

=14,48%M

+23,58%F

19,60%

=19,14%M

20,18%F

Wave 3

12.10.2011

Reminder

19.10.2011

500**

=285M

+215F

251

=145M

+106F

249

=140M

+109F

84

=42M

+42F

39

=20M

+19F

45

=22M

+23F

16,80%

=14,73%M

+19,53%F

15,53%

=13,79%M

+17,92%F

18,07%

=15,71%M

+21,10%F

Wave 4

03.11.2012

Reminder

10.11.2012

500

=285M

+215F

251

=145M

+106F

249

=140M

+109F

97

=54M

+43F

43

=24M

+19F

54

=30M

+24F

19,40%

=18,94%

+20%

17,13%

=16,55%

+17,92%

21,68%

=21,42%

+22,01%

* One subject (F) refused to take part in the wave 2 expressly and asked to be deleted from the mailing list.

** One subject (M) refused to take part in the wave 3 expressly and asked to be deleted from the mailing list after he received the reminder of the

wave 2. Another subject (M) asked to be deleted from the mailing list after he received the invitation to wave 3 but didn’t refuse to take part in it

expressly.

Label: M = male F = female

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

Response rates: distributions by age/gender classes

Wave1

Wave2

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

Wave3

Wave 4

ESRA 5th Conference of the European Survey Research Association

Ljubljana, Slovenia, July, 15–19, 2013

Results

saturation effect seems to be reduced by the long rest and more in control group, where an incentive wasn’t provided, than in the experimental group benefited by an incentive

(fourth wave shows in control group a response rate of 21,68% even better than W2 and in experimental group a response rate of 17,13% better than W3)

findings also point out that in a context of over-surveyed people left in peace for a long moment, inner motivations are more effective in boosting response rate than external inducements given in exchange for participation