1

Non-Response Bias Analyses of the Survey of Workplace Violence Prevention

Andrew Kato, Kathy Downey, William McCarthy, and Samantha Cruz

U.S. Bureau of Labor Statistics

The opinions expressed here are those of the authors and do not represent official policy of the Bureau of Labor Statistics.

2

Survey of Workplace Violence Prevention

Special 2005 study conducted for NIOSH (National Institute for Occupational Safety and Health)

Workplace Violence Prevention (WVP) Prevalence of security features, The risks facing employees, Employer policies and training, and Related topics associated with maintaining a

safe work environment

3

WVP: Sampling Sample taken from respondents to 2003

SOII (Survey of Occupational Injuries and Illnesses) SOII - Private industry, State and local

government public sector units from the fourth Quarter 2003

Longitudinal Database (LDB) and mining and railroad establishments

Total of 39,998 units Randomly selected units proportional to size and

oversampled within specific industries Used respondents so have prior relationship

4

WVP: Methodology Hardcopy was 12 pages, envelope and

insert; also available to non-respondents in Word via e-mail

Voluntary Protocol

Initial mailing to SOII respondent (Sept 05), Follow-up mailing to non-respondents, Address corrections for post office returns, and Telephone follow-ups to non-respondents Close-out June 06

Final response rate was 61%

5

Purpose of Non-Response Analyses

OMB requirement Examine potential bias due to non-

response since predicted response rate might be low

6

Predominant Approaches to Conducting Non-Response (Olson, 2006)

Comparing characteristics with a benchmark survey

Comparing frame information between respondents and non-respondents*

Simulating statistics based on restricted protocol (“level of effort” analyses)*

Mounting experiments to produce varying response rates across groups

7

Data Available for Non-Response Analyses

2003 SOII frame data Size class

1: 1-10 2: 11-49 3: 50-249 4: 250-999 5: 1000+

Sector (industry) – combined private and public

2003 SOII Rate (per 10,000 FTE hours) of job transfer or

restriction Rate (per 10,000 FTE hours) of days-away-from-

work cases

8

Data Analyses

Compare non-respondents and respondents on 2003 frame data and survey responses

Level of effort analyses: response propensity models

9

Comparing WPV Respondents and Non-Respondents: Size Class, by % NR

*p<.0001 for overall

5149

56.1

43.9

60.2

39.9

58.3

41.8

55.3

44.7

0

10

20

30

40

50

60

70

1 2 3 4 5

Respondents Non-Respondents

10

Comparing WPV Respondents and Non-Respondents: Size Class, by Average Days Away from Work Rate

*p<.003 for class 1, p<.0001 for class 3

0.4

1

1.2

1.4

1.9

2.1

1.71.8

1.5 1.5

0

0.5

1

1.5

2

2.5

1 2 3 4 5

Respondents Non-Respondents

11

Comparing WPV Respondents and Non-Respondents: Size Class, by Average Job Transfer or Restriction Rate

*p<.0001 for classes 3, 4, and 5

0.2

0.3

0.50.5

1.4

1.2

1.8

1.3

1.5

1.1

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

1 2 3 4 5

Respondents Non-Respondents

12

Comparing WPV Respondents and Non-Respondents: Selected Sectors, by % NR

*p<.0001 for selected sectors above

36

64

38.6

61.4

34.9

65.1

30.9

69.1

37.9

62.1

39.5

60.5

0

10

20

30

40

50

60

70

Agric Constr Healthcare

Manuf PubAdmin

Educ

Respondents Non-Respondents

13

Comparing WPV Respondents and Non-Respondents: Selected Sectors, by Average Days Away from Work Rate

*p<.02 transport, p<.004 rest

2.2

2 2

1.3

1.8

1.5

3.6

3.8

2.3

1.5

0

0.5

1

1.5

2

2.5

3

3.5

4

Health RealE Retail Trans Whole

Respondents Non-Respondents

14

Comparing WPV Respondents and Non-Respondents: Selected Sectors, by Average Job Transfer or Restriction Rate

*varying significant p values

2

1.3 1.3

0.9

1.61.5

1.8

2.1

0.6

0.4

2.4

2.1

0.8

0.4

0.8

0.5

0

0.5

1

1.5

2

2.5

Whole Retail Health Trans Info Manuf PubAd AdminS

Respondents Non-Respondents

15

Comparing WPV Respondents and Non-Respondents: Conclusions

Dealing with four projects when discussing: 2003 SOII sample and analyzed 2003 SOII

estimates of DAW and JTR (injuries/illness) WPV survey – existence of violence

prevention programs This NR project 2001 one-time Respirator Survey – used

1999 SOII sample

16

Comparing WPV Respondents and Non-Respondents: Conclusions

Non-respondents versus respondents Higher size class showed curvilinear

effect with size 3 highest Industry had some variation

possibly those with more public units

17

Comparing WPV Respondents and Non-Respondents: Conclusions

60.8

39.2

57.2

42.8

0

10

20

30

40

50

60

70

Public Private

Respondents Non-Respondents

*p<.0001

18

Comparing WPV Respondents and Non-Respondents: Conclusions – Days Away from Work, Job Transfer

Higher rates are responding more Size

Days away from work rate: highest class 3, lowest 1

Job transfer or restriction rate: highest 3, 4, 5 Industry

Days away from work rate: wholesale, retail, real estate, health care, transportation

Job transfer or restriction rate: same plus public admin, admin support, manufacturing

19

Level of Effort: Response Propensity Models for Contact and Cooperation

Predicting Contact

Predicting Cooperation

Coeff SE Coeff SE

Intercept 5.09** 0.16 5.59** 0.21

Size 1 0.07 0.17 -0.07 0.20

Size 2 -0.04 0.08 -0.04 0.09

Size 3 0.09 0.05 0.14* 0.06

Size 4 -0.01 0.05 0.01 0.06

Hours (FTE worked) 0.00 0.00 0.00 0.00

DART case rate* 0.01 0.00 0.01* 0.00

1st NR mail – Nov 05 -0.19 0.21 -0.82* 0.26

1st round calling 0.29 0.15 -0.83** 0.18

2nd NR mail – Apr 06 -5.10** 0.17 -3.88** 0.13

2nd round calling 12.46 163.1 -5.53** 0.15

*p<.05

**p<.0001

20

Level of Effort: Response Propensity Strata for Contact and Cooperation

Response Propensity Strata

Low Grp 2 Grp 3 Grp 4 High

Contact

Actual rate (n)

51% (5021)

55% (5021)

88% (5021)

99% (5021)

97% (5021)

Est. Non-contacts

51% 54% 58% 99% 99%

Est. Contacts

51% 54% 92% 99% 100%

Cooperation

Actual rate

36% (1410)

55% (2179)

98% (3883)

99% (3921)

100% (3935)

Est. Ref. 32% 55% 98% 100% 100%

Est. Coop 43% 56% 99% 99% 100%

21

Conclusions

Some differences between WPV respondents and non-respondents Size Industry SOII estimates

Respondents to WPV have more programs and more SOII incidents

Not sure what impact to WPV, possibly respondents have more programs and more risks (over-reporting?)

22

Limitations

Limitation: only as good as phone logs from vendor (contact/non-contact) Learned over surveys to not have

subcontractors so can have more detail in phone logs, manage mailings/contacts

23

Future Research

Level of effort analyses tied into data estimates How data might change for key WPV

estimates at different levels of effort (truncation)

Huge work to re-weight, though More WPV analyses – 3-digit NAICS,

like transportation