The Effects of Raking and Cell Phone Integration on BRFSS Outcome s

The Effects of Raking and Cell Phone Integration on BRFSS Outcome s

Machell Town, M.S. Carol Pierannunzi, Ph.D.

Division of Behavioral Surveillance

Office of Surveillance, Epidemiology, and Laboratory ServicesDivision of Behavioral Surveillance

Brief Agenda Weighting procedures

Design weights Post stratification Iterative proportional fitting

Why change weighting procedures now? Cell phone Computer capacity

Impact of changes on estimation BRFSS Examples of small and large impact Changes when cell phones are incorporated

Conclusions Brief look at state level phone use data

(preliminary)

WEIGHTING PROCEDURES

Design and GeoStrata Weighting Takes into account the geographic

region/strata of the sample. Design weight uses number of adults in

household and number of phones in household for landline sample.

BRFSS landline sample is drawn using low/high density strata within each of the regions (usually smaller than states)

Stratum weight (_STRWT) = NRECSTR/ NRECSEL

Calculating the Design Weight Design Weight = _STRWT* (1/NUMPHON2) *

NUMADULT NUMPHON2= number of phones within the household NUMADULT = number of adults eligible for the survey

within the household Questions for the design weights are asked

in screening questions and in demographic sections of the survey

WEIGHTINGPost -Stratification

Data Weighting Data weights take the design weighting and

incorporate characteristics of the population and the sample

Final Weights (_FINALWT) = Design Weight * some form of data weighting In past BRFSS used post stratification In 2008 Iterative Proportional Fitting was first used In 2011 Iterative Proportional Fitting will be only method

of data weighting for BRFSS

Where We Have Been---Post Stratification

Post Stratification is based on known demographics of the population. For BRFSS Post stratification included:

· Regions within states· Race/ Ethnicity (in detailed categories)· Gender· Age (in 7 categories)

Post-stratification forces the sum of the weighted frequencies to equal the population estimates for the region or state by race, age ,and gender.

Post stratification weights are applied to the responses, allowing for estimates of how groups of non-respondents would have answered survey questions.

Post-stratification Post-stratification Adjustment Factor is

calculated for each race/ethnicity, gender, and age group combination.

_POSTSTR = Population/Design weight within the weighting class cell.

Weight Trimming Sometimes post-

stratification resulted in very small or disproportionately large weights within age/race/gender/region categories.

Weight trimming or category collapsing would be done if categories were disproportionately large or too small (< 50 responses).

WEIGHTINGIterative Proportional Fitting (Raking)

Iterative Proportional FittingGender by

race/ethnicity

Age by gender

Age by race/ethnici

ty

Renter/owner

Education level

Marital status

Detailed race/ethnicity

Regions within states

Phone source

Rather than adjusting weights to categories, IPF adjusts for each dimension separately in an iterative process.The process will continue up to 75 times, or until data converges to Census estimates.

New Variables Introduced as Controls With IPF

Education Marital status Home ownership/renter Telephone source (cell phone

or landline)

Post Stratification vs. Iterative Proportional Fitting

Post Stratificatio

n

Iterative Proportional

Fitting

Operates with less computer time

Allows for incorporation of new variables.Allows for incorporation of cell phone data.Seems to more accurately represent population data (reduces bias).

Why Incorporate IPF Now? Computer capacity has increased. Cell phones are becoming larger percentage

of the total number of calls. Noncoverage with declining response rates

makes weighting more important than ever.

Raking – Iteration 1

16

First Control Variable

Output Weight Sum of

WeightsTarget Total

Sum of Weights

Difference

% of Output

WeightsTarget % of

WeightsDifference

in %Age 18-24,Male 87122.60 95468 -8345.40 6.533 7.159 -0.626Age 18-24,Female 77180.40 90249 -13068.60 5.788 6.768 -0.980Age 25-34,Male 109419.36 118670 -9250.64 8.206 8.899 -0.694Age 25-34,Female 114395.17 112007 2388.17 8.579 8.400 0.179Age 35-44,Male 121328.71 117184 4144.71 9.099 8.788 0.311Age 35-44,Female 115609.98 113779 1830.98 8.670 8.533 0.137Age 45-54,Male 138658.26 127077 11581.26 10.398 9.530 0.869Age 45-54,Female 136904.33 127439 9465.33 10.267 9.557 0.710Age 55-64,Male 90338.77 95032 -4693.23 6.775 7.127 -0.352Age 55-64,Female 91693.43 97422 -5728.57 6.876 7.306 -0.430Age 65-74,Male 57475.54 54171 3304.54 4.310 4.062 0.248Age 65-74,Female 62709.50 61828 881.50 4.703 4.637 0.066Age 75+,Male 49772.58 46515 3257.58 3.733 3.488 0.244Age 75+,Female 80867.37 76635 4232.37 6.064 5.747 0.317


17

Second Control Variable

Output Weight Sum of Weights

Target Total

Sum of Weights

Difference

% of Output

Weights

Target % of

WeightsDifference

in %WH NH 1151321.16 1156947 -5625.84 86.340 86.762 -0.422

OT NH 15305.42 12036 3269.42 1.148 0.903 0.245

HISP 85300.51 84230 1070.51 6.397 6.317 0.080

BL NH,AS NH,AI NH 81548.91 80263 1285.91 6.116 6.019 0.096

Raking - Iteration 1

Third Control Variable

Input Weight Sum of Weights

Target Total

Sum of Weights

Difference

% of Input

WeightsTarget %

of WeightsDifference

in %Less than HS 89962.05 143928 -53966.35 6.746 10.793 -4.047

HS Grad 412857.99 414505 -1646.81 30.961 31.085 -0.123

Some College 388163.96 448218 -60054.20 29.109 33.613 -4.504

College Grad 442492.00 326825 115667.37 33.183 24.509 8.674

18


19

Fourth Control Variable


WeightsTarget Total

Sum of Weights

Difference

% of Output

WeightsTarget % of

WeightsDifference

in %Married 816399.38 792326 24073.29 61.223 59.418 1.805

Never married, member unmarried couple

277180.73 300111 -22930.01 20.786 22.506 -1.720

Divorced, Widowed, Separated 239895.88 241039 -1143.29 17.990 18.076 -0.086

Fifth Control Variable


Target Total

Sum of Weights

Difference

% of Output

WeightsTarget % of

WeightsDifference

in %Phone interruption 78558.62 82944 -4385.49 5.891 6.220 -0.329

No Phone Interruption 1254917.38 1250532 4385.49 94.109 93.780 0.329


20

Sixth Control Variable


WeightsTarget Total

Sum of Weights

Difference

% of Output

WeightsTarget % of

WeightsDifference

in %Male, WH NH 553107.34 552171 936.34 41.479 41.408 0.070

Male, BL NH,AS NH,AI NH,OT NH,HISP

101008.49 101946 -937.51 7.575 7.645 -0.070

Female, WH NH 598213.82 604776 -6562.18 44.861 45.353 -0.492

Female, HISP 38304.69 32837 5467.69 2.873 2.463 0.410

Female, BL NH,AS NH,AI NH,OT NH

42841.66 41746 1095.66 3.213 3.131 0.082


21

Seventh Control Variable


WeightsTarget Total

Sum of Weights

Difference

% of Output

WeightsTarget % of

WeightsDifference

in %18-34, WH NH 308020.95 332809 -24788.05 23.099 24.958 -1.859

18-34, BL NH,AS NH,AI NH,OT NH,HISP

80096.58 83585 -3488.42 6.007 6.268 -0.262

35-54, WH NH 442299.71 421539 20760.71 33.169 31.612 1.557

35-54, BL NH,AS NH,AI NH,OT NH,HISP

70201.57 63940 6261.57 5.265 4.795 0.470

55+, WH NH 401000.50 402599 -1598.50 30.072 30.192 -0.120

55+, BL NH,AS NH,AI NH,OT NH,HISP

31856.70 29004 2852.70 2.389 2.175 0.214


22

Eighth Control Variable


WeightsTarget Total

Sum of Weights

Difference

% of Output

WeightsTarget % of

WeightsDifference

in %Cell Phone Only 210390.11 197088 13302.35 15.778 14.780 0.998

Landline Only 270206.34 280297 -10090.31 20.263 21.020 -0.757

Landline and Cell Phone 852879.55 856092 -3212.04 63.959 64.200 -0.241


23



Target Total

% of Output

WeightsTarget %

of Weights

Difference in % from

Iteration1Difference

in %Age 18-24,Male 94727.80 95468 7.104 7.159 -0.626 -0.056Age 18-24,Female 87222.36 90249 6.541 6.768 -0.980 -0.227Age 25-34,Male 116312.81 118670 8.723 8.899 -0.694 -0.177Age 25-34,Female 110348.83 112007 8.275 8.400 0.179 -0.124Age 35-44,Male 118670.65 117184 8.899 8.788 0.311 0.111Age 35-44,Female 113723.15 113779 8.528 8.533 0.137 -0.004Age 45-54,Male 130207.90 127077 9.765 9.530 0.869 0.235Age 45-54,Female 130419.01 127439 9.780 9.557 0.710 0.223Age 55-64,Male 93001.49 95032 6.974 7.127 -0.352 -0.152Age 55-64,Female 96092.37 97422 7.206 7.306 -0.430 -0.100Age 65-74,Male 54156.67 54171 4.061 4.062 0.248 -0.001Age 65-74,Female 62303.45 61828 4.672 4.637 0.066 0.036Age 75+,Male 47039.67 46515 3.528 3.488 0.244 0.039Age 75+,Female 79249.83 76635 5.943 5.747 0.317 0.196




WeightsTarget Total

% of Output

Weights

Target % of

Weights

Difference in % from

Iteration1Difference

in %Age 18-24,Male 95491.87 95468 7.161 7.159 -0.626 0.002Age 18-24,Female 90265.83 90249 6.769 6.768 -0.980 0.001Age 25-34,Male 118621.93 118670 8.896 8.899 -0.694 -0.004Age 25-34,Female 111985.21 112007 8.398 8.400 0.179 -0.002Age 35-44,Male 117205.13 117184 8.789 8.788 0.311 0.002Age 35-44,Female 113769.71 113779 8.532 8.533 0.137 -0.001Age 45-54,Male 127088.93 127077 9.531 9.530 0.869 0.001Age 45-54,Female 127437.46 127439 9.557 9.557 0.710 -0.000Age 55-64,Male 95037.18 95032 7.127 7.127 -0.352 0.000Age 55-64,Female 97426.08 97422 7.306 7.306 -0.430 0.000Age 65-74,Male 54168.73 54171 4.062 4.062 0.248 -0.000Age 65-74,Female 61831.76 61828 4.637 4.637 0.066 0.000Age 75+,Male 46503.23 46515 3.487 3.488 0.244 -0.001Age 75+,Female 76642.96 76635 5.748 5.747 0.317 0.001

24


Eighth Control Variable


Target Total

% of Output

Weights

Target % of

Weights

Difference in % at

Iteration 1

Difference in %

Cell Phone Only 197101.32 197088 14.781 14.780 0.998 0.001

Landline Only 280285.25 280297 21.019 21.020 -0.757 -0.001

Landline and Cell Phone

856089.43 856092 64.200 64.200 -0.241 -0.000

25

**** Program terminated at iteration 7 because all current percents differ from target percents by less than 0.025*****

IMPACT OF CHANGING TO RAKING (IPV) ON THE BRFSS

BRFSS 2010 Combined States a DataDifference In Weighted Percentages

A Excludes AK, DC, TN, SD

incom

e cate

gory1

incom

e cate

gory2

incom

e cate

gory

3

educa

tion c

ategor

y 1

educa

tion c

ategor

y 2

educa

tion c

ategor

y 3

marr

ied

age gr

oup 1

age gr

oup 2

age gr

oup 3 whit

e

black/

AA

Hispani

c0

10

20

30

40

50

60

70

80

LL Post stratified LL Raking LLCP Raking

Marginal Changes Weighted Percentages for Demographic Characteristics, BRFSS 2010

Lowest

Incom

e cate

gory

Middle

incom

e cate

gory

Highest

incom

e cate

gory

Lowest

educa

tion l

evel

Middle

educa

tion l

evel

Highest

educa

tion l

evel

Married

Youn

gest

age g

roup

Middle

age g

roup

Oldest

age g

roup

White

Black/

African

American

Hispan

ic

-0.08

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

0.08

0.1

BRFSS 2010 Combined States DataDifference In Weighted Percentages of Health

Outcomes

ASTHMA

STROKE

HEART A

TTACK

CORONARY HEA

RT DISEA

SE

DIABETES

FAIR/PO

OR HEALTH

NO PHYS

ICAL ACTIV

ITY

OBESITY

HEAVY D

RINKING

CURRENT S

MOKER

NO HEALTH

INSURANCE

0

5

10

15

20

25

30

35

LL Post stratified LL Raking LLCP Raking

A Excludes AK, DC, TN, SD

Marginal Changes for in Weighted Percentage s Health Outcomes, BRFSS

2010

Asthm

aStr

oke

Heart A

ttack

Corona

ry Hea

rt Dise

ase

Diabete

s

Fair/P

oor H

ealth

No Phy

sical

Activ

ity

Obesit

y

Heavy

Drinkin

g

Curren

t Smok

er

No Hea

lth In

suran

ce0

0.20.40.60.8

11.2

Weighted Prevalence Estimates for Current Smoker by Year, Weighting

Method

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100

5

10

15

20

25

Landline Post Stratification Landline Raking WeightingLandline/ Cell Phone Raking Weighting

Year

Prev

alen

ce E

stim

ate

NOTE: All US states and territories except SD and TN

STATE LEVEL OUTCOMES

In Some Cases, Small Changes(Landline Only)

Table 1State-level Responses to Question:

“Has a doctor, nurse or other healthcare provider ever told you that you have diabetes?”

By Type Of Weighting Procedure for Landline DataResponse Landline

Weighted frequency with Post-Stratification

Landline PercentWith Post-Stratification

Landline Weighted frequency with Raking

Landline Percent With Raking

Differences in Landline Percentages(Post-Stratification-Raking)

Yes 434,858 12.26 440,694 12.43 -0.17Yes, but only during pregnancy

26,306 0.74 26,262 0.74 0.00

No 3,031,681 85.44 3,029,545 85.42 0.02No, Pre-diabetes/ borderline diabetes

55,454 1.56 50,196 1.42 0.15

In Some Cases, Larger Differences– But Not Consistent Differences

(Landline Only)Table 2

State-level Responses to Question:“Would you say that in general your health is excellent, very good, good, fair or poor?”

By Type Of Weighting Procedure for Landline DataResponse Landline Weighted

Frequency With Post-Stratification


Landline Weighted Frequency With Raking


Differences In Landline Percentages(Post-Stratification - Raking)

Excellent 631,742 17.83 575,541 16.27 1.56Very Good 1,037,345 29.27 963,330 27.23 2.04Good 1,107,272 31.26 1,111,484 31.42 -0.16Fair 519,248 14.65 591,716 16.73 -2.07Poor 247,424 6.98 295,425 8.35 -1.37

In Some Cases, Consistent Differences(Landline Only)


“During the past month, other than your regular job, did you participate in any physical activities or exercises such as running, calisthenics, golf, gardening, or walking for

exercise?”By Type Of Weighting Procedure for Landline Data

Response Landline Weighted Frequency With Post-Stratification





Yes 2,448,288 68.97 2,342,381

65.98 2.99

No 1,101,378 31.03 1,207,643

34.02 -2.99

But Differences Go Away Sometimes When Cell Phones Are Added


“During the past month, other than your regular job, did you participate in any physical activities or exercises such as running, calisthenics, golf, gardening, or walking for exercise?”

By Type Of Weighting Procedure for Landline and Cell Phone Data

Response

Landline Weighted Frequency With Post-Stratification





Landline And Cell Phone Weighted Frequency With Raking

Landline And Cell Phone Percent

Landline And Cell Phone Differences In Percentages (Post-Stratification - Raking)

Yes 2,448,288

68.97 2,342,381

65.98 2.99 2,447,823

68.96 0.02

No 1,101,378

31.03 1,207,643

34.02 -2.99 1,102,053

31.04 -0.02

Persistent Differences May Exist Even When Adding Cell Phone Responses


“Do you smoke cigarettes every day, some days or not at all?”By Type Of Weighting Procedure for Landline and Cell Phone Data

Response Landline Weighted Frequency With Post-Stratification





Landline And Cell Phone Weighted Frequency With Raking

Landline And Cell Phone Percent

Landline And Cell Phone Differences In Percentages (Post-Stratification - Raking)

Every day

581,967 36.32 704,831

40.95 -4.63 676,129

40.40 -4.08

Some Days

213,724 13.34 248,782

14.45 -1.12 199,278

11.91 1.43

Not At All

806,827 50.35 767,708

44.60 5.75 798,181

47.69 2.65

CONCLUSIONS

Conclusions (1) New weighting procedures are needed to

keep pace with changes in personal communications.

The inclusion of new variables and more complex weighting procedures for large scale survey data are now feasible, because of improvements in computer capacity.

There will be some differences in estimates when weighting procedures change and when new variables for weighting are introduced.

Examples shown here are only depictions of potential outcomes of changes at the BRFSS.

Conclusions (2) Good news: demographic characteristics

adjusted to more closely match Census data.

Most health outcomes indicate increases in risk behaviors (especially when state data are aggregated).

Some increases in chronic conditions, but uneven across states.

Thank You

For more information please contact Centers for Disease Control and Prevention

1600 Clifton Road NE, Atlanta, GA 30333Telephone: 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348E-mail: [email protected] Web: http://www.cdc.gov

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Office of Surveillance, Epidemiology, and Laboratory ServicesDivision of Behavioral Surveillance

Date post:	26-Feb-2016
Category:	Documents
Upload:	marlis
View:	31 times
Download:	1 times

The Effects of Raking and Cell Phone Integration on BRFSS Outcome s

Documents