Consumer Expenditure Survey Redesign

Jennifer EdgarBureau of Labor Statistics

COPAFS Quarterly MeetingMarch 4, 2011

1. Challenges facing CE

2. Redesign overview

3. 2010 Methods Workshop

Outline

Challenges Facing CE:Motivation for

Redesign

Motivation for Redesign

Reducing measurement error Maintaining/improving response rates Meeting user needs Changes in society and technology Need for flexibility

Measurement Error: Evidence

The ratio of CE aggregate expenditures to PCE aggregate expenditures for many spending categories has shown a declining trend

Subgroup comparisons show differences in expenditure reporting by mode of interview and use of recall aids

The 2008 CE Internal Program Review Report cites item nonresponse and measurement error as areas of concern

5

Measurement Error: Possible Sources

Complexity: detailed information Conditioning: 22 sections, 5 waves Proxy reporting: household respondent Recall effects: 3-month reference period Survey length: 60 minute average Telephone administration: 33%

6

Motivation: Response Rates

Meeting User Needs

Varied users: CPI Bureau of Economic Analysis (BEA) IRS U.S. Defense Department U.S. Bureau of the Census Other federal and state agencies Businesses, researchers and the general

public

Meeting User Needs

Varied needs: Detailed monthly expenditure estimates Annual household expenditures

By demographic subgroupBy geographic region

Housing values and rental equivalence Sales tax information

Changes in Society and Technology

Shifts in consumer behaviors Automatic withdrawals Online bill paying Online shopping Big box stores (e.g. Target, Costco)

Technological advances Increasing respondent use of

technology (internet, smart phones) Emergence of new data collection

methods

Need for Flexibility

Allow for faster implementation of changes To respond to changes in marketplace To incorporate changes in data

collection technology

Allow for mixed-mode interviewing

Redesign Overview: Gemini

Redesign Overview: Gemini

Mission Improve data quality through a

verifiable reduction in measurement error, with a particular focus on under-reporting

Objective Develop a detailed research plan for

implementing a redesign of the CE surveys, including piloting, evaluation, and transition

Gemini Tasks

2009-2010 Produce database of CE research Refine data requirements and

priorities Define data quality for CE

2010-2011 Sponsor internal and external events

for information gathering and expert review

Gemini Tasks

2010-2013 Conduct research2012 Assess user impact of design

alternatives2013 Recommend a survey redesign Propose a transition roadmap2013 + Piloting, Evaluation, Transition

15

Gemini Research

Current research addressing: Recall period Interview frequency Split questionnaire designs Interview length and reporting quality Records availability and utility Feasibility of a balance edit Individual diaries Online diaries

Gemini Project Website

www.bls.gov/cex/geminimaterials.htm

Gemini overview Gemini events Gemini presentations CE survey methods research CE methods and survey forms Other redesign related events and

studies 17

2010 Methods Workshop

2010 Methods Workshop

Objectives:

Identify existing knowledge on key topics

Obtain recommendations for research to support and inform the CE redesign

Workshop Approach

Identified Five key survey topics Two experts for each topic

Write paper and give presentation Additional experts to participate in

discussion

BLS ‘issue papers’ Explaining why the topic is considered key Describing existing CE research Asking key questions for redesign process

Methods Workshop Topics

Proxy Reporting Recall Interview Structure Global Questions Split Questionnaire Designs CE “From Scratch” Next Steps

Methods Workshop Results

Papers and presentations provided useful information on each topic

Discussion allowed CE to ask follow-up questions and refocus discussion

No absolute answers or definite research projects identified, but promising leads in many directions

Key Recommendations 1

Identify objectives For the survey

Minimum CPI requirements−Can they be changed?

Key estimates produced−What is required in the future?

For the redesignIdentify metrics to evaluate redesign

alternatives and final implemented design

Key Recommendations 2

Reconsider recall Respondents may not be able to

accurately recall the expenditure information

Respondents may not have encoded the expenditure informationAutomatic paymentsProxy issues

Respondents may not align memories with CE categories or interview structure

24

Key Recommendations 3

“It depends...” The effectiveness/feasibility of

most issues vary by expenditure category, CU type, etc.Proxy reportingLong/short recall periodsFlexible or rigid interview structuresGlobal questionsUse of technology

Jennifer Edgar

Office of Survey Methods ResearchBureau of Labor Statistics

edgar.jennifer@bls.gov