I. Survey Design Basics

A. Foundations

• What is your idea or argument? – Ex. Public anger about the ACA will hurt the

Democrats in 2014. • What does that argument imply about data

(your hypothesis)?– Democrats will do worse than expected or normal– Their underperformance will be accounted for by

public attitudes toward ACA.

B. Conceptualization• What concepts are you trying to measure?

– What? Toward whom? When?– Electoral support (what) for Democratic candidates (who) in the

November 2014 election (when)• What Quantities are you trying to measure?

– Averages: Means, Percentages (expectations, sizes of groups in society)

– Quantiles: Medians, quintiles, etc. (value of X such that Q percent are less than X). (inequalities)

– Variance: Spread (risk)– Relationships: correlations, differences in means, regressions

(association, causation, prediction)

C. Population

• Definition. Universe of all persons (or units) you seek to study.

• Finite and infinite. Finite: known, fixed population. All people in US today. Infinite: Continuous variables, Future (distribution). Stock market value tomorrow.

D. Sample Construction

• Mode of Contact. How communicate. – In person, mail, phone, internet

• Who is contacted?– Random– Representative

E. Survey Instrument

• Means of collecting information• Question Format• Constraints – time limits, change behavior by

asking too much.

F. Examples1. Exit Poll

Questionnaire: 18 questionsSample Precincts (problem of clustering)Sample individuals as leaveRespondents and Non-RespondentsDevice (paper, handheld?)

2. Phone PollsQuestionnaire about 20 or so questions

Random Digit DialingVery high non-response (what’s random?)

II. Thinking About Data

Questions, Frequencies and Tables

Democrat Republican Independent Others

35.3%(4,866)

24.4%(3,360)

26.7%(3,687)

13.7%(1,888)

In politics today, do you consider yourself to be a Democrat, Republican, Independent, or something else?

Race and HispanicityWhite Alone(H and NH)

Black Alone(H and NH)

Asian Alone(H and NH)

Hispanic White Non-Hisp

White Hisp

77.9% 13.1% 5.1% 16.9% 63.0% 14.9%

White Non-White

Hispanic 14.9% 2.0%

Non-Hispanic 63.0% 20.1%

Census Version: Race Question

Black (13%) White (79%)

Asian (5%)

Census Version: Hispanicity Question

NONHipsanic (83%)

-Hispanic (17%)

Census Version: Race and Hispanicity are Separate Question

Black (13%)

Hipsanic (17%)

White (79%)

Asian (5%)

Tabular PresentationRace and Hispanicity

Non-White White Total

Hispanic 2.0%11.8%9.0%

14.9%88.2%19.0%

16.9%100.0%-

Non-Hispanic 20.1%22.6%91.0%

63.0%77.4%80.9%

89.1%100.0%-

Total 22.1%-100.0%

77.9%-100.0%

100.0%

Marginal and Joint Frequency

• Terminology– Variables

Y = i, i = 1, 2, … IX = j, j = 1, 2, … J

– Marginal Frequencies or Probabilities. P(Y=i) or P(X=j)

– Joint Frequencies or Probabilities.P(Y=i and X = j)

Tabular PresentationRace and Hispanicity

Non-White White Total

Hispanic 2.0%11.8%9.0%

14.9%88.2%19.0%

16.9%100.0%-

Non-Hispanic 20.1%22.6%91.0%

63.0%77.4%80.9%

89.1%100.0%-

Total 22.1%-100.0%

77.9%-100.0%

100.0%

Tabular PresentationMarginals on Hispanicity

Non-White White Total

Hispanic 2.0%11.8%9.0%

14.9%88.2%19.0%

16.9%100.0%-

Non-Hispanic 20.1%22.6%91.0%

63.0%77.4%80.9%

83.1%100.0%-

Total 22.1%-100.0%

77.9%-100.0%

100.0%

Tabular PresentationRace and Hispanicity

Non-White White Total

Hispanic 2.0%11.8%9.0%

14.9%88.2%19.0%

16.9%100.0%-

Non-Hispanic 20.1%22.6%91.0%

63.0%77.4%80.9%

83.1%100.0%-

Total 22.1%-100.0%

77.9%-100.0%

100.0%

Tabular PresentationJoint Frequencies for Race and Hispanic

Non-White White Total

Hispanic 2.0%11.8%9.0%

14.9%88.2%19.0%

16.9%100.0%-

Non-Hispanic 20.1%22.6%91.0%

63.0%77.4%80.9%

83.1%100.0%-

Total 22.1%-100.0%

77.9%-100.0%

100.0%

Thinking Conditionally

• Definition– A Conditional Statement is the Set of Values of a

variable (say Y) subject to the restriction that another variable or variables take on a specified set of values.

• Terminology.– Y given X=j or Y|X=j– Example: Y=Hispanic|X=Non-White. – That’s different from Y= Hispanic and X = Non-White.How So? How is a statement Y given X different from Y and X?

Conditional Frequency or Probability, Defined

• P(Y=i|X=j) = P(Y=i and X = j)/P(X=j)• P(X=j|Y=i) = P(Y=i and X = j)/P(Y=i)

• In statistical software these are called row and column percentages in tables. Joint frequencies are called cell frequencies

tab y x, row col cel

Census Version: Race and Hispanicity are Separate Question

Black (13%)

Hipsanic (17%)

White (79%)

Asian (5%)

Census Version: Race and Hispanicity are Separate Question

Black (13%)

Hipsanic (17%)

White (79%)

Asian (5%)

W and H

Ex. Race and Hispanic

• Marginal Probabilities Joint Probabilities– Hispanic = Yes: .17 H, W: .15– Hispanic = No: .83 H, NW: .02– White = Yes: .78 NH, W: .63– White = No: .22 NH,

NW: .20• Conditional– P(W|H) = .15/.17 = .88– P(H|W) = .15/.78 = .19

Example 2. Race and Party (CCES 09)Democrat Republican Independent Other Total

White 3,03729.862.4

2,93228.887.3

2,89728.578.6

10,17910073.8

Black 1,14370.823.5

654.01.9

24114.96.5

1,61510011.7

Hispanic 47240.69.7

21918.86.5

24020.66.5

1,1631008.4

Other 8431006.1

Total 4,86635.3

3,36024.4

3,68726.7

1,8877.8

13,800