1

SKEWING THE DATA:POLITICAL POLLING AND CONTROVERSY

Andrew S. Tanenbaum

The Votemaster at (www.electoral-vote.com)

THE ELECTORAL COLLEGE

•  The president is not elected by popular vote•  The president is elected by the electoral college•  Each state has a certain number of electors•  California has 55, Wyoming has 3•  In total there are 538 electoral votes•  In 2004, I started a site that kept track of the state polls

2

SITE ON 6 NOV. 2012

3

HOW DO POLLSTERS DECIDE WHO TO CALL?

•  Random digit dialing–  Phone numbers are of the format (212) 695-xxxx–  Computer picks random xxxx–  Problems: you call businesses, modems, fax machines, …

•  Pollster buys lists of voters (e.g., registered voters)

4

WHAT DO THEY ASK?

•  Candidate favorability•  Horse race questions•  Demographic questions

–  Age–  Party identification–  Gender–  Race–  Education–  Income

5

BACKGROUND ON POLLING

•  A poll using live interviewers costs about $15,000•  Strong bias on who will talk to interviewer•  Response rates are below 10%•  So what do polling firms do?•  Make small samples and do statistical correction

6

THE POLLS ARE IN. NOW WHAT?

7

Feb Mar Apr May Jun Jul Aug Sep OctJan

60%

55%

50%

45%

40%

Poll

THE POLLS ARE IN. NOW WHAT?

8

If candidate is above 50%, he wins

If candidate is below 50%, he loses

Feb Mar Apr May Jun Jul Aug Sep OctJan

60%

55%

50%

45%

40%

THE POLLS ARE IN. NOW WHAT?

9

Feb Mar Apr May Jun Jul Aug Sep OctJan

60%

55%

50%

45%

40%

If we count only the mostrecent poll, the candidate loses

THE POLLS ARE IN. NOW WHAT?

10

Feb Mar Apr May Jun Jul Aug Sep OctJan

60%

55%

50%

45%

40%

If we average the last 2,The candidate wins

* Average

THE POLLS ARE IN. NOW WHAT?

11

Feb Mar Apr May Jun Jul Aug Sep OctJan

60%

55%

50%

45%

40%

Should this poll count?It was in June

*Average

HOW TO AVERAGE POLLS

•  How you handle this is controversial•  If the window is too long, you miss rapid changes•  If the window is too short, not so accurate•  Should partisan pollsters be included?•  Should pollsters be weighted by previous accuracy?

12

POLLING CONTROVERSIES

•  Is the sample really random?–  RDD: Homes with 0 landlines? 3 landlines?–  Lists: Which ones? Are they current?–  Who to poll? All adults? Registered voters? Likely voters?–  Nonresponse bias (e.g. young mothers vs. old widows)–  It is illegal for a computer to call a cell phone

•  Methodology–  Live interviewer vs. robo vs. Internet polls–  Question order and wording

13

POLLING CONTROVERSIES

•  Polling Non-English-speaking voters–  Most firms just ignore them

•  Dealing with lying respondents–  Bernie-or-bust voters–  Bradley effect–  Trump does better in robo polls than live interviewer polls

•  Some pollsters just made up the data! No real calls

14

DATA CORRECTION

•  Pollsters know the demographics of the population•  If the sample demographics are different, they correct•  Suppose it is known that 52% of voters are women•  But in some poll, only 47% were women•  Then they weight each woman in the sample as 52/47•  They also correct for political party identification•  Suppose they know 38% of voters are Democrats•  But in this sample only 32% were•  Each Democrat then is weighted as 38/32

15

UNSKEWED POLLS

•  In 2012, a man named Dean Chambers thought Mitt Romney would beat Barack Obama

•  The polls showed otherwise•  He assumed the polls were wrong•  But how?•  He blamed it on the data correction•  He re-corrected the raw data based on a much

more Republican electorate•  His Website predicted a Romney win

16

UNSKEWEDPOLLS.COM

17

Romney got 206 electoral votes and lost

RIGHT-WING MEDIA BELIEVED IT TOTALLY

•  Right-wing media & Websites took this as the truth•  Millions of people believed Chambers•  Fox News saw him as a rising star•  Mitt Romney expected to win based on this•  On election night, Karl Rove refused to accept results

18

WERE THE 2016 POLLS “SKEWED”?

•  RealClear Politics had Clinton +3.3% ± 2%•  Final result nationally was Clinton +2.1%•  The national polls were spot on•  The state polls showed it too close to call in 12 states

19

CONCLUSION

•  Somebody can take the data and modify it•  People & media might prefer the “modified” data•  Millions of people believed the “modified” data

20

21

THANK YOU !

ELECTORAL-VOTE.COM

22