Being Smart with StatisticsTom Coupé

KSE and KEI

Structure of My Presentation

• Is a change really a change? Is a difference really a difference?

– The Margin of Error

• The Chicken or the Egg?

– Causality versus Correlation

Assignment 1

• What percentage of this class believes that the GDP of their country will double by 2021?

Population Statistic versus Sample Estimate

• Population – the group in which I am interested

• Sample – a subgroup of the population

Why Important?

• Sample Statistic =/= Population Statistic

• Different Samples, Different Sample Statistic

• The ‘variance’ of these estimates uncertainty

illustration.pdf

• Differences can be no differences

• Changes can be no changes

The ICPS Example

• gfk_icps_rel_February_66_eng.pdf

The Margin of Error: 3.2%

• in population: p =0.5

• yes =1, no= 0

• ask one person: our expected ‘answer’ =p

• (p percent says 1, 1-p percent says 0) = p*1+(1-p)*0

• ask many times one person: sometimes 1, sometimes 0

• Variation: p*(1-p)

• Standard Deviation: √variation

• If we ask n individuals (in the ICPS case, 1000)

• Standard deviation divided by √ 1000.

• The margin or error is twice that number

• 2*sqrt(0.5*0.5/1000)=0.032

Meaning of 3.2%

• Population Statistic is likely to be between

• Sample Estimate -3.2%

• and

• Sample Estimate +3.2% Implication for ICPS index

Implications for ICPS index I

• – ‘In January 2011, the Consumer Confidence Index (CCI) in Ukraine fell 7.2pp to 75.’

• or• ‘In January 2011, the Consumer Confidence Index

(CCI) in OUR SAMPLE of UKRAINIANS was 75.’

Implications for ICPS index II

• – ‘In January 2011, the Consumer Confidence Index (CCI) in Ukraine fell 7.2pp .’

• Based on Sample, yes!

• But what about the CCI based on the whole Ukrainian population of consumers?

• “Only expectations of changes in unemployment (IECU) improved marginally, losing 1.6pp to end at 127.4.”

Other Examples – Doing Business

• Changes in the Doing Business Index of the World Bank

• Doing Business in Ukraine - World Bank Group.htm

12

EBA Investment Attractiveness Index

2.56 2.57

3.083.25 3.20 3.28 3.40

2.64

2.222.22

3.14

0

1

2

3

4

5

III/2008 IV/2008 I/2009 II/2009 III/2009 IV/2009 I/2010 II/2010 III/2010 IV/2010 I/2011

EBA Investment Attractiveness Index

Good Examples – NRC University Ranking

• http://graduate-school.phds.org/rankings/economics/rank/__MH_____H________H_____________________H_______________________U

Practical Implications

• Point 1 – one ‘true’ number, many estimates

• Point 2 – estimates differ from one sample to another

• Point 3 – changes over time and differences between groups can be due just to changes in sample rather than changes in the underlying true value

Practical Implications

•The world is not simple – don’t fool your readers in believing it is•When reporting numbers, think rather than blindly accept the number you are given•a journalist who is aware of the margin of error, will not be ‘fooled’ into report big conclusions on small differences or small changes

Biased Estimators

• Bias:illustration.pdf

• Not all Samples are “ Good Samples”

– Big Samples

– Random samples!

• Not All Questions are Good Questions

Big Samples

• How do Belgians look like?

• ‘case studies’

Non-Random Samples - Selection

• Selection by Researcher (reporter)– ‘In Search of Excellence: Lessons from America’s

Best-Run Companies’• Self-Selection of Respondents

Problematic Questions

• Leading Questions• Biased Answers

Implication

• When reading numbers, always think about sources of bias!

Causality vs Correlation

Assignment II

• Flip a Coin!

Possible Explanations for Correlation

The Simpsons

(Season 7, "Much Apu About Nothing“)8d840544251f-1333453.vk.flv

Homer: Not a bear in sight. The "Bear Patrol" is working like a charm!

Lisa: That's specious reasoning, Dad.

Homer: [uncomprehendingly] Thanks, honey.

Lisa: By your logic, I could claim that this rock keeps tigers away.

Homer: Hmm. How does it work? Lisa: It doesn't work; it's just a

stupid rock! Homer: Uh-huh. Lisa: But I don't see any tigers

around, do you? Homer: (pause) Lisa, I want to buy

your rock.

Causality versus correlation

• Bain:

‘our clients outperform the market 3 to 1’

ICPS

• “The main reason for growing pessimism among consumers in January 2011 is increased inflationary pressure and the resulting inflationary expectations, as prices for most product groups and consumer services rose: food, fuel and residential services and utilities all showed marked increases,” says ICPS Deputy Director Maksym Boroda. “At the same time, unpromising economic expectations have been spurred by a noticeable financial pinch among ordinary Ukrainians, after having spent considerable amounts during the holiday season. This can be seen in the dynamic of the various components of the CCI, where estimates of current financial standing and the inclination to make major household purchases declined the most”

Happiness

• Two percent of Ukrainians unhappier over year of Yanukovych's presidency, according to poll.htm

• http://www.interfax.com.ua/ukr/press-conference/63304/

How to Prove Causality?

• Controlled Experiment!

Practical Implications for Journalists

• The world is not simple – don’t fool your readers in believing it is

• When reporting, think about causality rather than correlation – don’t confuse these two!

• Be critical about any causality claim – it is very hard to establish causality so as an interviewer you really can ask tough questions about proof of claims.

Misleading with statistics

• Graphs• Choice of Statistic• Per Capita?• Real versus nominal

Sources of Interesting Statistics - International

• World bank data archive

– http://data.worldbank.org/

• European Social Survey (includes Ukraine)

– http://www.europeansocialsurvey.org/index.php

• World Value Survey (includes Belarus and Ukraine)

– http://www.worldvaluessurvey.org/

• EBRD Surveys (includes Belarus and Ukraine)

– LITS http://www.ebrd.com/pages/research/analysis/publications/transition.shtml

– BEEPS http://www.ebrd.com/pages/research/analysis/surveys/beeps.shtml

• TiMMS – PISA – PIRLS (includes Ukraine)

– http://timss.bc.edu/

– http://www.pisa.oecd.org/pages/0,2987,en_32252351_32235731_1_1_1_1_1,00.html

• GOOGLE!!!

Sources of Interesting Statistics - Belarus

• The think tanks

– http://research.by

• IFC Business Environment Surveys

– http://belarus.bel.biz/?p=632

Sources of Interesting Statistics - Ukraine

• Kyiv Institute for International Sociology

– http://kiis.com.ua/ua/bank

• IFC surveys on Business climate

– http://www.ifc.org/ifcext/uspp.nsf/Content/PMSurveysRU

• Ukrainian Longitudinal Monitoring Survey

– http://www.iza.org/de/webcontent/research/ra5

• The think tanks

– www.ier.com.ua

– http://www.icps.com.ua/

– www.uceps.org

– www.case-ukraine.com.ua