Exploiting Big Data for Statistics:Exploiting Big Data for Statistics:Some Implications for Policy

Andrew Wyckoff*

Directorate for Science, Technology & IndustryOrganisation for Economic Co-operation and Development

F b 25 February 2013

* The views expressed in this presentation are those of the authors anddo not necessarily reflect the opinions of the OECD or its Membership.

Overview

• Some applications today

• Implications for public policy:– Economic & social policies;Economic & social policies;– Statistical policy

• Discussion

S & C

Mesh Networks / Ambient Computing

Supply Chain Management Security & Access Control

Work In Process Tracking Consumer ApplicationsWork In Process Tracking Consumer Applications

Asset Management Environmental Applications

3

Internet traffic flows continue to grow

90,000

100,000

MOBILEDATA

Monthly global IP traffic (Petabytes), 2005-2014

70,000

80,000MOBILE DATA

Consumer Internet video 

Consumer Voice over IP (VoIP) 

50,000

60,000Consumer Online gaming 

Consumer Video calling 

20,000

30,000

40,000Consumer Web, email, and data 

Consumer file sharing 

CONSUMERVOD

0

10,000

20,000 CONSUMER VOD

BUSINESS INTERNET / INTRANET

2010 2011 2012 2013 2014 2015

Source: CISCO VNI 2011

A lot of big data buzz

• “Data is the new oil.” Andreas Weigend, Stanford (ex Amazon)

• “The future belongs to companies and people that turn data into products”, Mike Loukides, O’Reilly Media

“The challenge–“Ten reasons why Big Data will

“Why big data is a big deal”InfoWorld 9/1/11

The challenge–and opportunity–of big data”McKinsey Quarterly 5/11

Big Data will change the travel industry”Tnooz 8/15/11

“Keeping Afloat in a Sea of 'Big

InfoWorld – 9/1/11 McKinsey Quarterly—5/11

“Getting a Handle on Big Data with

Tnooz -8/15/11

“The promise ofin a Sea of Big Data”ITBusinessEdge – 9/6/11

on Big Data with Hadoop”Businessweek-9/7/11

The promise of Big Data”Intelligent Utility-8/28/11

IT has always had an impact on Statistics

see www.abs.gov.au

Price Statistics MIT Billion Price ProjectMIT Billion Price Project

Demand for Jobs / SkillsHelp Wanted Statistics from the Conference BoardHelp Wanted Statistics from the Conference Board

New Job Starts / Job ChangesLinkedInLinkedIn

OECD Output ForecastsSWIFTSWIFT

Some economic policy implications

BenefitsBenefits

• Timeliness & now casting• Timeliness & now-casting• Robustness & granularity• Affordability & access• Democratisation & creativityDemocratisation & creativity

Unknown properties of Web Data

Source: www nature comSource: www.nature.com

Some economic policy implications

Ch llChallenges

•Unknown bias•Potential Instability•Quality

Some implications for NSOs: Will they get by-passed by “Big Data” ?Will they get by passed by Big Data ?

Some policy implications for NSOs

Good news

• Big Data tools are becoming widely availablebecoming widely available

• The Cloud can address infrastructure needs

• The “statistical commons” • The statistical commons grows

Some policy implications for NSOs

Challenges to AddressC a e ges o dd ess

A &O hi•Access &OwnershipP i•Privacy

•Liability•Liability•Skills•Skills

Access to and ownership of proprietary data

Privacy Issues

• image from http://wwwalthdatainnovation.com/sites/datawork.drupalgardens.com/files/styles/large/public/target.jpg

Liability

Skills “…the sexy job in the next 10 years

will be statisticians.”

Source: NYT, 5 August 2009

Possible new NSO roles

• Take on a new mission as a trusted 3rd party whose role would be to certify the statistical quality y q yof these new sources?

• Issue statistical “best practices” in the use of non-traditional sources and the mining of “big data”?g g

• Use non-traditional sources to augment (and g (perhaps replace) their official series?

Going ForwardG “A bl d d d t ld” b ildi • Groves: “A blended data world” – building on-top of existing surveys

lib i b d d– Calibrating web data to survey data

• Use of relative vs. absolute measures, Use of relative vs. absolute measures, now-casting

• Develop new methodologies

• Active Experimentation extracting • Active Experimentation, extracting lessons, devising “best practices”