data science @ The New York Times

chris.wiggins@columbia.educhris.wiggins@nytimes.com@chrishwiggins

references: bit.ly/icerm

data science: mindset & toolset

drew conway, 2010

references: bit.ly/icerm

modern history:2009

references: bit.ly/icerm

modern history:2009

references: bit.ly/icerm

data science @ The New York Times

chris.wiggins@columbia.educhris.wiggins@nytimes.com@chrishwiggins

references: bit.ly/icerm

biology: 1892 vs. 1995

references: bit.ly/icerm

biology: 1892 vs. 1995

references: bit.ly/icerm

new toolset, new mindset

1851

references: bit.ly/icerm

news: 20th century

church state

references: bit.ly/icerm

church

references: bit.ly/icerm

church

references: bit.ly/icerm

church

news: 20th century

church state

references: bit.ly/icerm

news: 21st century

church state

engineering

references: bit.ly/icerm

news: 21st century

church state

engineering: DSE

references: bit.ly/icerm

1851 1996

newspapering: 1851 vs. 1996

references: bit.ly/icerm

example:

millions of views per hour2015

references: bit.ly/icerm

references: bit.ly/icerm

"...social activities generate large quantities of potentially valuable data...The data were not generated for the purpose of learning; however, the potential for learning is great’’

references: bit.ly/icerm

"...social activities generate large quantities of potentially valuable data...The data were not generated for the purpose of learning; however, the potential for learning is great’’ - J Chambers, Bell Labs,1993

data science: the web

references: bit.ly/icerm

data science: the web

is your “online presence”

references: bit.ly/icerm

data science: the web

is a microscope

references: bit.ly/icerm

data science: the web

is an experimental tool

references: bit.ly/icerm

data science: the web

is an optimization tool

references: bit.ly/icerm

1851 1996

newspapering: 1851 vs. 1996 vs. 2008

2008

references: bit.ly/icerm

“a startup is a temporary organization in search of a repeatable and scalable business model” —Steve Blank

references: bit.ly/icerm

every publisher is now a startup

references: bit.ly/icerm

every publisher is now a startup

news: 21st century

church state

engineering

references: bit.ly/icerm

news: 21st century

church state

engineering

references: bit.ly/icerm

learnings

references: bit.ly/icerm

learnings

- predictive analytics- descriptive analytics- prescriptive analytics

references: bit.ly/icerm

learnings

- predictive analytics- descriptive analytics- prescriptive analytics

cf. modelingsocialdata.org

references: bit.ly/icerm

predictive analytics, e.g.,

cf. modelingsocialdata.org

predictive analytics, e.g.,

“the funnel”

cf. modelingsocialdata.org

interpretable predictive analytics

supe

r co

ol s

tuff

cf. modelingsocialdata.org

interpretable predictive analytics

supe

r co

ol s

tuff

cf. modelingsocialdata.org

arxiv.org/abs/q-bio/0701021

optimization & learning, e.g.,

“How The New York Times Works “popular mechanics, 2015

optimization & prediction, e.g.,

“How The New York Times Works “popular mechanics, 2015

(some models)

(som

e mo

neys

)

recommendation as predictive analytics

recommendation as predictive analytics

bit.ly/AlexCTM

descriptive analytics, e.g,

cf. daeilkim.com ; import bnpy

prescriptive analytics, e.g,

prescriptive analytics, e.g,

prescriptive analytics, e.g,

Reporting

Learning

Test

Optimizing

Exploredescriptive:

predictive:

prescriptive:

Reporting

Learning

Test

Optimizing

Exploredescriptive:

predictive:

prescriptive:

common requirements in data science:

common requirements in data science:

1. people2. ideas3. things

cf. USAF

things:what does DS team deliver?

things:what does DS team deliver?

- build data prototypes- build APIs- impact roadmaps

- build data prototypes

- build data prototypes

cf. daeilkim.com

- build data prototypes

cf. daeilkim.com

- in puppet, w/python2.7- collaboration w/pers. team

- build APIs

- impact roadmaps

flickr/McJex

data science: ideas

data skills

- data engineering- data science- data visualization- data product- data multiliteracies- data embeds

cf. “data scientists at work”, ch 1

data skills

- data engineering- data science- data visualization- data product- data multiliteracies- data embeds

cf. “data scientists at work”, ch 1

data skills

- data engineering- data science- data visualization- data product- data multiliteracies- data embeds

cf. “data scientists at work”, ch 1

data science: people

- new mindset > new toolset

data science: people

summary:pay attention to:

1. people2. ideas3. things

cf. USAF

thanks to the data science team!

data science @ The New York Times

chris.wiggins@columbia.educhris.wiggins@nytimes.com@chrishwiggins