Street Fighting Data Science

Pete Skomoroch@peteskomoroch

O’Reilly Strata ConferenceFebruary 28, 2012

To solve hard problems:

Think like a street fighter

Analyze

Improvise

Anticipate

Adapt

How does this apply to Data Science?

Pricing model decreases profit in test stores by 30%

What went wrong?

• Ran complex “black box” model

• Didn’t analyze the data first

• Didn’t anticipate elasticity errors

How could this have been avoided?

The Men Who Stare at Charts

Look at your data

Raw Data: FEC Contributions

retired 32938self-employed 25454information requested per best efforts 1313homemaker 4992the bank of new york 65john mccain 2008 57u.s. government 121idt corp. 54merrill lynch 273blank rome l.l.p. 51department of defense 100u.s. army 90us army 141none 642greenberg traurig 118northrop grumman 105at&t 141citigroup 134bridgewater associates 44univision communications inc. 36

not employed 118672self employed 92973information requested 17627refused 728unemployed 1493self-employed 5919university of california 825microsoft 915university of chicago 616harvard university 848google 662stanford university 716university of washington 614ibm 1016columbia university 782university of michigan 514freelance 372sa 150sidley austin llp 509na 999

retired 32938self-employed 25454information requested per best efforts 1313homemaker 4992the bank of new york 65john mccain 2008 57u.s. government 121idt corp. 54merrill lynch 273blank rome l.l.p. 51department of defense 100u.s. army 90us army 141none 642greenberg traurig 118northrop grumman 105at&t 141citigroup 134bridgewater associates 44univision communications inc. 36

not employed 118672self employed 92973information requested 17627refused 728unemployed 1493self-employed 5919university of california 825microsoft 915university of chicago 616harvard university 848google 662stanford university 716university of washington 614ibm 1016columbia university 782university of michigan 514freelance 372sa 150sidley austin llp 509na 999

Katherine Alexandra

“Don't indulge in any unnecessary, sophisticated moves. You'll get clobbered if you do, and in a street fight you'll have your shirt zipped off you.”

- Bruce Lee

Look at your errors

• Sanity check row counts

• Track errors over time

• Find patterns in the error data

• Add missing features to models

• Replace models entirely

Analyze

Improvise

Anticipate

Adapt

Think like a street fighter