The Evolution of Data ScienceKenny DanielCTO, Algorithmia

July 24, 2015

Kenny Daniel - CTO, Algorithmia• Graduate research in Artificial Intelligence and Mechanism Design• Multiple published algorithms and papers in Machine Learning• Received $1 million from DOT “Engineering Tomorrow’s Transportation Market”• B.S. Carnegie Mellon University, M.S., Ph.D. (on leave) USC• Data Scientist and Computer Vision specialist for Delectable, Inc• Initial and current overall architect of Algorithmia Platform

Make state-of-the-art algorithms

accessible and discoverable by

everyone.

Evolution of Data Science

● History of data science

● Modern data science

● Future speculation

Pre-cloud● Mainframes● Universities● Research Facilities● Finance● PhD researchers, highly specialized

More pre-planning, less exploratory

Source and Inspiration: http://www.slideshare.net/AlbertWenger/the-no-stackstartup

1990s Connectivity$10,000 per month

Servers$20,000 per box

Storage$1,000/GB

2000s Connectivity$1,000 per month

Servers$1,000 per box

Storage$10/GB

2010s Connectivity10 cents/GB

Servers20 cents/hour

Storage12 cents/GB

NOW Backend using ParseSearch using AlgoliaSynchronization using FirebaseVideo calls and SMS using TwilioPayments using StripeVideo recording using ZiggeoSend and track emails using MailgunCustomer service using IntercomShip product using Shyp

“no one got fired for using AWS”cost, security, convenience

“We used to leak memory.

Now we leak instances.

Soon we will leak entire data centers.”

- Dan Kaminsky

Previously, data analysis was done by domain experts

Now, shift toward data science as its own field

A new field is born

“Hi, I’m a Data Scientist”

Lots of Data

Little Intelligence

“Data is inherently dumb. It doesn’t actually do anything unless

you know how to use it...

The next digital gold rush will be focused on how you do

something with data.”

- Peter Sondergaard (Gartner Research)

1990s TechnologyHPC, Mainframes

2000s

2010s

NOW Generalist Big Data such as Amazon EMRLarge Data Processing such as DatabricksReal Time Processing such as Amazon KinesisData Repositories such as SocrataData Collectors such as KimonoDSaaS for Customer Analytics such as CaptricityDSaaS for Marketing such as AcxiomDSaaS for Security such as FortscaleHosted Machine Learning such as BigML, DatoAlgorithms-as-a-Service such as Algorithmia

TechnologyIn-house clusters

TechnologiesCloud, Hadoop, Spark

UsersCorporations, tech startups

UsersIndividual data scientists

UsersResearchers, hw engineers, committees

Behold...

Data Sciencein a Spreadsheet

Future of Data Science● How will these trends continue?

● What will future tools look like?

● What is the role of data scientists going forward?

Data is less structured, and less amenable to traditional

data analysis without pre-processing

● Unstructured text

● Images

● Video

Future… new data sources

Future… building blocks

Topic Analysis

Twitter Youtube Satellite Imagery

Computer Vision

Artificial Neural Networks

Future… more autonomous

AutoMLEnsemble learningHyperparameter optimization

JOIN: algorithmia.com/signup?invite=SeattleDS(will post to meetup group)

REACH OUT: kenny@algorithmia.com