+ All Categories
Home > Documents > The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical...

The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical...

Date post: 08-Jul-2018
Category:
Upload: nguyendung
View: 225 times
Download: 0 times
Share this document with a friend
17
The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore [email protected]
Transcript
Page 1: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors

Andrew W. Moore [email protected]

Page 2: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

This talk

• Examples from the largest scale commercial big data systems.

•My personal top five recommendations for critical technology investments for large data systems

Page 3: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu
Page 4: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu
Page 5: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

Decorated Entities

Ingested Unstructured Facts

Page 6: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

Images

Decorated Entities

Ingest Unstructured Facts

Normalize

Human-in-the-loop

Page 7: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

BA

CK

GR

OU

ND

SER

VIN

G

Images

Decorated Entities

Ingest Unstructured Facts

Normalize

Human-in-the-loop

Query

Delivery

Model Click Streams

Context

Result Page

Inventory

ConOps

Page 8: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

FLEET B

AC

KG

RO

UN

D

SERV

ING

Images

Decorated Entities

Ingest Unstructured Facts

Normalize

Human-in-the-loop

Query

Delivery

Model Click Streams

Context

Result Page

Inventory

Telemetry Weather Map Hot Swap

HwOps

ConOps

Page 9: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

FLEET B

AC

KG

RO

UN

D

SERV

ING

TR

UST

Images

Decorated Entities

Ingest Unstructured Facts

Normalize

Human-in-the-loop

Query

Delivery

Model Click Streams

Context

Result Page

Inventory

Telemetry Weather Map Hot Swap

HwOps

ConOps Recommender

Opinions

Mystery Shopping Anti Fraud

Page 10: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

FLEET B

AC

KG

RO

UN

D

SERV

ING

TR

UST

Knowledge Data Action

Images

Decorated Entities

Ingest Unstructured Facts

Normalize

Human-in-the-loop

Query

Delivery

Model Click Streams

Context

Result Page

Inventory

Telemetry Weather Map Hot Swap

HwOps

ConOps Recommender

Opinions

Mystery Shopping Anti Fraud

Page 11: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

My personal top five recommendations

1 The Top of The Stack

2 Entities

3 Data Intensive Computing Architectures

4 Delineation of the Data Science Stack

5 Human-in-the-loop

Page 12: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

My personal top five recommendations

1 The Top of The Stack

2 Entities

3 Data Intensive Computing Architectures

4 Delineation of the Data Science Stack

5 Human-in-the-loop

Page 13: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

My personal top five recommendations

1 The Top of The Stack

2 Entities

3 Data Intensive Computing Architectures

4 Delineation of the Data Science Stack

5 Human-in-the-loop

Page 14: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

My personal top five recommendations

1 The Top of The Stack

2 Entities

3 Data Intensive Computing Architectures

4 Delineation of the Data Science Stack

5 Human-in-the-loop

Decision Support Visualization, Consulting Workflow, Human-in-loop systems

Modeling Prediction, Clustering, Structure Discovery

ML Components Spatial Join, Fuzzy Join, MLE, Sampling

Data Science Kernel Layer Blobstore, KeyVal, Redundancy Management

Device Layer Multicore, GPU, Sensors

Page 15: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

My personal top five recommendations

1 The Top of The Stack

2 Entities

3 Data Intensive Computing Architectures

4 Delineation of the Data Science Stack

5 Human-in-the-loop Panstarr telescope image (Kaiser et al)

Page 16: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

My personal top five recommendations

1 The Top of The Stack

2 Entities

3 Data Intensive Computing Architectures

4 Delineation of the Data Science Stack

5 Human-in-the-loop

Page 17: The Data Science Technology Stack - NITRD · The Data Science Technology Stack Contrasting critical issues in the public, scientific and commerce sectors Andrew W. Moore awm@cs.cmu.edu

My personal top five recommendations

1 The Top of The Stack

2 Entities

3 Data Intensive Computing Architectures

4 Delineation of the Data Science Stack

5 Human-in-the-loop

Autonomy

Cognitive Assistance

Decision Support

Modeling

ML Components

Data Science Kernel Layer

Device Layer


Recommended