Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of...

Post on 27-Feb-2020

3 views 0 download

transcript

Nico Gaviola Head of Healthcare and Lifesciences UKIE nicogaviola@google.com

Data exploration at the speed of thought Lessons learned from inside Google

Google’s m ission is t o organize t he wor ld’s inform at ion and m ake it universally accessible and useful.

Sundar Pichai CEO, Google

uploads per minute

users

search index

query response time

500hrs

1B+

100PB+

0.25s

Google computing scale

Hitting the limits, early on...

The Anatom y of a Large -Scale Hype rtextua l Web Search Engine 1996, Se rgey Brin and Lawrence Page Com pute r Science Departm ent, Stanford Unive rsity, Stanford , CA 94305

2012 2013 2002 2004 2006 2008 2010

Google Research Publications referenced are available here: http://research.google.com/pubs/papers.html The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2009 http://research.google.com/pubs/pub35290.html

GFS

MapReduce

BigTable

Single Node to Cluster

Google’s Data Research

2002 2004 2006 2008 2010 2012 2014 2016

GFS

MapReduce TensorFlow

BigTable

Dremel

Colossus

Flume

Megastore

Spanner

Millwheel

PubSub

F1

Google’s Data Products

2002 2004 2006 2008 2010 2012 2014 2016

ML

PubSub

DataFlow

DataStore

DataFlow

Cloud Storage

BigQuery

BigTable

DataProc

Cloud Storage

Programming

Resource provisioning

Performance tuning

Monitoring

Reliability Deployment & configuration

Handling growing scale

Utilization improvements

Typical Big Data Jobs

Big Data with Google Focus on ins ights . Not infras tructure.

Programming

Understanding

Google’s Big Data Vision

Pay $5 per TB

Active contributor to numerous OSS projects

Make migrations eas ier with open APIs

Cus tomers s hould us e us becaus e they love us , not becaus e they are unable to move off

Open Source & APIs

12

Confidential & Proprietary Google Cloud Pla tform 13

You own your data and remain Data Controller

You can delete or remove your data a t

any time

Google does not s hare your content or

pers onal information google.com/ privacy

Strict Internal Policies : a ll acces s es to

cus tomer or cons umer data applications are

logged

Internal data acces s auditing tracks

Googlers

Google Security Model & You!

Example

“Right at the start of the partnership we were able to reduce time to insight from 96 hours to 30 minutes by using BigQuery”

Gary Sanders Head of Digital Analytics

What’s Next?

“Machine learning is a core, trans formative way by which we’re re-thinking how we’re doing everything”

Sundar Pichai CEO, Google

15% reduction in PUE

Fully trained, easy to use Machine Learning models

Cloud Trans la te

Cloud Vis ion

Cloud Speech

Cloud Natural Language Stay tuned…

Use your own data to train models

Cloud Storage BigQuery Cloud Datalab

Cloud Machine Learning

Develop, Model, Train, Tes t

One more thing

Free training courses coming near you!

Thank you!