+ All Categories
Home > Documents > HSP for LinkedIn

HSP for LinkedIn

Date post: 11-Jan-2017
Category:
Upload: daniel-charlu
View: 27 times
Download: 0 times
Share this document with a friend
13
© Hitachi Data Systems Corporation 2015. All rights rese © Hitachi Data Systems Corporation 2015. All rights rese Hyper Scale- Out Platform Daniel Charlu Principal Product Manager
Transcript
Page 1: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.1 © Hitachi Data Systems Corporation 2015. All rights reserved.1

Hyper Scale-Out Platform

Daniel CharluPrincipal Product Manager

Page 2: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.2 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only.

Hyper Scale-Out Platform (HSP)

Optimized for…

Hadoop and Analytics

Scale-Out, Hyper-Converged (compute and storage)

Private Cloud ServicesDevOps Approach

Compatible APIs

NFS

Page 3: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.3 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only. © Hitachi Data Systems Corporation 2015. All rights reserved.

HSP key advantages and benefits

Dramatically Reduce

Setup Time

Data-in-Place Processing

Better workflow =Accelerate Time to Results

Speed Data Ingest

Automated and Self-Managed

Run Virtualized Apps at the Data Source

APIs

Page 4: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.4

Elastic Scaling of Multiple Applications

KVMKVMKVMKVMKVMKVMKVMKVM

Elastic VM management – persistent data

HS

P

“Active” Data Lake – Global Namespace

Data-in-place processing for

fast results

Efficiently run analytics apps

at the data source

Scale-out pay-as-you-grow

“pop-up” VMs and storage

Hyper Scale-Out Platform

Manage in open cloud

environments

Enterprise Scalable File System

API

Page 5: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.5 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only. © Hitachi Data Systems Corporation 2015. All rights reserved.

12 cores, 192 GB memory

8 GB NVRAM

72 TB storage (12 x 6 TB HDD)

2x 40GbE internal fabric

10 GbE external interface

20 nodes per rack‒ 240 cores, 3.8TB memory‒ 1.44 PB Raw

Extend cluster to multiple racks

HSP-400 System

Turn Key, Highly Scalable

Scale Out

Page 6: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.6 © Hitachi Data Systems Corporation 2015. All rights reserved.6

Use Cases

Page 7: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.7

Analytic Flow vs. Data Flow Leveraging Hadoop and Pentaho

PDI

BA

Data Source Connectors

Dashboards

PDI

Other Use ofResults

PDI

Data Pre-processingOr Blending step

Map/Reduce Process

Elastic Search

No SQL DB

PDI

Another Analytic Step

Impact of Data Transfers

Data Collector

HSP Value: Simplify workflow & management

HSP Data Lake

Avoiding Many Data Transfers

Page 8: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.8 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only. © Hitachi Data Systems Corporation 2015. All rights reserved.

HSP Storage and Compute Platform

Equipment UsageFile System

Billing File System

MaintenanceFile System

Converged Protocols: Streaming, NFS, Swift, …

Converged Data PlatformIndustry Control: Equipment Health, Logistics & Scheduling

Converged storage and compute: spin-up/spin-down VMs on demand Converged workflows: REST APIs and OpenStack integration

Streaming Time Series Numeric

NFS Semi-Structured Text

SWIFT

Unstructured Text

VMs for Streaming

Health Checks

VMVMVMVMVMs

Running Hadoop

with Lucene and Solr

VMVMVMVM

Native or OpenStackManagement Client

VMs Running Hadoop

with HBASE

VMVMVMVM

Page 9: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.9 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only.

Smart Grids

Equipment health

Revisit Drill Sites‒ Analysis

‒ validation

New Analytics‒ ISV Partner app

Energy Analytics Power and Energy

Page 10: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.10 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only.

Automotive Purchasing Analytics‒ Dealer Customer 360 analysis

‒ Merge Credit, Social Media, Existing CRM

Customer 360 Analysis

Best Candidate

HSP Advantage- Parallel ingest- Simplified workflow- Simplified managementl

Page 11: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.11 © Hitachi Data Systems Corporation 2015. All rights reserved.11

DevOps &Cloud Services

Page 12: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.12 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only. © Hitachi Data Systems Corporation 2015. All rights reserved.

HSP Cloud Service with OpenStack DevOps and REST management

HSP Storage (scale-out)

NFS Protocol HSP Mgmt. API

- Cluster- File System- KVM

eScaleFS (Hadoop)

Glance(VM images)

Nova(VM deployment)

SwiftClient

Mgmt. Client<Horizon> (Orchestration)

Native HSPComponents

OpenStack and HSP

Components

OpenStackClients

Swift(Object)

Page 13: HSP for LinkedIn

© Hitachi Data Systems Corporation 2015. All rights reserved.13 © Hitachi Data Systems Corporation 2015. All rights reserved.13

Thank You


Recommended