HSP for LinkedIn

Post on 11-Jan-2017

27 views 0 download

transcript

© Hitachi Data Systems Corporation 2015. All rights reserved.1 © Hitachi Data Systems Corporation 2015. All rights reserved.1

Hyper Scale-Out Platform

Daniel CharluPrincipal Product Manager

© Hitachi Data Systems Corporation 2015. All rights reserved.2 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only.

Hyper Scale-Out Platform (HSP)

Optimized for…

Hadoop and Analytics

Scale-Out, Hyper-Converged (compute and storage)

Private Cloud ServicesDevOps Approach

Compatible APIs

NFS

© Hitachi Data Systems Corporation 2015. All rights reserved.3 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only. © Hitachi Data Systems Corporation 2015. All rights reserved.

HSP key advantages and benefits

Dramatically Reduce

Setup Time

Data-in-Place Processing

Better workflow =Accelerate Time to Results

Speed Data Ingest

Automated and Self-Managed

Run Virtualized Apps at the Data Source

APIs

© Hitachi Data Systems Corporation 2015. All rights reserved.4

Elastic Scaling of Multiple Applications

KVMKVMKVMKVMKVMKVMKVMKVM

Elastic VM management – persistent data

HS

P

“Active” Data Lake – Global Namespace

Data-in-place processing for

fast results

Efficiently run analytics apps

at the data source

Scale-out pay-as-you-grow

“pop-up” VMs and storage

Hyper Scale-Out Platform

Manage in open cloud

environments

Enterprise Scalable File System

API

© Hitachi Data Systems Corporation 2015. All rights reserved.5 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only. © Hitachi Data Systems Corporation 2015. All rights reserved.

12 cores, 192 GB memory

8 GB NVRAM

72 TB storage (12 x 6 TB HDD)

2x 40GbE internal fabric

10 GbE external interface

20 nodes per rack‒ 240 cores, 3.8TB memory‒ 1.44 PB Raw

Extend cluster to multiple racks

HSP-400 System

Turn Key, Highly Scalable

Scale Out

© Hitachi Data Systems Corporation 2015. All rights reserved.6 © Hitachi Data Systems Corporation 2015. All rights reserved.6

Use Cases

© Hitachi Data Systems Corporation 2015. All rights reserved.7

Analytic Flow vs. Data Flow Leveraging Hadoop and Pentaho

PDI

BA

Data Source Connectors

Dashboards

PDI

Other Use ofResults

PDI

Data Pre-processingOr Blending step

Map/Reduce Process

Elastic Search

No SQL DB

PDI

Another Analytic Step

Impact of Data Transfers

Data Collector

HSP Value: Simplify workflow & management

HSP Data Lake

Avoiding Many Data Transfers

© Hitachi Data Systems Corporation 2015. All rights reserved.8 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only. © Hitachi Data Systems Corporation 2015. All rights reserved.

HSP Storage and Compute Platform

Equipment UsageFile System

Billing File System

MaintenanceFile System

Converged Protocols: Streaming, NFS, Swift, …

Converged Data PlatformIndustry Control: Equipment Health, Logistics & Scheduling

Converged storage and compute: spin-up/spin-down VMs on demand Converged workflows: REST APIs and OpenStack integration

Streaming Time Series Numeric

NFS Semi-Structured Text

SWIFT

Unstructured Text

VMs for Streaming

Health Checks

VMVMVMVMVMs

Running Hadoop

with Lucene and Solr

VMVMVMVM

Native or OpenStackManagement Client

VMs Running Hadoop

with HBASE

VMVMVMVM

© Hitachi Data Systems Corporation 2015. All rights reserved.9 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only.

Smart Grids

Equipment health

Revisit Drill Sites‒ Analysis

‒ validation

New Analytics‒ ISV Partner app

Energy Analytics Power and Energy

© Hitachi Data Systems Corporation 2015. All rights reserved.10 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only.

Automotive Purchasing Analytics‒ Dealer Customer 360 analysis

‒ Merge Credit, Social Media, Existing CRM

Customer 360 Analysis

Best Candidate

HSP Advantage- Parallel ingest- Simplified workflow- Simplified managementl

© Hitachi Data Systems Corporation 2015. All rights reserved.11 © Hitachi Data Systems Corporation 2015. All rights reserved.11

DevOps &Cloud Services

© Hitachi Data Systems Corporation 2015. All rights reserved.12 CONFIDENTIAL – For use by Hitachi Data Systems employees and other audiences under NDA only. © Hitachi Data Systems Corporation 2015. All rights reserved.

HSP Cloud Service with OpenStack DevOps and REST management

HSP Storage (scale-out)

NFS Protocol HSP Mgmt. API

- Cluster- File System- KVM

eScaleFS (Hadoop)

Glance(VM images)

Nova(VM deployment)

SwiftClient

Mgmt. Client<Horizon> (Orchestration)

Native HSPComponents

OpenStack and HSP

Components

OpenStackClients

Swift(Object)

© Hitachi Data Systems Corporation 2015. All rights reserved.13 © Hitachi Data Systems Corporation 2015. All rights reserved.13

Thank You