Big Data From Microsoft and Atidan

transcript

BIG DATAAudio / Video

Log Files

Text/Image

Social Sentiment

Data Market Feeds

eGov Feeds

Weather

Wikis / Blogs

Click StreamSensors / RFID / Devices

Spatial & GPS CoordinatesMobile WEB 2.0

Advertising

Collaboration

eCommerceDigital Marketing

Search Marketing

Web Logs

Recommendations

Contacts

Deal Tracking

Sales Pipeline

ERP / CRMPayables

Payroll

Inventory

Data Complexity: Variety and Velocity

Terabytes

Gigabytes

Megabytes

Petabytes

How do I optimize my

fleet based on weather

and traffic patterns?

What’s the social

sentiment for my

brand or products

How do I better

predict future

outcomes?

Increases ad revenue by processing 3.5 billion events per day

Massive Volumes

Processes 464 billion rows per quarter, with average query time under 10 secs.

Measures and ranks online user influence by processing 3 billion signals per day

Cloud Connectivity

Connects across 15 social networks via the cloud for data and API access

Uses sentiment analysis and web analytics for its internal cloud

Real-Time Insight

Improves operational decision making for IT managers and users

MANAGE ANY DATA, ANY SIZE, ANYWHERE

010101010101010101101010101010101001010101010101101010101010

Extremely large volume of unstructured web logs

Ad hoc analysis of logs to prototype patterns

Hadoop data cluster feeds large 24TB cube

Business users analyze cube data

E.g. STRUCTURED & UNSTRUCTURED DATA

OPEN & FLEXIBLE

100% compatible with

Apache Hadoop

Accelerating the delivery

of Hadoop for Windows

Tools from a rich

ecosystem of partners

Built with close

community collaboration

Hadoop for Windows

JavaScript libraries

Hive ODBC drivers

The Apache Software Foundation

ENRICH BY CONNECTING TO THE WORLDS DATA

Discover

Combine

Refine

DISCOVER DATA

TOIDENTITY

DOC CONTEXT

SOCIAL GRAPHS

DATA EXPLORER

DATA HUB

POWER OF COMBINING THE WORLDS DATA

REFINE DATA

Enterprise Information Management & Full Analytic Spectrum

E.g. VALUE OF EXTERNAL DATA

INSIGHTS ON ANY DATA, ALL USERS, WHEREVER THEY ARE

010101010101010101101010101010101001010101010101101010101010

DEMO: FROM DATA TO INSIGHTS!

INSIGHTS FOR ALL USERS THROUGH FAMILIAR TOOLS

PB TB GB

BIG DATA REQUIRES AN END-TO-END APPROACH

INSIGHTS

SELF-SERVICE | COLLABORATIVE | MOBILE | REAL-TIME

NON-RELATIONAL

100111

DATA MANAGEMENT

RELATIONAL STREAMING

AND GOVERN

DISCOVER

AND RECOMMEND

TRANSFORM

AND CLEAN

DATA ENRICHMENT

Parallel Data Warehouse

PowerPivot

Power ViewHadoop on Windows

ADDITIONAL RESOURCES

LEARN MORE1. Microsoft Big Data Solution: www.microsoft.com/bigdata

2. Windows Azure: www.windowsazure.com/en-us/home/scenarios/big-data

3. Microsoft BI blog: http://blogs.msdn.com/b/microsoft_business_intelligence1/

TRY NOW1. Preview of the Hadoop-based service for Windows Azure:

https://www.hadooponazure.com

1. McKinsey&Company, McKinsey Global Survey Results, Minding Your Digital Business, 20122. IDC Market Analysis, Worldwide Big Data Technology and Services 2012–2015 Forecast , 2012

49% of top CEOs and CIOs are

currently using Big Data for

customer analytics

2012 2013 2014 2015

39% compound

annual growth

Big Data Software Growth

1.82.5

2012 2013 2014 2015B

s $ 34% compound

annual growth

Discover data with Data Explorer

Combine with information from

other sources via Azure Marketplace

Refine with Advanced Analytics

Connecting

with the World’s Data

MICROSOFT BIG DATA

Immersive insights for all users

Insights on any data

Embedded insights with simplified

programming

Immersive Insight,

Wherever you are

Extend data warehouse with Hadoop

Windows simplicity for Hadoop

Scale & elasticity of the cloud

Any Data, Any Size

Anywhere

Discover data with Data Explorer

Combine with information from

other sources via Azure Marketplace

Refine with Advanced Analytics

Connecting

with the World’s Data

MICROSOFT BIG DATA

Immersive insights for all users

Insights on any data

Embedded insights with simplified

programming

Immersive Insight,

Wherever you are

Extend data warehouse with Hadoop

Windows simplicity for Hadoop

Scale & elasticity of the cloud

Any Data, Any Size

Anywhere

Parallel Data Warehouse

PowerPivot

Power View

TRADITIONALRelational Database Management System

NEWPetabyte-Scale Services

Industry/Vertical Scenarios

Financial Services

Modeling True Risk

Threat Analysis

Fraud Detection

Trade Surveillance

Credit scoring and analysis

Web & E-Tailing

Recommendation Engines

Ad Targeting

Search Quality

Abuse and click fraud detection

Retail Point of Sales Transaction Analysis

Customer Churn Analysis

Sentiment Analysis

Telecommunications

Customer Churn Prevention

Network Performance optimization

Call Detail Record (CDR) Analysis

Analyzing Network to Predict Failure

Government Fraud Detection and Cyber Security

General (Cross Vertical) ETL & Processing Engine

WHILE DRAMATICALLY SIMPLIFYING PROGRAMMING ON HADOOP

Integration with .NET and new

JavaScript libraries for Hadoop

MapReduce

programs in

JavaScript

Simplified

programming

Simplified deployment of

MapReduce jobs

Benefit

Deploy JavaScript Hadoop jobs

from a simple web browser on

any supported device

HADOOP ON PREMISES AND IN THE CLOUD

Enterprise-class Big Data

platform on-premises

Hadoop-based distribution

on Windows Server

Elastic Big Data platform

in the cloud

Hadoop-based Service on

Windows Azure platform

Hadoop connectors for

SQL Server

Extend your EDW

with Big Data

MANAGE ANY DATA, ANY SIZE ANYWHERE

101010101010101001010101010101

101010101010

Hadoop Connectors & ETL

SIMPLICITY AND MANAGEABILITY OF WINDOWS FOR HADOOP

Integration with Microsoft

System Center

Simplified management Enterprise-class security

Integration with Windows

Server® Active Directory

Easy setup on-premises

and in the cloud

Hadoop-based service on

Windows Azure

Smart packaging of

Hadoop on Windows

STREAMING DATA WITH STREAMINSIGHT

Complex Event Processing with

SteamInsight (On-premise)

On-premises analysis of streaming

data in real time

Event Processing in the Cloud with

Windows Azure SQL StreamInsight

Cloud designed analysis of streaming

StreamInsight SQL StreamInsight

EXTEND YOUR DATA WAREHOUSE WITH HADOOP

Integration with

enterprise BI solutions

Microsoft SQL Server

connector for Apache Hadoop

with SQOOP (SQL to Hadoop)

Integration with Microsoft

Data Warehousing

SQL Server Parallel Data

Warehouse connector for

Apache Hadoop with

Deeper insights from structured

and unstructured data

CONNECT HADOOP TO THE WORLD VIA WINDOWS AZURE MARKETPLACE

Mashing up of internal and

public data sets

Integration with third-party

data and services

Sharing of data and

insights through Windows

Azure Marketplace

Integration with Windows

Azure Marketplace

through ODATA

ENRICHMENT VIA INTEGRATION WITH SOCIAL MEDIA

Integration of social

media data with

business applications

Microsoft Codename

"Social Analytics"

Stronger customer

relationships

Integration with social

media sites

Models augmented with

publicly available data

from social media sites

ADVANCED ANALYSIS WITH HADOOP

Unlock rare patterns from bespoke data

mining models

Mahout & Pegasus libraries

already supported on

New business insights with predictive

analytics from Microsoft

Hive ODBC Driver connects Hadoop to

SQL Server Data Mining tools in SSAS

Support for open source

Advanced Analytics tools

such as Mahout & Pegasus

MICROSOFT ENTERPRISE DATA WAREHOUSING

Software AppliancesReference Architectures

Fast Track for

Dell Parallel Data Warehouse

HP Enterprise Data Warehouse

Dell QuickstartData Warehouse

HP Business Data Warehouse