Cloud-based IT Log Analytics

Christian BeedgenKumar Saurabh

Agenda

Overview

Team

Market Size

Problem Statement

The Next Generation

Differentiators

Competition

Go To Market

Economics

Roadmap

Summary

OverviewCloud-based IT Log Analytics

Service to manage and analyze IT logs

$2.5 Billion market size

Current products have high TCO, are services-heavy

Easy to get started, lower TCO, superior intelligence

Team of log management veterans, to be completed

Series A – customer-focused development process

Team

Christian BeedgenArcSight since 2001, Chief Architect, Director of EngineeringLead ESM server developerBuilt ESM server team, managing 20 people in server and UI teamsNamed on 2 granted patents, 7 patent applications in processPast experience at Amazon, Gigaton, Cleverlearn

Kumar SaurabhData Architect at Mint.comSingle handedly built Mint’s data analysis infrastructureArcSight 2001-2008, Director of Engineering, managing 12 peopleLead for Analytics and Solutions TeamNamed on 2 granted patents, 2 patent applications in process

Tivoli, BMC, CA, HP, Microsoft, Quest

Source : Gartner/Dataquest

Key DriversCompliance is not optional

April 2009 “Market Overview: Security Information Management (SIM)”

Problem Statement

Premise-basedEnterprise sales cycles, installation and upgrade hassles, expensive hardware, DBAs, sysadmins required

Not scalableNot inherently clustered, scaling introduces tradeoffs and data fragmentation

Challenged with log parsingEither simply don’t parse or require parsing at collection time, need constant software upgrades

Not context-awareIdentities, network assets, service dependencies are all critical for correlation and prioritization

Customers operate in silosInsight gathered by one customer is hard to share; no cross-customer data mining

Not community-awareExchanging of solutions is a manual process, there’s no marketplace

Today’s market leading products are:

The Next Generation

Cloud-based serviceEasy sale, quick delivery, ongoing upgrades, no care and feeding

Seamless scalabilityBuilt from scratch for big data, leverages large-scale processing

Machine-driven log parsingExtracting structure from raw logs is foundation for analytics

1

2

3

4

5

6

Context modelingLogs need to be analyzed in their real world environment

Global IT log intelligenceData mining leads to insight shareable across all customers

Built-in communityNot everybody is an expert, and even experts exchange findings

Deliver superior log management for compliance, security and operations in a

scalable, easy-to-adopt cloud-based service

Target Market

CompliancePCI, SOX,

HIPAA, NERC

Log Retention & Review

User & Resource Access

Security

Incident Response

Data Protection

Threat Intelligence

Operations

Troubleshooting

Business Continuity

Service Levels

Use Cases

Medium Enterprises Large Enterprise Departments Large Enterprises

OperationsSecurityCompliance

IT Logs

High-level Solutions Architecture

Router/SwitchFirewall/Proxy

IDS/IPS

Context

Active DirectoryVulnerability Scans

Custom Source

Global IT Log Intelligence, Community

PCI, SOX, HIPAA, NERCLog Retention, ReviewUser, Resource Access

Threat AnalysisIncident Response

Data Protection

TroubleshootingBusiness Continuity

Service Levels

Collect à Normalize à Correlate à Context à Business Impact

NetworkOS Logs

File AccessVirtualization

Web ServerDatabase

Custom App

Systems Applications

High-level Platform Architecture

Customer A

Collector

Customer B

Collector

Customer C

Collector

Raw Log Store

Data Processing ContextFull Text Search

Log Parsing

Global IT Log Intelligence Community

Customer C Apps in Cloud

Collector

SaaSAccess Logs

Collector

5 6

24

1

3

AnalyticsWorkflow

Compressed log storeFull-text search

CollectionLog parsing

Log Management Architecture Today

Database Log Store

Correlation Engine

Collector Raw Data

Parser creation and maintenance is manual and expensive

Limited vendor support for custom log sources like applications

Data is often not parsed, or parsed incorrectly; custom applications ignored

Log Management Architecture TodayLog parsing challenges

Database Log Store

Correlation Engine

Collector Raw Data

Efficient parser development driven by structure inference

Customer can add knowledge about custom sources like applications

Better parser support enables more log sources, which enables superior analytics

Next Generation ArchitectureMachine-driven Log Parsing

Collector Collector Raw Data

Parsing3

1

Database hardware is expensive, performance is still a major problem

Most data in the Log Store is not available for analytics

Performance or intelligence –choose one

Log Management Architecture TodayScalability tradeoffs and data fragmentation

Database Log Store

Correlation Engine

Collector Raw Data

Distributed store is built for scale, no excuses

Elastic compute layer can tackle any analytics efficiently

Better intelligence and faster response times

Next Generation ArchitectureSeamless scalability

Collector Collector Raw Data

Distributed Store

Elastic Compute

Parsing

2

Advanced analytics cannot run due to database limits

Customers don’t always have analysts on staff, vendors don’t have the data

Incidents that could have been detected with better analytics are missed

Log Management Architecture TodayCustomers operate in silos

Database Log Store

Correlation Engine

Collector Raw Data

Makes it feasible to analyze all data from every customer

Enables deep analytics – root cause detection, network graph analysis, anomaly detection

Extracts and delivers actionable insights

Next Generation Architecture

Collector Collector Raw Data

Distributed Store

Elastic Compute

ReportingData Mining Alerting

Parsing

Global Log Intelligence

5Community

4

6

2

Differentiator Customer Benefit

Cloud-based serviceNo enterprise software release cycles

Quick initial service deliverySeamless ongoing upgradesNo deployment or sizing hassles

Seamless, transparent scalabilityHeadroom to deal with all the data

Instant slice and dice analyticsNo data fragmentation100 EPS today, 10,000 EPS tomorrow

Intelligently evolving log parsingAutomated structure inference

Custom app logs useful right awayStructured data beats unstructured dataOnce seen, available everywhere

Context modelingIdentities, network elements, services

Real world environment context for logsBusiness impact correlationEnables risk modeling

Global IT log intelligencePattern discovery

Solution recommendation systemZero day discovery of emerging threatsDefense for all customers

Built-in communityFrictionless sharing

Expert exchangeSharing of analytics contentService à Platform

1

2

3

4

5

6

Competition

Enterprise software & services, appliances

RDBMS limitations, data fragmentation

One schema, parse at collection, limited

SDK

Limited context model, only network

No intelligence shared among

customers

Informal community

Appliance-based

RDBMS limitations, data fragmentation

Schema per device, limited device

support

No context model

No intelligence shared among

customers

Informal community

Downloadable software

Full-text index, cost of repeated parsing

Parsing on access via regular expressions

No context model

No intelligence shared among

customers

Informal community

SaaS, Appliance

Storage backend tradeoffs unknown

Schema per device

Limited context model, only network

No shared intelligence, some

SOC service

Informal Community

Cloud-basedLog Analytics

Cloud-based service

Seamless, transparent scalability

Intelligently evolving log parsing

User, network, service model, extensible

Global IT log intelligence

Community baked into the service

Go To Market

Self-serve, easy to try, buy and useInstant gratification

Free trials, freemium model also possibleValue-before-commit

Tiered pricingPay for what you use

Web sales, telesales, focused direct touch, channelLower cost of sales

Leveraged partnershipsPaaS add-on sale

EconomicsDeliver service at high gross margin

Tier Events/Sec GB/Day COGS$/Month

MRR* ACV* AlertLogicACV

ArcSightDeal Size

Trial 5 0.5 $5 $2,148

Silver 120 10 $101 $506 $6,072 $36,000 $100-500kGold 1,200 100 $1,012 $5,063 $60,756 $153,000 $0.5-2MPlatinum 3,000 250 $2,531 $12,656 $151,872 $324,000 $2-6MDiamond 12,000 1000 $10,125 $50,625 $607,500

Network, Storage, CPUInbound network traffic dominates outbound trafficStorage needs to consider monthly charge due to retentionCPU can be optimized because of elasticity

*Priced at 80% gross margin

Release 2 Release 3Release 19 – 12 Months

Operations

Intelligence

Security

Compliance

Platform Collection, Search,Reporting, Assets

PCI Pack

Threat Analysis

Workflow, Dashboards, Trending,

Identities

SOX, HIPAA, NERC

Incident Response

Troubleshooting Business Continuity

Context Modeling, Community

Fraud, Risk

Data Protection

Service Levels

Predictive AnalyticsPattern Mining

Roadmap

Anomaly Detection

SummaryCloud-based IT Log Analytics

Large opportunity in growing enterprise market

Team of veteran log management experts

Game-changing functionality and ease of adoption