Date post: | 15-May-2015 |
Category: |
Technology |
Upload: | sridhar-karnam |
View: | 443 times |
Download: | 2 times |
Sponsored byTop 5 Truths about Big Data Hype and Security
Intelligence
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2
Thanks toMade possible by
www.hpenterprisesecurity.comSRIDHAR KARNAMHP ArcSight Product Marketing
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3
Preview of Key Points
1. There’s More to Big Data than “Big” 2. The Real-Time Requirement for Big Data Security Analytics 3. There’s More to Big Data Security Analytics than Big Data
Technology 4. The Trap of Data Silos within Big Data Repositories 5. The 3 Vs of Big Data Aren’t New to Enterprise SIEM
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4
1There’s More to Big Data than “Big”
Volume is only one dimension of “big”
Record quantity better metric than byte
About analysis or lots of information
BigDataIs.. Data
Science
Data Volume
Data Variety
Data Velocity
Put all data together; find relationships we didn’t know existed
Variety – total record types
Big data even with small volume
Velocity usually considered rate of new data to be stored
Not analyzed
But BDSA has a bigger velocity issue
The type of questions being asked and the analytical techniques being used to answer them is what distinguishes Big Data from traditional data
Cluster analysis Topological data analysis Machine learning Multi-linear subspace learning
Data visualization
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.5
2The Real-Time Requirement for Big Data Security Analytics
Big Data Security Analytics (BDSA) is a specialized application of the more general concept of Big Data. Most Big Data scenariosHigh velocity data aquisition
Human driven analysis
Long shelf life for conclusions drawn
3 types of velocityInsertion or append speed into Big Data repository
Processing speed for queries upon data rest
Analysis of events in real time
Human driven analysis has a place in BDSAImmediate tactical investigations in response to warning signs detected by automated correlation enginesForensic investigations Strategic research to tease out indicators of long-term, ongoing attacks
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6
2The Real-Time Requirement for Big Data Security Analytics
But what about tactical, second-to-second monitoring?Core of security operation center workAnalysis must be done automatically and in a streaming fashion
Current Big Data toolsRun a query, analyze results, tweak query, analyze results, repeatNot a streaming scenario in which a constantly updated tactical situation is plotted
But real-time analytics require a purpose-built correlation engine Enterprise SIEM correlation enginesDesigned to handle a constant stream in real timeMaintain in memory a massive amount of partial pattern match objectsChurning in and out of existence at a fantastic rate
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7
2The Real-Time Requirement for Big Data Security Analytics
SIEM Real-Time Correlation
Big DataBatch Analytics
Trigger for tactical investigations
Event feed
Context Criteria for better
correlation rules
Wide and deep trolling to identify
ongoing attacks too low and slow to
trigger SIEM alerts
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8
3There’s More to Big Data Security Analytics than Big Data Technology
BDSA requires 3 kinds of advanced skills
Big data platform
technology
Still more of a concept and developer-level movement than a mature technology
platform with available off-the-shelf solutions
Data science
To make any sense of Big Data, analysts using Big Data farms need to
know how to use advanced analytics
Information
securityTo detect cyber-attacks and internal malicious
agents, analysts need to be more than data
scientists
Must also be technical information security professionals that
understand the organization’s IT infrastructure.
Network security, host security, data protection,
security event interpretation, and
attack vectors
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.9
3There’s More to Big Data Security Analytics than Big Data Technology
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10
4The Trap of Data Silos within Big Data Repositories Point Solution for
Monitoring Application B
Point Solution for Monitoring
Application B
Point Solution for Monitoring
Application B
Big Data Repository
ApplicationA
ApplicationB
ApplicationC
ApplicationA
ApplicationB
ApplicationC
Even after migrating from point solutions to Big Data, the same silos can persist.
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11
4The Trap of Data Silos within Big Data Repositories
Example: consider usernames and email addressesIf you are trying to track a user’s actions and communications through a variety of data, you must be cognizant of the fact that a given email address, such as [email protected], could be one of the following:
Email sender Email recipient Actor in a audit log event (e.g., jsmith opened a file)Object of an action in an audit log event (e.g., Bob changed jsmith’s reset password)Subject of a memo
Simply querying certain data can lead to extremely inaccurate results unless one of the following occurs:The analyst filters the results manually after the queryThe analyst builds knowledge into the query about the structure or format of the various data queried to do the filteringThe system understands the various formats and does the filtering automatically
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12
4The Trap of Data Silos within Big Data Repositories
Silos in Big Data is failure to deal with variety Being able to store all types of data and query it for keyword occurrences does not satisfy BDSA requirements. Some enterprise SIEMs takes a more effective and pragmatic approach that embraces data variety Normalizing security events into a common event format Integrate non-event data sets into the correlation and analytics process. Directory informationIP reputation listsGeolocation datasocial network feeds
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.13
5The 3 Vs of Big Data Aren’t New to Enterprise SIEM
Big data architectureEnterprise SIEMs abandoned relational databases a long time agoProprietary correlation and storage engines• Allow rapid storage and query of massive amounts of event data
Real-time situational awarenessReal-time analysis is a manifest requirement of security analyticsEnterprise SIEMs analyze data as it arrivesCombines • real-time, in-memory, event-log data • asset awareness, asset vulnerability• identity correlation
Prioritize critical events and correlations to assist operating teams with immediate detection of threats
No data scientists required No silos
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14
Bottom line
Hidden skill requirement of BDSA: data scientists Real-time requirement for security intelligence, often misunderstood in relation to Big DataRisk of data silos persisting in Big Data repositoriesInvesting in a Big Data cluster that runs search and a schemaless database is only the beginning of building a BDSA practiceAn enterprise SIEM like HP ArcSight provides BDSA that is specialized for event data
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How HP Solves
Big Data Security Analytics Problem? • With CORR• With Hadoop• With Autonomy• With HAVEn• Why HP?
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16
Big data opportunities – won and lost
% of the Digital Universe that actually is being tagged and
analyzed
Competitive Advantage in the Digital UniverseMassive amounts of useful data are getting lost23%
3%% of data that would be
potentially useful IF tagged and
analyzed
% actually being tagged for Big Data Value (will grow to 33% by 2020)
0.5%¹Source: IDC The Digital Universe in 2020, December 2012
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17
Collect, normalize, and categorize machine data such as logs, events, and flows from any device, any time, anywhere from any vendor
Collect & correlate up to 100,000 events per second from 350+ connectors
Collect, store, correlate, and analyze big data across IT
HP ArcSight Universal log management platform
High-performance universal log management to consolidate machine data across IT
HP ArcSight
The unified machine data through filtering and parsing is enriched with rich metadata, which allows you to search machine data through simple text-based keywords without the need of domain expertise
Search over 2,000,000 events per second
The unified data is stored through high compression ratio in any of your existing storage formats, eliminating the need for expensive databases and DBAs
Store years’ worth of data
Built-in content packs, algorithms, rules, and the unified machine data help you deploy IT security, IT operations, IT GRC, and log analytics
Analytics & intelligence
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.18
Volume
VarietyVelocity
Complexity
ArcSight CORRe for Big Data SecurityArcSight has been dealing with Big Data since 2007 with CORR engineVolume• Cross-device, real-time correlation of data across IT• Long term archival at 10:1 compression ratio with
ArcSight• Send it to Hadoop at over 100,000 EPS
Velocity• SmartConnectors collects logs, events, flows at over
100,000 EPS from almost any log generating source• Search data at over 2,000,000 EPS
Variety• Collects machine generated data from 350+ distinct
sources• Autonomy collects human generated data from 400+
distinct sources• Collect from Hybrid network such as physical, virtual,
and cloud
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.19
Success StoriesBeyond theory to practice:
U.S. Department of Health and Human Services “HP solutions have helped us transform from a reactive to a proactive IT Operations function, and to align our priorities to match the business and drive business value, delivering 300% ROI in one year.” - Dan Galik, CISO
Heartland Payment Systems
“ ArcSight solution will give us a more comprehensive threat and risk management platform that optimally enables enterprise-wide visibility to identify illegal activity in progress and take prompt, preemptive action.” - Kris Herrin, CTO
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.20
ArcSight and Hadoop
Security Intelligence
Storage Analytics Live/ Historical data
Hadoop
Live (Real-time, cross-device correlation of security events)
Historical (security intelligence)
ESM/Logger
ESM/Logger
Live (Real-time analytics on unlimited data)
Historical (Security analytics)Hadoop
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21
Sentiment AnalysisArcSight with Autonomy
Meaning based security Predictive security – Moving from proactive securityAnswers critical questions:• Where is our sensitive information? Who has access to
it?• Which systems store sensitive information?• Do we have the right controls in place to
protect sensitive information?
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22
HAVEn – big data platform
HAVEn
Social media IT/OT ImagesAudioVideoTransactional
dataMobile Search engineEmail Texts
Catalog massive volumes of distributed data
Hadoop/HDFS
Process and index all information
AutonomyIDOL
Analyze at extreme scale in real-time
Vertica
Collect & unify machine data
Enterprise Security
Powering HP Software+ your apps
nApps
Documents
hp.com/haven
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.23
How we help our customers?
5 minutes to generate IT GRC report Compliance packs generates IT GRC reports that otherwise would take 4 weeks3 days to run an IT auditSearch results yield audit-quality data that otherwise would take 6 weeks
10 minutes to fix an IT incidentFull-text based searching and integration with HP portfolio detects and corrects IT incident that otherwise would take 8 hours
4 hours to respond to a breachQuick forensic tools enable instant response to a data breach that otherwise would take 24 days
2 days to fix a threat vulnerabilityArcSight & TippingPoint solution builds threat immune that otherwise would take 3 weeks
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.24
HP Enterprise Security Momentum
HP Security SaaS
2.5Blines of code under SaaS subscription
HP ESP Customers
900+Customers
All Major BranchesUS Department of Defense
9 out of 10Major banks
9 out of 10Top software companies
10 of 10Top telecoms
35New ProductsReleased
in the last 12 months
HP Security Technology
#1
In all markets we play in#
2
10,000+Manage
d Securit
y Service
s
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
25
More Information:www.hp.com/go/ArcSight