Post on 18-Oct-2014
description
transcript
Raffael Marty, CEO
Cloud and Big Data
iTech Gulf, Kingdom of Bahrain April 10th, 2014
Secur i ty. Analyt ics . Ins ight .2
The Intersection of Three Domains
Big DataCloud
Security
Secur i ty. Analyt ics . Ins ight .3
Why use the Cloud?
• Increased efficiency due to better use of resources (elasticity)
• More predictable cost
• Design with redundancy and failure tolerance needed
• Automation necessary, but that’s a good thing
• Outsource non core capabilities / responsibilities
!
New Use Cases / New Applications
• Enables new business models (pay as you go)
• Access to large compute and storage
Cloud
Secur i ty. Analyt ics . Ins ight .4
• Storage has gotten cheap ($0.01 / GB)
• Access to large compute on demand
—> Use larger data to gain insights
!
!
• Search engines • NoSQL / NewSQL / Key-value stores • Map Reduce [really parallel computing (HPC)]
• On commodity hardware • Bring compute to the data
Cloud has Enabled Big Data
Big Data = New technologies to deal with large amounts of data
Secur i ty. Analyt ics . Ins ight .5
Big Data Has Changed Data Analytics
“memory has become the new hard disk, hard disks are the tapes of
years ago”-- unknown source
Secur i ty. Analyt ics . Ins ight .6
Questions to answer:
• “Show me all documents mentioning ‘pixlcloud’”?
• “Which document contains the most relevant information about ‘bahrain’”?
Big Data Stack - Search
Raw Data (Documents)Storage
Search
Interface REST API Web Interface
Index
Secur i ty. Analyt ics . Ins ight .
7
Questions to answer: “Visualize user activity in clusters based on their behavior.”
Big Data Stack - Large-Scale Processing
Map Reduce
Distributed Filesystem
Impala, Stinger, HawQ
SQL Layer
Analytics
Visualization
0xdata, Revolution
Tableau
Hadoop FileSystem (HDFS)
Secur i ty. Analyt ics . Ins ight .
We can store and process PB of data …
• How to analyze the data? What algorithms, what technology, …
• How to get to insights?
• How to do data science on all of that data?
• Adopting machine learning / data mining to
larger amounts of data is hard
• Setups can get complicated - many components
8
Big Data - Limits
Secur i ty. Analyt ics . Ins ight .9
Data Storage and Access • Isolation management / data multi-tenancy • Data retention issues • Data dispersal and international privacy laws
• EU Data Protection Directive and U.S. Safe Harbor program • Exposure of data to foreign governments and data subpoenas
Processing Infrastructure • Application multi-tenancy • Reliance on hypervisors • Process isolation / Application sandboxes
What Has Changed
Secur i ty. Analyt ics . Ins ight .10
Trusting vendor’s security model
• Obtaining support for investigations
• Inability to respond to audit findings
Risk = (Threat, Vulnerability)
• Hypervisor escaping
• Stored credentials
• Web ubiquity
• Shared resources
• Using external services - Proprietary implementations can’t be examined - Availability of services - Confidentiality of services
• Malicious insiders
• Data storage
Secur i ty. Analyt ics . Ins ight .11
The Good • Cloud homogeneity makes security auditing/testing simpler • Clouds enable automated security management • Redundancy / Disaster Recovery • Distributed denial of service (DDoS) protection
The Bad • Loss of physical control • No more network-based Intrusion Detection • No data leak prevention (DLP) • Little network routing mechanisms • Reliance on third parties
Changes in Security
Big Data For Cyber Security & Intelligence
"There are 1000 ways for someone to steal information. If we knew how, we could prevent it. Visualization helps find that one way.”
• Visualization to gain
insight into big data
• Cloud as an intelligence
and data sharing platform
• Security through insights
• Security as a profit center