Practical examples of Big Data, security analytics and visualizationh41382. · 2014-09-12 ·...

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Practical examples of Big Data, security analytics and visualization Jeff McGee, Data Scientist Josh Stevens, Enterprise Security Architect

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3

Objective

• Identify problems in Security that could be solved with better analytics

• Discuss recent efforts on Big Data and Visualization

• Share examples of how HP’s Cyber Defense Center has leveraged these capabilities


Big Data and the data overload


Good guys are making things less predictable

Challenge: There is more noise

• Mobile • Bring your own device • Virtual machines and “the Cloud” • SAAS • New sources of logs • HIPPA, SOX, PCI…


And bad guys know how to stay inside the bell curve. Challenge: There is less signal

Unknown: Harder to detect

• New behavior • Goes to an approved place • Works encrypted • Authorized Use • Inside of baseline • Outside monitored infrastructure

• Matches a signature • Goes to a bad place • Works in the clear • Unauthorized Use • Outside of baseline • Within monitored infrastructure

Known: Easier to detect


Solutions to big data problems

Let’s take techniques originally built for other domains and apply them to security: • Map-reduce • Columnar Data Stores • Machine Learning • Visualization Tools


Tools and technologies

• Hadoop – Framework for distributed computing • Vertica – Columnar database • Tableau – Visualization software • Numpy/Scikit-learn – Machine learning tools

ArcSight

Vertica analytic platform

Hunt teams

Security intelligence


Overview: The Vertica analytic platform

Rapid iterative conversations with your data

Real time Analytics

Purpose built for Big Data from the first line of code

Store & Analyze PBs Ingest 30 TB/hour

Proven Scalability Works with Hadoop, R Ecosystem of Visualization Tools, SDKs and Community

Open & Extensible Efficient compressed storage Scale-out architecture Easy to setup & manage

Low TCO

Private Cloud Public Cloud Appliance Software Only

Flexible to deploy


Security visualization Practical examples

Router

Operating System/Informational

Network-basedIDS/IPS

/Suspicious

Network-basedIDS/IPS/Recon

Network-based IDS/IPS/Informational


/Compromise

Network Monitoring/Informational

Network Monitoring

Firewall

Firewall/Normal

Firewall

Firewall

Content Security

Applications/InformationalApplications

bust6Category Device Type

ApplicationsContent Security

Firewall

Host-based IDS/IPS

Mainframe

Network Monitoring

Network-based IDS/IPS

Operating SystemRouter

VPN


Security management


Category Significance

Informational/Error /Hostile /Compromise /Suspicious /Normal /Informational/Warning /Recon /Informational0M

10M

20M

30M

40M

50M

60M

70M

80M

90M

100M

Co


Network-based IDS/IPS Network-based IDS/IPS



Host-based IDS/IPS

Network Monitoring

Operating System

Applications

Applications

Applications

Firewall

Firewall

Firewall

VPN

fromAfarSourcePtCount of Destination Port

0

20,000,000

40,000,000

66,854,010

0 66,854,010

Count of Destination Port

View at a glance


Router

Operating System/Informational


/Suspicious

Network-basedIDS/IPS/Recon

Network-based IDS/IPS/Informational


/Compromise

Network Monitoring/Informational

Network Monitoring

Firewall

Firewall/Normal

Firewall

Firewall

Content Security

Applications/InformationalApplications

bust6Category Device Type


Firewall

Host-based IDS/IPS

Mainframe

Network Monitoring


Operating SystemRouter

VPN

Proportional relationships


Security analysts


Category Device Type

Applications ContentSecurity

Database Firewall Host-basedIDS/IPS

Mainframe NetworkMonitoring

Network-based ID..

OperatingSystem

Policy Management

Router SecurityMangement

VPN

0M

100M

200M

300M

400M

500M

600M

700M

Cou

DeviceSeveritybyDevice

Count of Device Severity for each Category Device Type. The view is filtered on Category Device Type, which keeps 13 of 20 members.

Starting points


Category Outcome / Category Significance/Attempt /Failure /Success

/Comp /Informational

/Informational/Error

/Normal /Recon

/Suspicious

Null

/Compromise /Informational

/Informational/Alert /Informational/Error/Informational/Warning

Null



/Normal /Recon

/Suspicious

0M

50M

100M

150M

200M

250M

300M

350M

Cou

destHostnameAttemptFailSuccess

Count of Destination Host Name for each Category Significance broken down by Category Outcome. The data is filtered on Destination Host Name, which excludes Null. The view is filtered on Exclusions (Category Outcome,Category Significance) and Category Outcome. The Exclusions (Category Outcome,Category Significance) filter keeps 35 members. TheCategory Outcome filter excludes Failure.

Carving in


Trending attempts


Category Outcome / Category Significance/Attempt /Failure /Success

/Comp /Informational

/Informational/Error

/Normal /Recon

/Suspicious

Null



Null



/Normal /Recon

/Suspicious

0M

50M

100M

150M

200M

250M

300M

350M

Cou

destHostnameAttemptFailSuccess

Count of Destination Host Name for each Category Significance broken down by Category Outcome. The data is filtered on Destination Host Name, which excludes Null. The view is filtered on Exclusions (Category Outcome,Category Significance) and Category Outcome. The Exclusions (Category Outcome,Category Significance) filter keeps 35 members. TheCategory Outcome filter excludes Failure.

Successes


Trending success by hostname


Actual hostnames


Keep in mind this is demo data – however a quick internet search shows this domain has a reputation as a bullet proof server, delivering malware. Our visualization shows us it’s been accessed every day for the last 30 days.


Bullet proof servers

White spac e


Hunt teams Use case 1


Device Receipt Time

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

Cat Network

0K

10..

20..

30..

40..

50..

60..

70..

80..

90..

count of device severity 30 dall all scatter Device Severity

High

Medium

Unknown

Very-High

60 days IPS data


Jun 3 Jun 8 Jun 13 Jun 18 Jun 23 Jun 28 Jul 3 Jul 8 Jul 13 Jul 18Hour of Device Receipt Time [2014]

Cat Network

0K

20K

40K

60K

80K

100K

120K

140K

Co

IPS EventsDevice Severity

High

Medium

Unknown

Very-High

30 days


By technique


fromVictim

Jun 7 Jun 12 Jun 17 Jun 22 Jun 27 Jul 2 Jul 7 Jul 12Minute of Device Receipt Time [2014]

Network

0

100

200

300

400

500

600

Cou

tenacleCategory Technique

/Exploit/Vulnerability

/Policy/Breach/Traffic Anomaly/Network Layer

/Traffic Anomaly/Network Layer/Flow

/Traffic Anomaly/Network Layer/IP Fragments

Aggregate from victim a




Sonar


Source addresses

Dest

inat

ions

Sonar trend




VPN


Network Monitoring

Host-based IDS/IPS

FirewallApplications

fromAfar2infoOnlyCategory Device Type


Database

Firewall

Host-based IDS/IPS

Mainframe

Network Monitoring


Operating System

Policy Management

Security MangementVPN

Bottom of the stack “Informational”


VPN logging


Who’s scanning via VPN?


For more information

After the event

• Contact your sales rep • Visit the HP Security Product Blog:

hp.com/go/securityproductsblog


Please fill out a survey. Hand it to the door monitor on your way out.

Thank you for providing your feedback, which helps us enhance content for future events.

Session TB3273 Speaker Joshua Stevens, Jeff McGee

Please give me your feedback


Thank you

Date post:	08-Jun-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Practical examples of Big Data, security analytics and visualizationh41382. · 2014-09-12 ·...

Documents