Cisco Tetration AnalyticsEnhanced security and operations with real time analytics

Christopher Say (CCIE RS|SP)

Consulting System Engineer

csaychoh@cisco.com

Visibility into traffic path for

every flow in real time

Time-series view of events

for faster diagnostics

Which traffic is going

through which links?

Know your applications:

what is running and

what is critical

Where is congestion, and

which application

flows are affected?

Key performance indicators

across the path

workload <-> fabric

Where are the packet drops

happening? What is the

latency?

Challenges in operating a hybrid data center

© 2018 Cisco and/or its affiliates. All rights reserved.

Rapid application deployment

Continuous development

Application mobility

Microservices

Policy enforcement

Heterogeneous network

Zero-trust security

Policy compliance

Security Challenges in Modern Data Centers

Securing applications has become complex

Applications are driving modern data center infrastructure

© 2018 Cisco and/or its affiliates. All rights reserved.

© 2018 Cisco and/or its affiliates. All rights reserved.

Introducing Tetration

APPLICATION

INSIGHT

FLOW SEARCH

& FORENSICS

SEGMENTATION

& COMPLIANCE

v

Open Access

Web Rest API Event Bus Lab

Billions of EventsMeta-Data generated

from every packet

Software & Network Sensors: See everything

OS SensorWindows

LinuxMid-RangeUniversal

Network SensorCloud-Scale Nexus

Nexus 9000 ‘X’

Data Analytics & Machine Learning Engine

Analytics ClusterAppliance model

On-Premise or Cloud

▸ Ingest

▸ Store

▸ Analyse

▸ Learn

▸ Simulate

▸ Act

© 2018 Cisco and/or its affiliates. All rights reserved.

Opera

tions

Cisco TetrationUse cases

Se

cu

rity

Cisco Tetration™

Visibility and

forensics

Application

insight

Policy

Neighborhood

graphs &

Cloud

Migration

Application

segmentation

Compliance

Policy

simulation

Process

inventory

Cisco TetrationArchitecture overview

Software sensor and

enforcement

Embedded network

sensors(telemetry only)

Analytics engine

Web GUI REST API Event notification Cisco Tetration apps

Third-party

sources(configuration data)

Data collection layer

Access mechanism

Bring your own

data(streaming telemetry)

Cisco Tetration data sources

Main features

Low CPU overhead (SLA enforced)

Low network overhead

New: Enforcement point (software agents)

Highly secure (code signed and authenticated)

Every flow (no sampling) and no payload

*Note: Not all network performance functionality is supported on this switch series

Software sensors

Linux servers(virtual machine and bare metal)

Windows servers(virtual machines and bare metal)

Windows Desktop VM(virtual desktop infrastructure only)

Cisco Nexus 9300 EX*

Cisco Nexus 9300 FX

Network sensors

Next-generation Cisco Nexus® Series Switches

Third-party sources

Asset tagging

Load balancers

IP address

management

CMDB

…

Third-party data sourcesAvailable today

Real-time asset tagging

User-uploaded asset tags

• Discovered inventory

• User-uploaded inventory and metadata (32 arbitrary tags)

• Inventory tracked in real time, along with historical trends

User-uploaded tags

Cisco Tetration Analytics™

sensor feed

Real-time inventory merged with

information with historical trends

Cisco Tetration

Analytics

merge

operation

VMware vCenter

(virtual machine attributes)

AWS attributes

(AWS tags)

Virtual machine attributes and tags

Cisco Tetration

Analytics™

Virtual machine attributes

• Cisco Tetration Analytics can be configured to connect to VMware vCenter and AWS • Virtual machine attributes from vCenter

• Instance tags from AWS

• Can connect to multiple vCenter instances and AWS regions

• Administrator provides necessary parameters to connect to vCenter and AWS

• Only read-only access required

• Information about all virtual machines is extracted

• Queries for updates and changes (default time is 10 seconds; this setting is configurable)

• Uses vCenter and AWS standard APIs

Fabric performance monitoring

Network performance features inDatacenter fabric

• Currently there is very little visibility into data

plane traffic within the fabric, resulting in

visibility and operational gaps

• Cisco Nexus 9000 Series Switches with the

built-in hardware flow cache with Cisco Tetration

platform enables the following Network

Performance features:

• Provide visibility into fabric topology

• Map and trace every flow path on the fabric topology

through switch ports and queues

• Search flows for individual fabric links or queues

• Provide per-link statistics and time series

• Provide per-queue statistics and time series

• Highlight important links for further diagnostics based

on specified performance metrics

Cisco Tetration

Analytics™

Cisco ACI™ Infrastructure using Cisco

Nexus® 9300-FX leaf switches and Cisco

Nexus 9300-FX line cards in spine

• Switches with analytics enabled have a Cisco Tetration™ agent running

• Switch reports its type (leaf or spine) and ports to Tetration

• Switch reports LLDP neighbors to Tetration

• For example, Leaf7 may report following neighbors• P1 connected to (Spine1, P3)

• P2 connected to (Spine2, P3)

• P3 connected to (Host1, mac1)

• Fabric topology is built based on neighbors reported by all the switches on the ports

• Tetration platform also maintains a time-series view of the topology

Network topology discovery

• Time-series hop-by-hop view for traffic flows:

• Forward path

• Reverse path

• Where available, includes ingress port, egress

port, and queue information

• If software sensors are installed and LLDP is

enabled on the host, path information also

includes the workloads

Hop-by-hop view within the fabric

Launch in a topology view

Hop-by-hop view overlay in topology

• Click Fwd or Rev link to navigate to fabric page

• Hover on flow path to view class info and

other details

• Path Only (default): A subset of fabric topology

graph relevant to the flow path is shown

• Show All: Show full network topology with flow

path highlighted

• Partial flow path if any of the fabric links does

not exist in the current topology

• Switch reports latency information for

each flow

• Cisco Tetration™ platform computes and

provides the latency information for each link as

well as across fabric

• Tetration provides forward and reverse

latency information

• Average latency for each flow across each link

is provided by Tetration

• Latency calculation requires PTP clock sync in

the fabric

• Latency resolution is 0.1 microsecond

• Switch uses 16 bits for latency measurements,

which means it wraps around at 6.8 ms

Hop-by-hop latency information

Packet drop indicators

• Switch provides indication of packet drops for a

flow, along with the interface and queue

information

• In a time-series view, Cisco Tetration™ platform

shows the export intervals where packet drops

where reported for the flow

Note: Switch does not provide information about

how many packets where actually dropped within

the export interval.

End-to-end

drops flow—

in each

direction

Fabric link statistics

• Link level statistics in the charts

are bidirectional

• Time-series chart for each link shows:• Transport throughput

• Average latency

• Drop indicators

• Per-class time series aggregates flow metrics

that go through a particular egress queue of the

fabric link

• Time-series information per fabric link for long-

lived flows (if available):• Latency

• Drop indicators

• Fwd/Rev path information to find flows

for a given:

• Fabric link ID

• Switch name

• Port name

• For a given link, we can narrow results by:

• Drops: True/false

• Latency buckets

• Class

Search for flows based on fabric details

Top n charts based on fabric performance

• Highlight top n links by performance metrics:

• Transport throughput: Average aggregation over

selected time range

• Avg Latency: Maximum aggregation over selected

time range

• Drop Indicators: Maximum aggregation over selected

time range

• Histogram chart for distribution of nonzero

metric values:

• Bucket values are percentage of links in the

metric range

• Select an arbitrary range of values to update

highlighted links

Bandwidth with distribution (nonzero values)

Avg Latency distribution (nonzero values)

Drop Indicators distribution (nonzero values)

Performance monitoring using software sensors

• Correlate network traffic to a process

on a server

• For each flow, track the process

response times

• Drill down into flow details to get process

information for forward and reverse direction

(where available)

• Time-series view of the information allows you

to go back in time and analyze the information

Tracking process response times

TCP handshake intervals

• Track processes with longer handshake times:

• Longer duration to establish connections

• Group by TCP handshake interval buckets

• Search for flows with longer handshake

intervals

TCP retransmissions

• Track any TCP retransmissions for the flows

• Determine if the retransmissions are happening

in forward or reverse direction

• Drill down to a single flow to identify

retransmission details:

• Find details about number of packets retransmitted at

any particular time along with direction

• Correlated to identifying broader network or

application bottleneck

TCP window size changes

• Cisco Tetration™ platform tracks the following TCP window parameters:

• Forward and reverse congestion window reduced

• Forward and reverse MSS changed (Boolean)

• Forward and reverse TCP receive window zeroed (Boolean)

• Search based on these parameters to identify specific flows in time-series view

Identifying bottlenecks

Identify where the potential bottleneck could be:

• Network

• Application (consumer or provider)

• Both

Information is correlated based on:• TCP retransmissions

• Window size changes

• Latency and other factors

Cisco Tetration application insight

Application dependency and cluster grouping

Bare-metal, VM,and switchtelemetry

Cisco Tetration

Analytics™ platform

Unsupervised machine learning

Behavior analysis

On-premises and cloud workloads (AWS)

Bare-metal and VM telemetry

VM telemetry (AMI …)

BM VM

BMVM

VM BM

BMVM

BM

VM BM

VMVM

Bare metal and VM

BM VM VM BM

Brownfield

BM VM VM BM

Network-only sensors, host-only sensors, or both (preferred)

BM VM VM VM BM

Cisco Nexus® 9000 Series

What is really running on my network?Cisco Tetration Analytics application insight dependency map

Use Cisco

Tetration Analytics™

outcome to generate

whitelist policies

Security

Dependencies

Application

Service offering

Service

Service category

(Service owner)

Server process inventory

Cisco Tetration: Server process and process hash

• Computed process hash for all the processes

running on the server

• Search based on:• Process

• Process ID

• All servers running a particular process

• Details for long-running processes

• User ID associated with process and process ID

• Use process hash information to search for

suspicious processes against any indicators of

compromise (IOCs)

Cisco Tetration Analytics™

Search for process and process hash

Search for all servers that ran

a certain process

Search for all servers that ran a certain

process binary hash

Search for process command line or binary process hash across all servers

Server process inventory details

Drill down to a specific host to look at the complete process inventory

Process inventory

accessed through

the Process tab

Search for process

within a host

Process details

Neighborhood graphs

Insight-based notification: Neighborhood graphs

Cisco Tetration

Analytics™

Kafka

broker

Northbound

consumers

Northbound

consumers

Message publish

Kafka

Neighborhood

graphs

• Find up to two-hop

communication

neighbors for a selected

workload

• Drill down into details

about communication

between these

neighbors

• View dashboard display

using graph database

• Determine the number

of server hops between

two workloads

• Get out-of-the-box

and customer alerts

through Kafka

Neighborhood graph and summary information

Two-hop communication

summary with network traffic

details

Search for an Inventory

filter, scope, or cluster

Nodes in radial tree are

clickable for exploration

• Determine the number of hops between two

entities in an application

• Quickly identify protocols connecting

those entities

• Drill down to get the communication details

between two entities

• Launch flow search view with relevant filters

Neighborhood graphs: Path view

Neighborhood application: Alerts

Allows users to configure alerts in three scenarios:

• Path between two nodes has decreased below some minimum hop count• Example: “Database should never be directly communicate to Scope X”

• Minimum path between two nodes is above threshold• Example: “Database should not be more than two hops away from Scope Y”

• Path between two nodes must pass through a third node• Example: “Everything between Scope A and Scope B must pass through firewall or VPN”

Bring your own data (BYOD)

Cisco Tetration: Bring your own data

Main features

• Stream any JSON-based telemetry to a data sink

• Support up to 10 simultaneous streaming topics

• Bring up to 5 GB of data per hour per streaming topic

• Analyze and write your results through alerts or UI

Northbound

consumers

Data

sink

Public Cloud

Streaming JSON telemetry

Cisco Tetration: Bring your own data

Data sink: Streaming data Upload batch data

• Securely stream data to Cisco Tetration™

through Kafka

• Ingested data can be written to data lake through

data sink Dumper application• Data sink Dumper application supports only

JSON format

• Producer applications provided on the platform to

work with Cisco Tetration data sink • User application can be built on top of data lake

• Upload data through UI (maximum limit is 10 GB)• Parquet, CSV, and JSON formats only

• Directories can be uploaded as tar.gz and gzip

• Uploaded data will be written to data lake

• Data available to all users under that

specific tenant

Open API

Rest API

• Cisco Tetration

flow search

• Sensor management

Push notification

• Out-of-the-box events

• User-defined events

Cisco Tetration

applications

• Access to data lake

• Write your

own application

Cisco Tetration Analytics: Open API

Northbound

application

Programmatic interface

Rest API

Kafka

broker

Northbound

consumers

Northbound

consumers

Message publish

Cisco

Tetration

Analytics™

platform

Kafka

Cisco Tetration™

applications

Deployment options

Cisco Tetration™ Cloud

• Software deployed in AWS

• Suitable for deployments of

less than 1000 workloads

• AWS instance owned

by customer

Cisco Tetration™ Platform

(large form factor)

• Suitable for deployments of more

than 5,000 workloads

• Built-in redundancy

• Scales to up to 25,000 workloads

Includes:

• 36 x Cisco UCS® C220 servers

• 3 x Cisco Nexus® 9300

platform switches

Cisco Tetration-M (small form

factor)

• Suitable for deployments of less

than 5,000 workloads

Includes:

• 6 x Cisco UCS C220 servers

• 2 x Cisco Nexus 9300

platform switches

Cisco Tetration: Deployment options

AmazonWeb Services

On-premises options Public cloud

Microsoft

Azure

© 2018 Cisco and/or its affiliates. All rights reserved.

Cisco Tetration Analytics: Ecosystem

Service visibility Layer 4-7 services integration

Security orchestration Service assurance

Insight exchange

Cisco TetrationAnalytics™

© 2018 Cisco and/or its affiliates. All rights reserved.

Cisco IT: Business value

70% reduction in cost and time

3600 person hours of skilled staff time

saved for every 100 applications

20-40% reduction in virtual machine

footprint

Traditional Cisco Tetration™ platform

Hire a consultant1

Collect logs, interview teams…2

Identify application dependencies

Verify with every group

Static map, change requests

Implement policy, apps break

3

4

5

6

US$1M-$5M project; several months

In summary: Platform built for scale and flexibility

OpenReal time and scalableGranular policy

enforcementEasy to use

• Every packet, every flow

• Application segmentation

for thousands of

applications

• Long-term

data retention

• Consistent policy

enforcement

• Identify policy deviations

in near-real time

• Support for

workload mobility

• One-touch deployment

• Self-monitoring

• Self-diagnostics

• Standard web UI

• REST API (pull)

• Event notification (push)

• Cisco Tetration™

applications

© 2018 Cisco and/or its affiliates. All rights reserved.

FAQQ. What is the difference between a software sensor and a hardware sensor?

• Software sensors are installed on the servers (virtual machine or bare metal)

o full-visibility sensors collect telemetry data from every packet and every flow and also act as policy

enforcement points

o limited-visibility sensors provide only the conversation view required for application insights and policy

generation on certain older operating systems

• Hardware sensors are embedded into the switch Application-Specific Integrated Circuit (ASIC) itself

o collect flow data within the switch ASIC from all the ports o Supported on Nexus 9000

© 2018 Cisco and/or its affiliates. All rights reserved.

FAQQ. What is the impact of enabling telemetry capture on the server and switch CPU?

• Software sensors will consume no more than 3 percent of CPU• This threshold is configurable• Bandwidth consumption at about 1% only• Hardware sensors are performed in the switch ASIC without any impact on the CPU

© 2018 Cisco and/or its affiliates. All rights reserved.

FAQQ. How do users access information from the Cisco Tetration Analytics platform?

• Web GUI• REST API• Kafka-based push notification • Custom applications using programming languages to access to the Hadoop data lake

© 2018 Cisco and/or its affiliates. All rights reserved.

FAQQ. How does the Cisco Tetration platform work with existing data center infrastructure ?

• Customers with existing data center infrastructure, which can be Cisco or third party, can deploy the Cisco Tetration platform. Deployment is achieved by installing software sensors on virtual machines or bare-metal servers. These sensors, installed on the servers themselves, collect the required telemetry data for the analytics platform and can also act as enforcement points for the segmentation policy. Another option is to use ERSPAN sensors to generate the telemetry data based on the copied traffic

© 2018 Cisco and/or its affiliates. All rights reserved.

FAQQ. Is the policy information updated as the application behavior changes?

• Using the rich telemetry data, Cisco Tetration continuously monitors for policy compliance and deviation. For example, if additional instances of a specific application component are added, Cisco Tetration will enforce the same policy automatically on those instances. Also, if the workload moves, policy moves with it, and no additional action is required from administrators

Q. Can the Cisco Tetration Analytics platform send notification when policy deviations are identified?

• Yes. Cisco Tetration Analytics supports northbound notification through the Kafka message bus. Any northbound system can subscribe to those notifications and take additional actions. For example, a Security Incident Event Management (SIEM) system could subscribe to those events and open tickets automatically