Topic 5 NX-OS Management

NX-OS Management

• SPAN and ERSPAN• NetFlow• Smart Call Home• Manage System Files• Implement NTP, PTP• Implement Configure and Verify DCNM

Functionality

SPAN• Switchport analyzer : Copies the traffic (Tx/RX/both)

from source interface/VLANs to destination port.• Destination port cannot be a port-channel, FEX’s

host interface or a shared interface (between storage and LAN VDC) and cannot be part of two different span session. Must be in same VDC as source port.

• Destination port can be an access port or trunk port but MUST BE configured with “switchport monitor” command.

• Source interface can be a 10gig port and destination can be 1gig port in same span session.

SPAN• N7K introduces virtual SPAN session to

monitor multiple VLAN sources and choose only VLANs of interest to transmit on multiple destination ports.

• IMP thing to remember is Virtual SPAN sessions cause all source packets to be copied to all destinations, whether the packets are required at the destination or not. VLAN traffic filtering occurs at the egress destination port level.

SPAN configuration exampleNexus 7000N7K(config)# interface ethernet 2/5N7K(config-if)# switchportN7K(config-if)# switchport mode trunkN7K(config-if)# switchport monitorN7K(config-if)# no shutN7K(config-if)# exitN7K(config)# monitor session 3N7K(config-monitor)# source interface

ethernet 2/1-3, ethernet 101/1/2N7K(config-monitor)# source interface port-

channel 2 rxN7K(config-monitor)# source interface sup-eth

0 bothN7K(config-monitor)# source vlan 3, 6-8 txN7K(config-monitor)# filter vlan 3-5, 7N7K(config-monitor)# destination interface

ethernet 2/5N7K(config-monitor)# no shut

Nexus 5500N5K(config)# interface ethernet 1/3N5K(config-if)# switchport monitorN5K(config-if)# exitN5K(config)# monitor session 3N5K(config-monitor)# source interface ethernet

1/16N5K(config-monitor)# source interface port-

channel 2 rxN5K(config-monitor)# source vlan 1N5K(config-monitor)# destination interface

ethernet 1/3

N5K(config)# interface fc 2/4N5K(config-if)# switchport mode SDN5K(config-if)# switchport speed 1000N5K(config-if)# exitN5K(config)# monitor session 2N5K(config-monitor)# source vsan 1N5K(config-monitor)# destination interface fc

2/4

SPAN configuration differences

• In N7Kswitch(config)# monitor session 3switch(config-monitor)# rate-limit 10switch(config-monitor)# no shut

• In N5Kswitch(config)# no monitor session 3 shutswitch(config-if)# switchport monitor rate-limit 1G

I do not have good answer for this configuration differences.

N5K Valid span source and destination

Source SPAN Dest SPAN

Ethernet Ethernet

Fibre Channel Fibre Channel (speed must be configured)

Fibre Channel Ethernet (FCoE) (10G port Only)

Virtual Fibre Channel Fibre Channel

Virtual Fibre Channel Ethernet (FCoE) (10G port Only)

Feature Parameter Verified Limit (Cisco NX-OS 6.0)

SPAN and ERSPAN

Number of configured (not active) SPAN sessions per VDC

48

Number of active SPAN or ERSPAN source sessions per system

2

Number of active ERSPAN destination sessions per system

23

Number of source interfaces per SPAN or ERSPAN session

128

Number of destination interfaces per SPAN or ERSPAN session

32

Number of source VLANs per SPAN or ERSPAN session1

32

N7K configuration limits for SPAN

Encapsulated Remote Switched Port Analyzer (ERSPAN)

• ERSPAN transports mirrored traffic over an IP network in GRE encapsulated packets.• There are two types of ERSPAN sessions : source and destination.

• Sample for erspan-source type configurationNX-7000# config tNX-7000(config)# interface e1/30NX-7000(config-if)# no shutNX-7000(config-if)# exitNX-7000(config)# monitor erspan origin ip-address 3.3.3.3 globalNX-7000(config)# monitor session 1 type erspan-sourceNX-7000(config-erspan-src)# source interface e1/30NX-7000(config-erspan-src)# erspan-id 1NX-7000(config-erspan-src)# ip ttl 16NX-7000(config-erspan-src)# ip dscp 5NX-7000(config-erspan-src)# vrf defaultNX-7000(config-erspan-src)# destination ip 9.1.1.2NX-7000(config-erspan-src)# no shutNX-7000(config-erspan-src)# exitNX-7000(config)# show monitor session 1

Encapsulated Remote Switched Port ANalyzer (ERSPAN)

• Sample for erspan-destination type configurationNX-7000# config tNX-7000(config)# interface e1/30NX-7000(config-if)# ip address 9.1.1.2/24NX-7000(config-if)# no shutNX-7000(config)# interface e2/20NX-7000(config-if)# switchport mode accessNX-7000(config-if)# description Laptop with wireshark connected to itNX-7000(config-if)# switchport monitorNX-7000(config)# monitor session 1 type erspan-destinationNX-7000(config-erspan-dst)# source ip 9.1.1.2NX-7000(config-erspan-dst)# destination interface e2/20NX-7000(config-erspan-dst)# no shutNX-7000(config-erspan-dst)# erspan-id 5NX-7000(config-erspan-dst)#vrf defaultNX-7000(config)# show monitor session 1

Encapsulated Remote Switched Port ANalyzer (ERSPAN)

• Nexus 1000v does not support destination erspan• Capability L3-control has to be specified for port-profile of ERSPAN source. This port profile is applied to vmk

port of hypervisor and it is used as source of a GRE tunnel.

Netflow• It is a Cisco NX-OS application that provides statistics on packets flowing

through the router. • NetFlow captures data from ingress and egress packets. • Egress packets

– Egress NetFlow Accounting: IP traffic only– NetFlow MPLS Egress: MPLS-to-IP packets

• Key Fields– Src IP – Dst IP– Src Port– Dst port– Layer 3 protocol type– Type of service (ToS)– Input Logical Interface

• You can configure NetFlow on a per-subinterface basis.

Netflow Export• Expired flows in NetFlow Cache grouped together into “NetFlow

Export” datagrams for export from the device. • Versions

– V9: more flexible and extensible. Supports IPv4, IPv6, Multicast, MPLS and BGP. Record formats are defined by templates.

– V8: A format added to support data export from aggregation cache. – V5: most commonly used format. Adds BGP AS information and flow

sequence number. – V1: Initially released export format

• Datagram sizes– V1: 24 flows, 1200bytes– V5&v9: 30 flows, 1500bytes

Netflow• Sample configuration

feature netflow flow exporter Exporter_name destination 192.168.2.12 use-vrf management source mgmt0 ! It can be any other interface. version 9flow record Record_name match ipv4 source address match ipv4 destination address collect counter bytes collect counter packets flow monitor FOO record Record_name exporter Exporter_name interface Ethernet2/45 ip flow monitor FOO output ip address 10.20.1.1/24 no shutdown

Smart Call Home (SCH)• SCH provides an automated notification system

for policies that Network admin has defined.• E.g SCH can automate process of opening a TAC

case with Cisco TAC for hardware failure and attach appropriate corresponding CLI output.

• SCH is email based application and it supports– Test based– XML basedMessage formats (destination-profiles format).

Smart Call Home (SCH)

• Prerequisites for SCM configuration.Requires a SMARTNET support contract from CiscoCCO ID that has this contract attached. SNMP system contact has o be configured “snmp-

server contact sys-contact”. ip domain-name and ip name-server for DNS look-

ups or ip host for static entries in order to resolve host-names that may appear in destination addresses.

Register device using call home registration process

Smart Call Home (SCH) Configuration• Everything is configured under “callhome” and configuration can be seen using show run

callhome. • Configure SNMP syscontact

NX-7000(config)# snmp-server contact person@company.com• Configure the mandatory contact information

NX-7000(config)#callhome NX-7000(config-callhome)#email-contact email-address NX-7000(config-callhome)#phone-contact +1-000-000-0000 NX-7000(config-callhome)#streetaddress a-street-address

• Configure the mandatory email server information and from email address NX-7000(config-callhome)#transport email smtp-server ip-address port 25 use-vrf vrf-name NX-7000(config-callhome)#transport email from email-address

• Set the destination profile and attach alert group NX-7000(config-callhome)#destination-profile CiscoTAC-1 email-addr callhome@cisco.com NX-7000(config-callhome)#destination-profile CiscoTAC-1 transport-method emailNX-7000(config-callhome)#destination-profile CiscoTAC-1 alert-group Cisco-TAC

• Add additional command in alert groupNX-7000(config-callhome)# alert-group Cisco-TAC user-def-cmd show ip route

• Finally commit and enable callhomeNX-7000(config-callhome)#commitNX-7000(config-callhome)#enable

Smart Call Home (SCH) Configuration• Testing Smart Call Home Communications

NX-7000(config-callhome)# callhome send [diagnostic | configuration ]

NX-7000(config-callhome)# callhome test• Callhome configuration (except for SNMP sysContact

and device priority) can be distributed thru CFS over IP or CFS over Ethernet just like device-alias, zones and DPVM in fiber channel world to other switches.NX-7000(config)# callhome NX-7000(config-callhome)# distribute NX-7000(config-callhome)# show callhome status Distribution : Enabled NX-7000(config-callhome)# commit

Smart Call Home (SCH) verificationNX-7000# show callhome ? <CR> > Redirect it to a file >> Redirect it to a file in append mode destination-profile Show callhome destination profile information last Show the status of the last cfs commit/abort operation merge Show the status of the last cfs merge operation pending Show the status of pending callhome commands pending-diff Show the difference between running and pending config session Show the status of the last cfs commit/abort operation status Show if CFS distribution is enabled/disabled for callhome transport Show callhome transport configuration (email and http) user-def-cmds Show the cli commands configured for each alert group | Pipe command output to filter

Smart Call Home (SCH)• Sample Lab question: Configure call home feature on NX-AGG01 VDCs. Create a desitnation profile call it N0c101, use admin

@company.com,1 800 123 4567 and address: 123 Anystreet st. Anytown,AnyWhere. Send configuration along with routing table of default vrf.set urgency level to “Major”. Use 192.0.2.10 as email server make sure to use proper vrf which is used to reach this email server. All configuration has to be done on NX-AGG01 switch and it should distribute this configuration to NX-AGG02 switch.

• Solution: switchto vdc NX-AGG01 cfs ipv4 distributesnmp-server contact person@company.com callhome distribute email-contact admin@company.com phone-contact +1-800-123-4567 streetaddress 123 Anystreet st. Anytown,AnyWhere destination-profile Noc101 format full-txt destination-profile full-text-destination callhome@cisco.com destination-profile full-text-destination message-level 5 destination-profile Noc101 alert-group Configuration alert-group Configuration user-def-cmd show ip route transport email mail-server 192.0.2.10 priority 1 transport http use-vrf Blue enable commit switchto vdc NX-AGG02 cfs ipv4 distributesnmp-server contact person@company.com callhome

Management of System FilesThere are 5 files systems in Nexus 7000 and nexus 5500

NX-7000# dir ?bootflash: Directory or filename debug: Directory or filename log: Directory or filename logflash: Directory or filename

on log flash slot0: Directory or filename

on expansion flash usb1: Directory or filename usb2: Directory or filename volatile: Directory or filename

NX-5500# dir ?bootflash: Directory or filename debug: Directory or filename log: Directory or filename modflash: Directory or filename usb1: Directory or filename volatile: Directory or filename

Management of System Files• Bootflash: Internal CompactFlash memory located on the supervisor

module used for storing image files, configuration files, and other miscellaneous files. The initial default directory is bootflash. In N7K it has two modules sup-remote sup-local.

• Debug: Memory on a supervisor module used for debug logs.• Log: Memory on the active supervisor that stores logging file statistics.• Logflash or modflash: • System: Memory on a supervisor module used for storing the running-

configuration file.• Volatile: Volatile random-access memory (VRAM) located on a supervisor

module used for temporary or pending changes.• NVRAM: Nonvolatile random-access memory (NVRAM) located on a

supervisor module used for storing the startup-configuration file.• Usb1 or usb2: External USB flash memory installed in a supervisor module

used for storing image files, configuration files, and other miscellaneous files.

Management of System Files• Identifying the Current Directory

NX-7000# pwdbootflash:

• Creating a DirectoryNX-7000# mkdir trace

• Changing the Current DirectoryNX-7000# cd traceNX-7000# pwdbootflash:trace

• Displaying Directory ContentsNX-7000# dir bootflash:161980383 Sep 07 16:29:26 2011 n7000-s1-dk9.5.2.1.bin 30674944 Aug 20 16:41:54 2011 n7000-s1-kickstart.5.2.1.bin 4096 Jul 04 14:49:28 2012 trace/

• Deleting a DirectoryNX-7000# rmdir trace

• Accessing Directories on the Standby Supervisor ModuleNX-7000# dir bootflash:? bootflash:/// bootflash://module-5/ bootflash://module-6/ bootflash://sup-1/ bootflash://sup-2/ bootflash://sup-active/ bootflash://sup-local/ bootflash://sup-remote/ bootflash://sup-standby/

Management of System Files• Moving Files: move [filesystem:[//module/][directory /] | directory/]source-

filename {{filesystem:[//module/][directory /] |directory/}[target-filename] | target-filename}

• Copying Files: copy [filesystem:[//module/][directory/] | directory/]source-filename | {filesystem:[//module/][directory/]] |directory/}[target-filename]

• Deleting Files: NX-7000# delete bootflash:hardware.txt ? <CR> no-prompt Do not prompt for multiple deletion of files

• Displaying File Contents: show file bootflash:startuplogs.txt • Displaying File Checksums: show file bootflash:startuplogs.txt [md5 | cksum]• Compressing and Uncompressing Files (Creating an Archive Files):

NX-7000# g? gunzip Uncompresses LZ77 coded files gzip Compresses file using LZ77 coding

• Displaying the Last Lines in a File: NX-7000#tail bootflash:startuplogs.txt

Management of System Files• Redirecting show Command Output to a File

NX-7000# show hardware ? <CR> > Redirect it to a file >> Redirect it to a file in append modeNX-7000# show hardware > bootflash:hardware.txt

• Finding Files in directory and its all subdirectories. V-BAN1-NX7K01# find hardware.txt/usr/bin/find: ./lost+found: Permission denied./hardware.txt

• Collecting core files from core: partition, command “show cores” then Collect the core file number e.g. 1123 then “copy core:1123 tftp:”

Network Time protocol (NTP)

• Protocol used to synchronize timing on network devices. All NTP communications use Coordinated Universal Time (UTC).

• Only default VDC synchronizes the system clock at any given time but multiple instances of NTP on different VDCs are supported.

• To configure NTP, you must have connectivity to at least one server that is running NTP.

• VRF aware• Configuration can be distributed via CFS (but not ntp

authentication key).

Network Time protocol (NTP)• Feature ntp• ntp server {ip-address | ipv6-address | dns-name}

[key key-id] [maxpoll max-poll] [minpoll min-poll] [prefer] [use-vrf vrf-name] ! Forms an association with a server.

• ntp authentication-key number md5 md5-string• ntp trusted-key number• ntp access-group {peer | serve | serve-only | query-

only} access-list-name• ntp source-interface interface | ntp source ip-address• ntp logging

Network Time protocol (NTP)NX-7000# sh ntp ? access-groups Display NTP access groups authentication-keys Display authentication keys authentication-status NTP Authentication Status internal NTP internal info logging-status Display NTP logging status peer-status Show the status for all the server/peers peers Show all the peers. pending Show the NTP temporary database pending-diff Show the pending database diff. rts-update Show if the RTS update is enabled session Show the session information source Source IP address configured source-interface Source interface configured statistics Show the NTP statistics status Show the NTP distribution status trusted-keys Display trusted keys

Reference

• http://www.cisco.com/en/US/docs/switches/datacenter/sw/6_x/nx-os/system_management/configuration/guide/sm_3ntp.html