The Five Problems Facing Business-Critical NFS Deployments

In this webinar you will learn how to detect and overcome:● Metadata Bottlenecks● Rogue Clients & Noisy Neighbor issues● Server/VM Latency issues● Poor Write Performance● Cluster Node Bottlenecks

On Demand Webinar

For audio playback and Q&A go to:

http://bit.ly/5NFSProblems

● Analyst firm focused on storage, cloud and virtualization

● Knowledge of these markets is gained through product testing and interaction with end users and suppliers

● The results of this research can be found in the articles, videos, webinars, product analysis and case studies on our web site:http://storageswiss.com

Who Is Storage Switzerland?

Our SpeakersGeorge Crump is the founder of Storage Switzerland, the leading storage analyst focused on the subjects of big data, solid state storage, virtualization, cloud computing and data protection.

He is widely recognized for his articles, white papers, and videos on such current approaches as all-flash arrays, deduplication, SSDs, software-defined storage, backup appliances, and storage networking. He has over 25 years of experience designing storage solutions for data centers across the US.

Our SpeakersCTO John Gentry is responsible for being the voice of the customer and understanding the key IT infrastructure industry trends that affect product strategy and strategic alliances. With Virtual Instruments since 2009, John brings over 20 years of IT industry experience and has held a number of senior level sales, sales engineering and product marketing positions at industry leaders such as Qlogic, Borland, McData, and CNT. John earned his BS degree from the University of California at Santa Cruz.

Who Is Virtual Instruments?

Global Leader in Infrastructure Performance Analytics

• Founded in 2008

• HQ in San Jose, CA

• Global 2000 Customers

• Every Major Vertical

• 45 of the Fortune 100

• Merged with Load DynamiX in April 2016

ChangeImpactAnalysis

TechnologyEvaluation

ProductEvaluation

InfrastructureOptimization

ProductionPerformanceAnd AvailabilityManagement

Mission Critical NFS Use Cases

• Databases on NFS

• Virtualization on NFS

• NFS as a Front End to Object/Cloud Storage

• NFS for Performance Sensitive Unstructured Data

Polling Question

What % of Your Workloads are Running on NFS?

The Advantages Of Mission Critical NFS

• Simple to Manage

• More Granular control over Data (file vs. volume)

• In many cases performance neutral

Metadata Bottlenecks

• Metadata operations are at least 50% of NFS traffic, often 80-90%

• Several NFS solutions have the ability to move metadata traffic to flash, but this is often still insufficient

• Identifying a metadata performance issue is very difficult

• Typical workarounds

Scale

• As NFS inherits more workloads, workload variability becomes a problem, especially with virtualization

• More workloads means greater capacity consumption and more metadata

• Typical workaround is scale-out NAS

Scale-Out NAS Challenges

• Scale-out NAS is an interconnected set of servers called storage nodes

• The file system is typically striped across those nodes

• Most metadata handling is performed by one node in the cluster (bottleneck)

• The nodes must stay in sync - Extra network I/O

Rogue Clients & Noisy Neighbor Issues

• Rogue Clients/Noisy Neighbors can starve other resources

• Particularly difficult to identify IP based rogue clients/noisy neighbors

• Compounded by virtualization

• Typical workarounds

• Need real-time monitoring to assess latency issues

• Ability to measure SLA adherence is practically non-existent

Server/VM Latency Issues & SLA Adherence

• Large sequential files xfer rate a challenge

• One fix is to use ‘async’ at cost of potential data loss

• Another is to go from RAID 5 or 6 to RAID 10

• Or play with datasync and writesync, and transfer settings

Poor Write Performance

●Most IP Infrastructures Are not Optimized for Mission Critical NAS

○ Storage Traffic is Different

○ Typical Workarounds

■ Buy more hardware

■ Faster Network

■ Faster or Scale-Out NAS

■ Flash, Flash and more Flash

●Infrastructure Optimization Should Happen First!

Summary

MAXIMIZE AVAILABILITY

Identify & resolve problems before users are affected

Prevent & eliminate unplanned outages and slowdowns

OPTIMIZE COST

Match purchasing & deployment decisions to your application

workload I/O profiles

Maximize utilization of existing IT assets

GUARANTEE PERFORMANCE

Monitor and optimize infrastructure & workload performance

Accelerate & de-risk IT infrastructure changes and

transformations

The 3 Pillars of Virtual Instruments Value for IT

Making Applications & Infrastructure Perform Better Together

The VI Solution Architecture

Production storage Lab storage

SAN and NAS Performance

Probes

Virtual Server Probe

NTAP Storage Software

Probe

Network Switch Probe

Workload Generation Appliance

Workload Data Importer

TAP

Switch

Servers and VMs

VirtualWisdom Management Platform

WorkloadSensor

18

VirtualWisdom Entity Centric Model

Intelligent Topology

Case-based Alarms

Live Reports

Applied Analytics

Entity Centric view of Application Infrastructure

The New Virtual Wisdom NAS Performance Probe

• Full 10G line rate monitoring of NAS protocols for 16 concurrent ports in a single 2U device

• Initial support for NFSv3; software upgradability to SMBv3 and NFSv4 in 2017

• Workload and response time metrics captured for every read and write operation

• Provides performance, capacity and health info for every attached client and server

• Out-of-band, vendor-agnostic on the wire approach

• Enables unprecedented visibility to incoming requests by client to identify rogue clients

• Compatible with existing optical TAP / TAP Patch Panels

Metrics Analyzed by VW NAS Performance Probe

• Link metrics• Health, Utilization, SFP Diagnostics

• Flow Metrics for Commands• Procedure rates/counts, pending procedures• Response times, Avg payload, sum of

payload, …• RPC Statistics

• RPC counts, NLM counts, …• Hot file metrics

• Reported for top X files per interval• File size/path attributes, …

New NAS Performance Probe

• Released with VirtualWisdom 5.0

• New wide-screen dashboard with improved navigation and dark background

• Extremely customizable dashboard – here showing addition of VM probe metrics

ProbeNAS NFSv3 Performance & Flow Analysis

The VW NAS Probe allows you to understand NFS overall performance

ProbeNAS NFSv3 Performance & Flow Analysis

and by Client/Server flows.

Using VirtualWisdom Applied Analytics to find Root Causes

Balance Finder Trend Matcher

Event Advisor

Balance Finder automatically determines if the environment is balanced or imbalanced, and tracks indicators of any

change in the balance of an environment

Trend Matcher enables you to identify the probable source of a recognized event and the

other entities that might also be affected

Event Advisor lets you quickly determine if there are any trends or events that should be

investigated or noted across the entire environment

Data-informed prediction of resource needs. Learns from “seasonal” business patterns—whether a

season is hourly, daily, weekly, monthly, quarterly, yearly, etc.

Seasonal Trend Advisor

Using VirtualWisdom Applied Analytics to Suggest Changes

Queue Solver

Queue Solver examines actual historical host configuration settings (HBA queue depths) and performance data to provide recommendations to optimize the system-wide

performance

VM Coordinator

VM Coordinator allows you to see the optimal placement of your virtual machines across your cluster to eliminate over-

provisioning and unnecessary re-balancing

VM Deployment Advisor

Identifies the optimal cluster and host to deploy a VM, based on available capacity and expected VM workload across CPU, Memory, I/O and Network.

ProbeNAS Advanced Analytics Example

Event Advisor:

Lets you quickly determine if there are any trends or events that should be investigated or noted across the entire environment

Trend Matcher:

Automatically correlates events across entire set of relevant metrics to quickly determine root cause.

Customer Case Study: NAS Performance Probe (Financial Services Beta user)

Challenge NAS Performance Probe Solution1. On-going performance

problems for months that were not resolved even after a storage upgrade.

With VirtualWisdom, in a few hours, customer discovered the issue was a single rogue client issuing ~30,000 requests/sec, doing file based replication and scanning the file system. The VW Trend Matcher analytics found the client issuing the thousands of GetAttr procedures. Change in scheduling and frequency solved the problem

2. New users could not access the NFS storage once a max number of client sessions were reached.

The NFS storage would not accept additional client sessions once it reached its maximum. The customer used VirtualWisdom metric ‘Maximum Concurrent Total NFS Procedures’ with a time-based comparison; set up an alarm threshold for proactive notification at 80% of limit to avoid the problem.

3. Customer did not have a consistent way of resolving NFS issues.

With VirtualWisdom, customer was able to develop a report that any admin could use to investigate when a user complains. Type in IP address and see which mount point they’re having problems with.

Summary: The industry’s 1st real-time NAS Monitoring Solution

• VI expanding into new markets

• Enhanced GUI for VirtualWisdom5

• Meeting the performance needs of NAS users

• Leveraging our SAN monitoring expertise

• Part of our comprehensive IPA platform

Thank you!Storage Switzerlandhttp://www.storageswiss.comgeorgeacrump@storageswiss.com

StorageSwiss on Twitter:http://twitter.com/storageswiss

StorageSwiss on YouTube:http://www.youtube.com/user/storageswiss

Virtual Instrumentshttp://www.virtualinstruments.com

Virtual Instruments on Twitter:http://twitter.com/Virtual_Inst

Virtual Instruments on YouTube:https://www.youtube.com/user/sos4sans

Virtual Instruments on Facebook:https://www.facebook.com/virtualinstruments

The Five Problems Facing Business-Critical NFS Deployments

For complete Audio and Q&A please register for the On Demand Version at:

http://bit.ly/5NFSProblems