Standards

Certification

Education & Training

Publishing

Conferences & Exhibits

Test Tool for Industrial Ethernet Network Performance

Jim Gilsinn & Freemon JohnsonNational Institute of Standards & Technology (NIST)

Manufacturing Engineering Laboratory (MEL)

Author Bio: Jim Gilsinn

• Over 16 years of experience at NIST• Variety of manufacturing projects, including intelligent

unmanned vehicles, welding standards, smart sensors, and nano-manufacturing

• Current research focus is industrial network reliability, including performance and security

• Masters in Electrical Engineering from Johns Hopkins University, specialty in Control Systems; Bachelors in Electrical Engineering from Drexel University, specialties in Robotics and Control Systems

2

Author Bio: Freemon Johnson

• Over 10 years of experience; previously worked at NASA Goddard Space Flight Center and Motorola

• Expertise in communication protocol security, architecture, and embedded real-time systems

• Masters in Computer Science from Johns Hopkins University; Masters in Communication and Information Studies from Rutgers University; Bachelors in Electrical Engineering from New Jersey Institute of Technology

3

Overview

• Background• Performance Testing Methodology• Data Analysis• Test Tool• Future Plans

4

Background

• The Problem– Ethernet & TCP/IP inherently non-deterministic– Users have many questions about industrial Ethernet devices– Automobile Example

• Why NIST?– Aid U.S. businesses and industries, specifically manufacturing– Many companies are global– Industrial Ethernet standards are available worldwide

5

Performance Testing Methodology:Performance Metrics

`

• Publish/subscribe or peer-to-peer communications• Main performance metric: Cyclic frequency

variability/jitter• Real-time EtherNet/IP uses publish/subscribe

– Requested/Accepted Packet Interval (RPI/API)– Measured Packet Interval (MPI)

6

Performance Testing Methodology:Performance Metrics

`

• Command/response or master/slave communications• Main performance metric: Latency

7

Performance Testing Methodology:Testing Methodology

• Overall methodology for testing performance:1. Begin recording network traffic

2. Establish a connection with the device under test (DUT)

3. Begin transmitting background network traffic, based on the particular test conditions

4. Wait for a given amount of time

5. Stop transmitting background network traffic

6. Close the connection with the DUT

7. Stop recording network traffic

8. Analyze the network traffic capture and report the results

• Current test tool only for data analysis & reporting

8

Performance Testing Methodology:Test System

9

Data Analysis: Overview

• Current test tool focuses on cyclic frequency variability/jitter of the MPI

• Capable of analyzing multiple network packet streams from different devices during a single test run

• Conduct a distribution analysis on the MPI results– Determine minimum, maximum, mean, standard deviation,

skewness, and kurtosis of data set

• Produce graphs of MPI results

10

Data Analysis: Example Jitter GraphM

easu

red

Pac

ket

Inte

rval

(s)

Test Time (s)

This ExampleTest Duration ≈ 61 sMean ≈ 2 msMinimum ≈ 1.2 msMaximum ≈ 2.9 ms

11

Data Analysis: Observations

12

Data Analysis: Observations

13

Test Tool: Design

• Geared towards vendors & users– Vendors: develop better products during design process– Users: determine how well devices operate in a real system

• Multi-phase design approach– Analysis → Testing

• Meaningful results– Mathematical foundation– User selectable complexity for results

14

Test Tool: Technical Details

• Free software package– Microsoft Windows-based (Windows 2000 or later)

• Public domain source-code– Visual C# modular design– Commercial graphing package

• Open-source installed package– No special software or hardware needed to run– Integrated with Wireshark for packet analysis

15

Future Plans

• Planned versions of IENetP Test Tool– Data Analysis Tool

– Version 1.x – Basic functionality & test engine– Version 2.x – Additional data analysis capability & metrics– Version 3.x – Additional industrial networks

– Active Testing Tool– Version 4.x – PC-based test tool– Version ?.? – Specialized hardware test tool

16

Summary

• NIST is working with industry partners to develop industrial Ethernet performance metrics, tests, and tools– Determined metrics– Developed basic methodology

• NIST is currently working on a test tool– Designed as a development tool for industrial Ethernet vendors– Focused on data analysis presently

• Some patterns have emerged while analyzing data• Additional functionalities are planned for the test tool

17

Contact Information

• Jim Gilsinn, 301-975-3865, james.gilsinn@nist.gov• Freemon Johnson, 301-975-3429, freemon.johnson@nist.gov• Mailing Address

– National Institute of Standards & Technology (NIST)Manufacturing Engineering Laboratory (MEL)100 Bureau Drive, Mailstop 8230Gaithersburg, MD 20899-8230

• Websites– Project: http://ienetp.sourceforge.net– Division: http://www.isd.mel.nist.gov

18