Ethernet and TCP optimizations

Cisco Confidential 1© 2012 Cisco and/or its affiliates. All rights reserved.

Ethernet: Hidden Secrets Jeff Squyres

First: some backgroundinformation…

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3

Jeff’s work: Parallel computing at Cisco

Using lots and lots and lots of servers simultaneouslyto solve one computational problem


Supercomputing applications

Racks of36 1U

servers

Tend to send lots and lots and lots of small messagesacross the network to stay in sync with each other


Network message traversal

Underlying network

Send amessage

Receive themessage

A B


Network message traversal

Underlying network

Send amessage

Receive themessage

Today’s fastest networks:1-3μs (!)

A B


Today’s fastest networks

• Typically not Ethernet networks

• Usually have supercomputer-specific networksExample: highly tuned for short message latency

• …but that is changing

Ethernet Ethernot


Cisco’s ultra low latency Ethernet

• Userspace NIC (“USNIC”)Expose Cisco NIC hardware directly to Linux userspace

Bypass the OS

Bypass the TCP stack

• Send raw Ethernet frames directly from user applicationsMuch, much faster than traditional TCP-based networking

Especially for latency of short messages


Kernel

Cisco VIC hardware

TCP / IP stack

Cisco VIC driver

Normal TCP software architecture

UserspaceUserspace sockets library

MPI library

Application


Kernel

Userspace verbs library

Cisco VIC hardware

Cisco USNIC software

MPI library

Userspace

Verbs IB core

Cisco USNIC driver

Bootstrappingand setup

Send and receivefast path

Application

With all that background…


Doing some performance testing last week…

Two servers



Two servers

Each with a 2 x 10Gb NIC



Two servers

Each with a 2 x 10Gb NICConnected back-to-back


“Ping pong” latency test

Send a messagefrom here

Receive the messagehere

Ping!



Get the messageback

Send the messageback

Pong!



Because each ping and pong are soooo short,do this ping-pong exchange N times

Ping! / Pong!



Total time for N ping-pongs

N

Time for one ping-pong



Total time for N ping-pongs

N

Time for one ping-pong

2

Time for one ping


Time for one ping

Half-round trip (HRT)ping pong latency

=


Results: using 1x10G Ethernet port

1 byte~60μs

8MB~150ms


Results: using 2x10G Ethernet ports

1 byte~60μs

8MB~150ms

8MB~8.3ms

1 byte~30μs (!)


Results: using 2x10G Ethernet ports

1 byte~60μs

8MB~150ms

8MB~8.3ms

1 byte~30μs (!)

WHOA!


Results: just the small messages

The facts:From 1-1024 bytes: flat latency

Using 1 interface: ~60μsUsing 2 interfaces: ~30μs

~60μs

~30μs



The facts:From 1-1024 bytes: flat latency

Using 1 interface: ~60μsUsing 2 interfaces: ~30μs

~60μs

~30μsWHY?


Must look at how TCP works…

1. Ethernet frame arrives




2. NIC sends interruptto OS Ethernet driver





3. OS Ethernet drivercopies the packet to RAM





3. OS Ethernet drivercopies the packet to RAM

4. OS TCP stack handspacket off to (whatever)


The Costco Rule

It’s always better in bulk


Why copy one packet at a time?

Let’s optimizethis part


Two (commonly used) optimizations

1. Copy a bunch ofpackets across PCI

at one time



2. Only raise oneinterrupt for all of

those packet copies


at one time



2. Only raise oneinterrupt for all of

those packet copies


at one time

A.k.a. “Interrupt Coalescing”


Interrupt coalescing





2. Has N time passedsince we sent an

interrupt to the OS?






No: queue up the frame✖






No: queue up the frame✖✔ Yes: Send all queued frames and interrupt

Ok… So what?


The key: NIC interrupt coalescing timers


Timeline of a ping pong

NIC A

NIC B

1. A sends ping frame

2. B receives ping frame

Periodic interruptcoalescing timeout

125μs



NIC A

NIC B

3. Coalesce timer expires; B sends interrupt4. B sends pong frame



NIC A

NIC B

5. Coalesce timer expires; A sends interrupt6. A sends ping frame7. Rinse, repeat



NIC A

NIC B

4 ping-pongs in ~8x timer duration



NIC A

NIC B

In general, coalescing interrupts is a very Very Good Thing



NIC A

NIC B

But it definitely hurts low-latency traffic

How do we reduce those artificial delays?


Two Ethernet ports with out-of-sync timers

NIC A

NIC B

NIC A

NIC B

Por

t 0

Por

t 1


Get more round trips in same amount of time

NIC A

NIC B

NIC A

NIC B

Por

t 0

Por

t 1


Get more round trips in same amount of time

NIC A

NIC B

NIC A

NIC B

Por

t 0

Por

t 1

In reality, sender and receiver timers on each port are wholly unrelated; they don’t line up

nicely like I used in these examples.

Meaning: in general, you actually usually get better overlap



~60μs

~30μs

In this case, we got such good asymmetry, that the 2 port case is ~2x as fast (i.e., roughly twice as many interrupts in the same amount of time)


Lies, damn lies, and statistics

Remember:these are AVERAGE

latencies!

Individual ping-pong timesare the same as the

1 port case (from the network)

…but you get higher throughputbecause we’re reducing the

gaps between each ping-pong

Now let’s trysomething else…


Set the coalesce timer at 0


New ping-pongs much faster!

1 port~10.5μs

2 ports~10.6μs

1 port~7.2ms

2 ports~5.5ms


What are the tradeoffs?

Pros• (Much) faster TCP latency

…without changing app!

• Faster speeds seem to scale up to large messages, too

• Great for low-latency, sparse comms apps

• Best for NICs that are dedicated to MPI comms

Cons• May not scale well for

case of MPI process running on every core

• Lots and lots of interrupts going to socket:0.core:0

• May need to run (N-1) MPI processes…?

May also want to avoid socket:0.core:0, or move IRQ affinity

Your mileage may vary


But it’s interesting, nonetheless!

• Some experimentation might be worth trying with real world HPC apps:

• Allow TCP to wholly utilize core 0 (i.e., run MPI processes only on cores 1-15)

• Set the coalesce timer to something more than 0μs, but less than 125μs – there’s a whole spectrum with which to play


My overall points:

• Many in HPC have Ethernot networks …but as HPC continues to commoditize itself, lots of HPC users have Ethernet-based environments

• Today’s Ethernet switches and NICs are actually quite a bit faster and more advanced than what we old-time-HPCers grew up with

• Even good ol’ TCP is amazingly fast and optimized today

• You may be able to tune your NIC and/or fabric to extract pretty darn good MPI TCP performance

The default settings on your Ethernet NIC / fabric are likely set for general TCP traffic – which effect very different performance characteristics than what HPC applications typically need

Thank you.

Date post:	12-May-2015
Category:	Technology
Upload:	jeff-squyres
View:	1,985 times
Download:	3 times

Ethernet and TCP optimizations

Technology