Date post: | 31-Dec-2015 |
Category: |
Documents |
Upload: | philip-kidd |
View: | 28 times |
Download: | 0 times |
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester1
VLBI & Protocols
vlbi_udp Multiple Flow Tests
Richard Hughes-Jones The University of Manchester
www.hep.man.ac.uk/~rich/ then “Talks”
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester2
Setting up the Tests
Thank you to the people at all the sites for your help and making facilities available
Including: Onsala JIVE Torun and Pioner Poznan GARR Bologna Jodrell and Manchester Network Group Dante
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester3
Dedicated
Gbit link
OnsalaSweden
Gbit link
Jodrell BankUK
DwingelooDWDM linkMedicina
Italy
Chalmers University
of Technolog
y, Gothenbur
g
TorunPoland
Gbit link
Metsähovi
vlbi_udp: Multi-site Streams
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester4
iGrid2002 monolithic code Convert to use pthreads
control Data input Data output
Code branch for Simon’s file transfer tests
Work on vlbi_recv: Output thread polled for data in the ring buffer – burned CPU Input thread signals output thread when there is work to do – else wait on
semaphore – packet loss at high rate, variable thoughput Output thread uses sched_yield() when no work to do
Multi-flow Network performance – being set up Nov/Dec06 3 Sites to JIVE: manc UKLight; Manc production; Bologna GEANT PoP Measure: throughput, packet loss, re-ordering, 1-way delay
To do Add code for: MarkV card and PCEVN interface
vlbi_udp: UDP on the WAN
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester5
vlbi_udp: Some of the Problems
JIVE made Huygens, mark524 (.54) and mark620 (.59) available Within minutes of Arpad leaving the Alteon NIC of mark524 lost the
data network! OK used mark623 (.62) – faster CPU
Firewalls needed to allow vlbi_udp ports Aarrgg (!!!) Huygens is SUZE Linux
Routing – well this ALWAYS needs to be fixed !!! AMD Opteron did not like sched_getaffinity() sched_setaffinity()
Comment out this bit udpmon flows Onsala to JIVE look good udpmon flows JIVE mark623 to Onsala & Manc UKL don’t work
Firewall down stops after 77 udpmon loops Firewall up udpmon cant communicate with Onsala
CPU load issues on the markV systems Don’t seem to be able to keep up with receiveing UDP flow AND
emptying the ring buffer Torun PC / Link lost during the tests
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester6
vlbi_udp Onsala mark623
It worked ! No not quite !!! Asked for a flow of 600Mbit/s (-w20 us)
got ~300 Mbit/s (-w58 us) 1.8% packet loss 1.1% re-ordered (mainly by 1 or 2 packets)
Onsala gig7 (Academic internet to Ams then UKLight)Onsala gig8 (all Academic internet)
~4.5% lost and ~3.5% re-ordered
vlbi_udp_Onsala-gig8_w20_4Dec06
1
10
100
1000
10000
100000
1000000
10000000
0 20 40 60 80 100 120 140 160
Time between frames us
N(t
)
vlbi_udp_gig7-Huy_w20_2Dec06
1
10
100
1000
10000
100000
1000000
10000000
0 20 40 60 80 100 120 140 160
Time between frames us
N(t
)
2.2GHz AMD Athlon Udpmon OK but single thread Vlbi_udp uses recvmsg() &
iovectors
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester7
Multiple vlbi_udp Flows Gig7 Huygens UKLight 15 us spacing 800 Mbit/s Gig8 mark623 Academic Internet 20 us spacing 600 Mbit/s Bologna mark620 Academic Internet 30 us spacing 400 Mbit/s
SURFnet Access link
SJ5 Access link
GARR Access link
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester8
Multiple vlbi_udp Flows Gig7 Huygens
UKLight 15 us spacing 816 Mbit/s del <1Mbit/s
step 1 Mbit/s Zero packet loss Zero re-ordering
Gig8 mark623 Academic Internet 20 us spacing 612 Mbit/s 0.6 falling to 0.05% packet loss 0.02 % re-ordering
Bologna mark620 Academic Internet 30 us spacing 396 Mbit/s 0.02 % packet loss 0 % re-ordering
vlbi_udp_3flows_6Dec06
0100200
300400500600700
800900
1000
0 2000 4000 6000 8000 10000 12000 14000
Time s
Wir
e R
ate
Mb
it/s
00.10.2
0.30.40.50.60.7
0.80.91
Pa
ck
et
los
s %
vlbi_udp_3flows_6Dec06
0100
200300400500
600700800
9001000
0 2000 4000 6000 8000 10000 12000 14000
Time s
Wir
e R
ate
Mb
it/s
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Pa
ck
et
los
s %
vlbi_udp_3flows_6Dec06
0100
200300
400500600
700800
9001000
0 2000 4000 6000 8000 10000 12000 14000
Time s
Wir
e R
ate
Mb
it/s
0
0.02
0.04
0.06
0.08
0.1
0.12
Pa
ck
et
los
s %
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester9
Any Questions?
ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester10
Backup Slides