VLBI_UDPVLBI_UDPAn application for An application for

transferring VLBI data via transferring VLBI data via UDP protocolUDP protocol

Simon Casey Simon Casey

e-VLBI meeting, Haystacke-VLBI meeting, Haystack20 Sep 200620 Sep 2006

OverviewOverview

Originally developed by Richard Hughes-JonesOriginally developed by Richard Hughes-Jones Sent multiple, continuous memory-memory UDP Sent multiple, continuous memory-memory UDP

streamsstreams Multi-threaded, contained ring bufferMulti-threaded, contained ring buffer Web interface to manage streams & graph Web interface to manage streams & graph

throughputthroughput

Modified to allow disk-disk transfersModified to allow disk-disk transfers Currently implementing selective packet Currently implementing selective packet

droppingdropping

Ring buffer

Input thread

Ring buffer

Send thread

Disk Memory

Receive thread

Output thread

Disk Memory

UDP Data

TCP Control

Control thread Control thread

Vlbi_control

vlbi_send vlbi_recv

vlbi_udp Flow Diagramvlbi_udp Flow Diagram

Geant2 Launch event, Geant2 Launch event, June 2005June 2005

StabilityStability

Instability in sending Instability in sending rate when not rate when not saturating linksaturating link

~20Mbit/s variation~20Mbit/s variation

Change thread Change thread implementation to use implementation to use sched_yield()sched_yield()

Stable to within 1Mbit/sStable to within 1Mbit/s

vlbi_recv_k269_110d_w20_29Aug06

600

602

604

606

608

610

612

614

616

618

620

0 1000 2000 3000 4000 5000

Time between frames us

Wir

e R

ate

Mb

it/s

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Packet

loss

Packet loss & correlationPacket loss & correlation

Packets are lost - networks Packets are lost - networks (congestion) and/or end-hosts (lack (congestion) and/or end-hosts (lack of processing power)of processing power)

Lost packets will have some effect Lost packets will have some effect on final correlationon final correlation

Want to measure this effectWant to measure this effect

GoalsGoals

Determine optimal way to lose Determine optimal way to lose packets whilst having least effect on packets whilst having least effect on correlationcorrelation

Find reasonable loss rate which can Find reasonable loss rate which can be sustained by station unitsbe sustained by station units

Implement findings into VLBI_UDPImplement findings into VLBI_UDP Can be used to selectively drop packets Can be used to selectively drop packets

if congestion present yet keep if congestion present yet keep correlation runningcorrelation running

Correlation tests at JIVECorrelation tests at JIVE

Run disk2file to create standard linux fileRun disk2file to create standard linux file Use VLBI_UDP in disk-disk modeUse VLBI_UDP in disk-disk mode

Add extra mode to write to local file instead of Add extra mode to write to local file instead of sending to networksending to network

Filter each packet through a dropping Filter each packet through a dropping functionfunction Algorithm can be selected by user and Algorithm can be selected by user and

parameters variedparameters varied Place modified data file(s) back on to disk Place modified data file(s) back on to disk

packpack Play back into correlatorPlay back into correlator

Packet dropping functionPacket dropping function

Pass pointer to valid packet as Pass pointer to valid packet as parameterparameter

Function decides if packet will be Function decides if packet will be droppeddropped Returns either original pointer or Returns either original pointer or

pointer to some dummy datapointer to some dummy data What dropping algorithms are used?What dropping algorithms are used?

Keeping in syncKeeping in sync

Station units look for valid headersStation units look for valid headers No header? Work out where next No header? Work out where next

header should be and keep lookingheader should be and keep looking How many missing headers before sync How many missing headers before sync

is lost?is lost? Lose only VLBI data, keep headersLose only VLBI data, keep headers

SUs will be happy – only care about SUs will be happy – only care about headersheaders

Actual correlation will sufferActual correlation will suffer

Keeping in syncKeeping in sync Assume losing every other header Assume losing every other header

acceptableacceptable At 256Mbit/s, headers occur every 80 000 At 256Mbit/s, headers occur every 80 000

bytesbytes 32 * 2500 byte tracks = 1 'frame'32 * 2500 byte tracks = 1 'frame' 79360 bytes data sandwiched between 384 bytes 79360 bytes data sandwiched between 384 bytes

header and 256 byte footerheader and 256 byte footer ~54 UDP packets/'frame'~54 UDP packets/'frame' Lose Lose ≤≤ 1 packet in 56 (1.79%) – will never lose 2 1 packet in 56 (1.79%) – will never lose 2

consecutive headersconsecutive headers N.B. assumes losses don't occur randomly!N.B. assumes losses don't occur randomly!

Dropping strategiesDropping strategies

Leave first 2 mins of each run intactLeave first 2 mins of each run intact Use loss rates:Use loss rates:

0.5%; 1%; 1.5%; 1.75%; 2%; 2.25%0.5%; 1%; 1.5%; 1.75%; 2%; 2.25% Run each rate twice, dropping in series Run each rate twice, dropping in series

of long bursts or evenly spaced of long bursts or evenly spaced individual packetsindividual packets

Expect rates 2%, 2.25% sync may be Expect rates 2%, 2.25% sync may be lost, below 2% should stay in synclost, below 2% should stay in sync

Total 12 runsTotal 12 runs

Dropping strategiesDropping strategies

Prepare 13Prepare 13thth file to test SUs... file to test SUs... Remove only headers, different loss Remove only headers, different loss

rate every 2 mins (file 14 mins long)rate every 2 mins (file 14 mins long)

Preliminary ResultsPreliminary Results Packet losses produced Packet losses produced

clear effectclear effect Average amplitude Average amplitude

differences for each differences for each loss rateloss rate

Linear relationshipLinear relationship % Amp decrease = 1.8x % Amp decrease = 1.8x

% loss rate% loss rate Header dropping Header dropping

partially successfulpartially successful Dropping every other Dropping every other

header, SU LEDs orange header, SU LEDs orange but no difference in but no difference in correlationcorrelation

Run 3 - 0.005% bunch Diffs

0.000E+00

5.000E-05

1.000E-04

1.500E-04

2.000E-04

2.500E-04

3.000E-04

3.500E-04

4.000E-04

4.500E-04

5.000E-04

30:30

31:00

31:30

32:00

32:30

33:00

33:30

34:00

34:30

35:00

35:30

36:00

36:30

37:00

37:30

38:00

38:30

39:00

39:30

40:00

40:30

41:00

41:30

42:00

42:30

43:00

43:30

44:00

44:30

45:00

Time

Am

plit

ude

Bsln= Ef-Wb SB= sub0 Pol= LL

Bsln= Ef-Jb SB= sub0 Pol=

Bsln= Wb-Jb SB= sub0 Pol= LL

y = 1.82x + 0.0917

R2 = 0.9995

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 0.5 1 1.5 2 2.5

Packet loss rate (%)

% d

ecre

ase

in c

orr

elat

ed a

mplitu

de

Future workFuture work

Enable VLBI_UDP to communicate Enable VLBI_UDP to communicate with Mk5A Streamstor cardwith Mk5A Streamstor card

Deploy and test within e-VLBI test Deploy and test within e-VLBI test runrun