14
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 1 DAQ BNet
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 1
DAQBNet
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 2
DAQBNet Currently investigated structure
switch n × n • • • • switch n × nn
n * (n - 1) / 2 bidirectional connections
• • • • • • • n
CNet PNet
n - 1
n - 1 ports n - 1 ports
H: histogrammerTG: event taggerHC: histogram collectorBC: schedulerDD: data dispatcherED: event dispatcher
TG/BC
DD/ED
CNet PNet
DD/ED
HCNet
DD/HC active buffer
BNet controller
n=4 : 16x16
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 3
DAQBNet Simulation with SystemC
• event generator
• data dispatcher (sender)
• histogram collector
• tag generator
• BNet controller (schedule)
• event dispatcher (receiver)
• transmitter (data rate, latency)
• switches (buffer capacity, max. # of package queue, 4K)
Running with 10 switches and 100 end nodes.
Simulation takes 1.5 *105 times longer than simulated time.
Various statistics (traffic, network load, etc.)
Modules:
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 4
DAQBNet Some statistic examples
single buffers excluded!
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 5
DAQBNet Scheduling strategy
• use data channels for meta data
• traffic shaping by schedule
• calculate schedule in advance
• data transfers are triggered at absolut times
• requires deterministic behavior of network
• delivery control by buffer monitoring
• no transfer acknowledge
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 6
DAQBNet Factorisation of 32x32 switch
1-1
1-2
1-3
1-4
1-5
1-6
6-1
6-2
6-3
6-4
6-5
6-6
1-1
1-2
1-3
1-4
1-5
1-6
6-1
6-2
6-3
6-4
6-5
6-6
• • • •
2 layers of 6 6x6 switches
Variance in latency increases!
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 7
DAQBNet Factorisation of 32x32 switch
Variance in latency increases!
factorized switch 1
factorized switch 2
2 switches
1 switch 4 switches
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 8
DAQBNet Synchronization of senders
• Extra time clock distribution (~µs)
• or from concentrator (must then be synchronized)
The active buffers must have a common clock to start transfers at a given absolut time.
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 9
DAQBNet Broadcast problematic
• Broadcast through factorized switches
• How to guarantee packet delivery?
Broadcast is needed to distribute event tags andschedule start time to all data dispatchers.
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 10
DAQBNet Error detection
• How to ensure that all data arrived?
• How to recover if not?
• Recover from component fail (or removal)?
• When emergency?
BNet controller gets buffer status from all DD/EDs.These are sent according schedule.
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 11
DAQBNet Network performance measurements
chassis IXIA 1600T
IXIA testbed components
10 GE load module (traffic generator/analyzer)
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 12
DAQBNet Network performance measurements
TALLY: measurement of Force10 E1200
Point to point
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 13
DAQBNet Topics for investigations
• Event shaping
• Separate meta data transfer system
• Addressing/routing schemes
• Broadcast
• Synchronization
• Determinism
• Fault tolerance
• Real test bed
GSI, Mar 2005 Hans G. Essel, Sergey Linev: CBM - DAQ BNet 14
DAQBNet
