Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

ELMB NetworksELMB Networks

• CAN/CANopen Interoperability of the ELMB• Usage of the ELMB in ATLAS• ELMB Networks• Full Branch Test• Conclusions • Future work

Outline

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

Software Interface Communication LayerSofting CANopen OPC Server CAN-AC2-PCI CANopenBeckhoff TwinCat SoftPLC FC5120 CAN & CANopenVector CANalyzer CAN-AC2-PCI CAN & CANopen

Custom CANopen OPC Server NICAN-II CAN & CANopen

120 120

Beckhoff Wago

Compliance of the ELMBCompliance of the ELMB

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

Aims of the Test:

• TileCal Cooling and LV: 8 ELMBs• Pixel Cooling: 7 ELMBs• COMPASS 10 ELMBS

Currently

• Pixel Cooling : 12 ELMBs• MDT 10 ELMBs• COMPASS 16 ELMBs

Next Year

• 5000 ELMBs in ATLAS– 1200 MDT (test case, should cover all subdetectors)– 1000 RPC– 800 TGC

ATLAS

• Powering• Performance:

• tuning of parameters• identifying bottlenecks

50 mm

66

mm

• Reliability• Robustness• Definition of recovery procedures

Usage of the ELMBUsage of the ELMB

>> Give guidelines and recommendations to set up ELMB networks in ATLAS <<

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

• The maximum number of ELMBs per bus is 64 for addressing reasons (6 bits), i.e. 4096 Analog Inputs• However, the number of ELMBs per branch must be a compromise:

– Cost; a higher number of ELMB modules per bus => a lower number of buses and PCs– Affordable number of channels to be lost in case of a branch failure.– Performance and reliability of the readout system.– Powering of the CAN network (PS placed in USA15).

120 120

120 120Branch_1

Branch_2

ELMB NetworksELMB Networks

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

SCX

1U

SA15

Front-End SystemB

ack-End System

Local Area Network

Partition 3Supervisory only Partition m

UX

15

Partition 2

Common Infrastructure

Local Control Stations (LCS)

Partition 1

Expert Workstations

ServerOperation

Subdetector 1 Subdetector 2a Subdetector n

DetectorSub-system

SensorsActuators

Fieldbus 200m

ELMBElectronicRack

ELMBELMB

ELMB

Rack PC

Power Supply

CoolingInterlock

Box

Test

Rack PC

ELMB

Alarms

CFS

HVBarrel

Architecture of DCSArchitecture of DCS

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

ELMB Full Branch TestELMB Full Branch Test

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

ELMBELMB Full Branch Test

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

ELMBELMB Full Branch Test

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

Set UpSet Up

• Only one PS used.• Digital and CAN parts of the ELMB and NI-CAN card powered via the bus.• Analogue part powered from the digital (Power monitored by a scope). Normally powered from the FE equipment• Bus and Interface card power lines were de-coupled (Allows for independent reset of both elements).

120

0x3F 0x30

CANbus 200 m

16 V PS

LCS

120

D9 Connectors

Powering

Cable Resistance 37.5 Ohm / Km 7.5 Ohm in 200mVoltages

Currents (mA) VPS_CAN (V) 16CAN 16x20 320 Velmb_16_CAN (V) 11.05ADC 16x10 160 VPS_Dig+Ana (V) 16Analogue 16x15 240 Velmb_16_Dig+Ana (V) 8.26

GroundingDrop G_CAN (V) 2.63 Drop DP+AP (V) 3.88Drop P_CAN (V) 2.31 Drop DG+AG (V) 3.88Difference (V) 0.32 Difference (V) 0

Digital + Analogue Voltage drop during ADC activity (V) 1.44

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

• 16 x 8 = 128 digital input lines (Sync + Async), 16 x 2 x 8 = 256 digital output lines (Async)• Input and output lines interconnected• 16 x 64 = 1024 Analogue channels (Sync)• ELMBs reprogrammed to bypass filtering in OPC server and ensure maximum data volume transfer to PVSS-II (Worst possible case)

ELMB I/O FunctionalityELMB I/O Functionality

t

SYN

C_1

ELM

B_3

0

ELM

B_3

1

ELM

B_3

F

ELM

B_3

0_0

ELM

B_3

1_0

ELM

B_3

F_0

SYN

C_2

ELM

B_3

0

SYNC Interval

...

...

ELM

B_3

0_1

ELM

B_3

1_1

...

Digital Signals(Higher Priority)

Analogue Channels(Lower Priority)

ELM

B_3

F_1

ELM

B_3

F_63

Bus Cycle

...

If adc = 32.5 Hz and Bus Speed = 125 kbit/s• ~ 0.8 ms => Time difference between messages on the bus; ~ 30.8 ms => Time needed for ADC conversion

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

CAN Analyzer Diagnostic ToolCAN Analyzer Diagnostic Tool

SYNC

DigitalSignals

AnalogueChannels

Bus load

No. of CAN frames

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

Bus LoadBus Load

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

Sync Interval vs adc for 16 ELMBs @ 125 kbits/sTransfer Rate 100%, best case

0

5

10

15

20

25

30

35

40

0 5 10 15 20 25 30 35adc (Hz)

SYN

C In

terv

al (s

)

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

SYNC IntervalBus PeriodBus Period - Sync Interval

Time diff Bus Period - SYNC Interval (s)

PerformancePerformance

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

Data get lost at higher data rates when SYNC Interval -> bus cycle

Bottlenecks:• NI CAN interface card (buffer overflow) => Need for a new interface (cost and characteristics).• PVSS archiving (CPU consumption) => Better distribution of the tasks performed by SCADA.

Transfer Rate Bus - PVSS-II vs Sync Intervaladc=32.5 Hz, OPC Update Rate 1 s, 125 kbits/s

60

65

70

75

80

85

90

95

100

105

02468101214161820222426283032

Sync Interval (s)

Tran

sfer

Rat

e (%

)

Performance (II)Performance (II)

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

Comparison of the bus load peak at different bus speeds

0

5

10

15

20

25

30

0 5 10 15 20 25 30 35adc (Hz)

Bus

Loa

d Pe

ak (%

)

125 kbits/s250 kbits/s

Comparison at different Bus SpeedsComparison at different Bus Speeds

Results shown at 125 kbits/s were reproduced, i.e. 100 % transfer rate for a bus having 16 ELMBs and SYNC Interval equal to 4 s.

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

• 200 m long ELMB Branch with up to 16 nodes successfully operated.

• I/O channels of the order of magnitude of some subdetectors in ATLAS: 1024 AI, 256 DO, 128 DI

• Preliminary studies of the powering has been carried out.

• The test tried to reproduce the “worst possible case” => All ELMBs transmitting at the same time.

• The system has shown excellent performance:Transfer rate of 100% for

• messages of different priorities on the bus

• and of different transmission types.

• Maximum readout rate 4s at 32.5 Hz.

• Results are reproducible at 250 kbits/s => Less bus load (but also smaller bus length).

• Several issues were identified at higher adc when SYNC Bus Cycle

• Overflows in the read buffer of the NI-CAN interface

• Strange behavior of PVSS-II manager as a function of the OPC update rate.

• High CPU consumption by the PVSS archiving manager

However, the CANalyzer showed the correct operation of all ELMBs on the bus.

ConclusionsConclusions

Fernando Varela

ATLASDCS

ELMB PRR, CERN, March 2002

• Optimize powering of the bus => minimize voltage drop in the CANbus cable.

• Window-checking in the ELMB.

• Increase number of ELMBs up to 32 or 64.

• Several buses per PC.

• Test of network supervision and management -> Definition of error recovery procedures

• Long-term operability of the system in TCC2.

• Automatic power cycle of the bus needed due to SEEs (see below).

Future WorkFuture Work