10-Nov-2005 1

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Thoughts on Data TransmissionThoughts on Data Transmission

US-ATLAS Upgrade R&D Meeting

UCSC

10-Nov-2005

A. A. GrilloSCIPP – UCSC

10-Nov-2005 2

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Old Guidelines for the Present SCT DATOld Guidelines for the Present SCT DAT

• Avoid shared parallel write busses where one failed IC tends to take out the whole bus, i.e. use serial data paths.

• Provide redundant data transmission paths onto and off of the detector.

• Employ optical transmission for all high speed communication between the detector unit and the off detector electronics.

• Saves some services volume (fibre vs. wire).

• Minimizes interference between different detector units making scale up based upon test results of a small subset more viable.

• Provide a mechanism to bypass a dead chip.

• All electrical communication between ICs or between the detector units must be via low amplitude fully differential signals.

• Minimizes electrical noise.

10-Nov-2005 3

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Present SCT Data Transmission SystemPresent SCT Data Transmission System

PATCHPANAL

SCT

hybrid

n

data 1

data 2

clock

command

LDC

DORIC3

LED

LED

PIN

PIN's

LED

R

O

D

radiation hardfibre (8m)

non-radiation hardfibre (80m)

SCT

hybrid

n+1

Apologies for the extremely old picture.VCSELs not LEDsIgnore patch panel and fibre lengths

Two data links,

1 clk, 1 comper module

Redundant

clk & com

10-Nov-2005 4

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Interconnects on Each ModuleInterconnects on Each Module

One Detector Module with Twelve ICs

2 serial outputs2 serial outputs

“N” normal

“B” bypass

Clk & Com not shown but bussed in parallel to all ICs.Likewise for redundant Clk & Com.

10-Nov-2005 5

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Transmission Data Rates & MultiplexingTransmission Data Rates & Multiplexing

The natural clock frequency of the readout ICs is the beam crossing frequency or perhaps x2 the beam crossing frequency.

• Now at BC Frequency = 40 MHz

• Could become 67 MHz, 80 MHz or even 100 MHz

Optical transmission now typically operates at GHz rates

• If such rates are attainable with rad-hard components, fibre count will be reduced.

Operating the optical transmission at GHz rates implies multiplexing

• Difficult to operate electrical transmission at those rates

Multiplex several serial chains with several ICs in each chain (figure).

The number of chains and the number of ICs in each chain will depend upon the details of the detector unit (e.g. stave).

• Higher rates for each chain will add flexibility for layout options.

• Each chain may require phase adjustment.

10-Nov-2005 6

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Possible Multiplexed ArchitecturePossible Multiplexed Architecture

10-Nov-2005 7

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

The Present Bypassing SchemeThe Present Bypassing Scheme

The present bypass scheme has worked well but some difficulties:

• The layout of the hybrid circuit has added complexity with the addition of four extra traces for each IC. The differences in capacitive load of the normal and bypass traces required slightly different designs for the two drivers/receivers in order to keep power to a minimum.

• The connections to bypass an IC at the edge of the hybrid required very long traces. These long traces required a much more powerful differential driver (i.e. more power consumed) and these signals became the function limiting the operational speed of the IC.

• Some of the redundant paths so complicated the topology of the hybrid layout that one was not implemented precluding the bypassing of one IC.

10-Nov-2005 8

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

A Simpler Bypassing SchemeA Simpler Bypassing Scheme

Define two directions for the data flow (e.g. left and right)

If an IC fails, the data flow direction of ICs are arranged such that all to the left of the dead IC send data to the left and all to the right of the dead IC send data to the right (figure).

Both left and right drivers drive only short path to adjacent IC

Add third driver (powered only when needed) to drive long line to multiplexer or displaced section of chain. (Equivalent of master driver on present IC.)

Number of drivers and hybrid traces are not reduced but trace routing is simpler and drivers are more efficiently matched to loads.

10-Nov-2005 9

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Bypassing a Dead ICBypassing a Dead IC

10-Nov-2005 10

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Configuration Read BackConfiguration Read Back

The present SCT readout IC, the ABCD, does not allow for read-back of its configuration registers.

• Tests for single event upset in a particle beam indicate that we should not have a problem if we periodically reload the registers.

• The higher fluences of the upgraded ATLAS and the smaller geometries of the IC structures could create a worse problem.

• Cannot tell in situ what the real error rate is; the re-load rate must be set to the worst case estimate.

Upgrade IC should provide for read back of configuration registers.

• To minimize fibres, must use same output as for data.

• Protocol developed for the SCT readout provided for such configuration read back but not implemented because of space required for extra circuitry. Need to re-examine the protocol to see if it can be improved, then implement in the new IC design.

10-Nov-2005 11

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Command ProtocolCommand Protocol

Present command protocol developed to allow triggers, short commands and long configuration commands using the same input stream.

A primary 3-bit identifying sequence was chosen in order to allow consecutive triggers to appear with only a gap of two beam crossing periods. (An ATLAS trigger requirement)

Later relaxed to a longer gap. The Pixel IC made use of a longer bit stream making it more immune to bit flips in the command sequence.

Keep longer pattern as Pixels (if still allowed by trigger) or increase data rate to allow longer sequence as long as SEU rate acceptable.

• If beam crossing rate increases, the command data rate must increase proportionally.

10-Nov-2005 12

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Encoding of Clock and Command SignalsEncoding of Clock and Command Signals

Clock and Commands carried on one fibre with biphase-mark encoding

• Signal decoded on module into separate LVDS signals

• Decoding could be in same IC as multiplexer

• Would allow path for configuring multiplexer

Possible to multiplex more than one command stream on the same fibre.

• Reduce time for frequent re-configuration (if necessary)

• But … higher data rates will aggravate SEU problems

Need separate electrical drivers for each chain of ICs

• Isolate failures

• Possible need for phase adjustments for each chain

Could be advantage to maintain encoding in electrical chains and decode at each IC (figure).

• Reduce number of drivers and dual traces

10-Nov-2005 13

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Keeping Clk/Com Encoded in E-StreamsKeeping Clk/Com Encoded in E-Streams

10-Nov-2005 14

Thoughts on Data Transmission A.A. GrilloSCIPP-UCSC

ATLASATLAS

Redundant Clock & Command SignalsRedundant Clock & Command Signals

Important to maintain redundant clock and command signals.

Present scheme of sharing clk/com fibre of adjacent module as redundant stream creates some problems.

• Electrically couples adjacent modules destroying isolation

• Electrical connection between modules is mechanically awkward.

Preferable to have two clk/com fibres per detector unit to maintain electrical isolation.