M.J. LeVine 1STAR HFT meeting, Sept 27-28, 2011

STAR

SSD readout upgrade

M. LeVine, R. Scheetz -- BNL Ch. Renard, S. Bouvier -- Subatech

J. Thomas -- LBNL

M.J. LeVine 2STAR HFT meeting, Sept 27-28, 2011

STAR

DAQ PC

DDL

DAQ roomOuter support

cone

South platform VME crate

RDO (1 of 8)

SlaveFPGA

SlaveFPGA

SlaveFPGA

SlaveFPGA

SlaveFPGA

Master FPGA

DA

Q in

terf

ace

TR

G in

terf

ace

VME FPGA

Fiber links

Ladder cards

VM

E in

terf

ace

Readout components

M.J. LeVine 3STAR HFT meeting, Sept 27-28, 2011

STARPrototype ladder board (inner side)

M.J. LeVine 4STAR HFT meeting, Sept 27-28, 2011

STARPrototype ladder card (outer side)

Debug connector

Optical transceiver

Flex cable connectors to ladder modules

M.J. LeVine 5STAR HFT meeting, Sept 27-28, 2011

STARInterposer

FPGA layout error• Choice of redoing the PCB• or designing an interposer

between the PCB and FPGA• Interposer design, fabrication,

and assembly: 6 months• Present status: installed and

operational

M.J. LeVine 6STAR HFT meeting, Sept 27-28, 2011

STARDebug cards (Subatech)

• USB port• JTAG (configuration)• JTAG (slow controls)

M.J. LeVine 7STAR HFT meeting, Sept 27-28, 2011

STARFake static source

M.J. LeVine 8STAR HFT meeting, Sept 27-28, 2011

STARAnalog level shifting

M.J. LeVine 9STAR HFT meeting, Sept 27-28, 2011

STARLadder card commissioning

• Using debug card– JTAG FPGA configuration– JTAG slow controls – USB simple protocol

• Using fake static source– Map analog response

• ADCs and level shifting circuitry

– Verify packing of ADC data working correctly

M.J. LeVine 10STAR HFT meeting, Sept 27-28, 2011

STARMapping analog response

• Software– Python script driving– Multiple .exe (C code)

• Time to map response for 1 ADC: 30 sec• Time to map all 16 ADCs: 20 minutes

– Disconnect/connect flex cable

• Basis for future slow controls software• SC uses JTAG header on debug card

– Will be replaced by fiber protocol

M.J. LeVine 11STAR HFT meeting, Sept 27-28, 2011

STARVerification of packing code

• USB output for ADC data• Install USB spy at output of FIFO

module

module

X16

5 MHz

5 MHz

adc 12bit16 bit serial output

adc 12bit16 bit serial output

80 MHz

80 MHz

register

16

50 MHz

FIF

O

2

JTAG

TD

Os

20

40 MHz

serializer to fiber

16 bit width4 words temp

Write enable:true on 10 clocks only

USB output USB output

M.J. LeVine 12STAR HFT meeting, Sept 27-28, 2011

STAR

-1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.40

128

256

384

512

640

768

ADCs vs. calculated

adc0 adc1 adc2 adc3 adc4 adc5 adc6 adc7 adc8

adc9 adc10 adc11 adc12 adc13 adc14 adc15 calc

Analog response for all ADCs

M.J. LeVine 13STAR HFT meeting, Sept 27-28, 2011

STARAnalog response

• Non-linear behavior of N-face needs to be understood

• Discovered we are sensitive to PS fluctuations via DAC– Will be separately regulated in production

version

M.J. LeVine 14STAR HFT meeting, Sept 27-28, 2011

STARRDO proposed layout

VME driver

Opt

ical

xc

vr

Opt

ical

xc

vr

Opt

ical

xc

vr

Opt

ical

xc

vr

Opt

ical

xc

vr

241 124 241 124241 124 241 124 241 124

Slave FPGA484 FBGA

EPCS4

VME FPGAVME

EEPROM

VME P2 connector (TRG: rows A,C) VME P1 connector

VME driver VME driver VME driver

X X

PS

he

ade

r

Slave FPGA484 FBGAEPCS4

Slave FPGA484 FBGAEPCS4

Slave FPGA484 FBGA

EPCS4

Slave FPGA484 FBGA

EPCS4

GB

ICKeep top layer clear for LVDS serial lanes (master to/from each slave)

SlaveEEPROM

AddressSW 1

AddressSW 2

Master FPGAMasterEEPROM

Reb

oot

PB

Res

et P

B

SIU

40

mm

x

16

0m

m

SIU connectors

Opt

ical

xc

vr

M.J. LeVine 15STAR HFT meeting, Sept 27-28, 2011

STARQRDO layout

Opt

ica

l xcv

r241 124

Slave FPGA484 FBGA

EPCS4

VME P1 connector

X X

PS

he

ade

r

SlaveEEPROM

Reb

oot P

B

Res

et

PB

DC regulator1.2V2.5V3.3V

+5V, GND only

USBconn

FT

24

5R

US

B

9.1 MHz oscillator

Test header (32 signals)

+5V external pwr iinput

EPCS16

LE

MO

co

nn

ecto

r

M.J. LeVine 16STAR HFT meeting, Sept 27-28, 2011

STARQRDO assembled

M.J. LeVine 17STAR HFT meeting, Sept 27-28, 2011

STARQRDO commissioning

• Orsay USB protocol implemented– Message layer on top of byte pipe– Goal: replace VME (4-byte messages)

• Problems –– Bad synthesis by Synopsys tool !!– Now resolved

• Message protocol working

M.J. LeVine 18STAR HFT meeting, Sept 27-28, 2011

STARUSB message protocol

4-byte write

4-byte read

6-bit subaddress used for register addressing

M.J. LeVine 19STAR HFT meeting, Sept 27-28, 2011

STARIntegration of ladder card/QRDO

• Generate test patterns on ladder card• Spy on incoming data via fiber with logic

analyzer

M.J. LeVine 20STAR HFT meeting, Sept 27-28, 2011

STARADC data received in QRDO

DREADY: ADC/STATUS

WORD1: Synch

Test pattern as sent (almost)

Birds-eye view

Zoom in on data phase

M.J. LeVine 21STAR HFT meeting, Sept 27-28, 2011

STARStatus words received in QRDO

Word   Value Comment (from Table 52, master document)

  0 07000 configured, OK, serdes clock used

  1 04000 deserializer lock OK

  2 00000 (no optical transceiver problems)

  3 05000 usb present, debug present

  4 00000

  5 00000

6 01000

  7 00000 ladder 0 (not yet assigned by QRDO)

serial #1 (agrees with hardware assignment on board)

M.J. LeVine 22STAR HFT meeting, Sept 27-28, 2011

STARMaster FPGA development (CR)

• Original plan for communication– 5 High speed serial lanes

• 5 slaves master

– As implemented by Altera• Requires more PLLs than available

• CR implemented his own serial receivers– Share common PLL

• Validation of new scheme in simulation– In progress– See next slides

M.J. LeVine 23STAR HFT meeting, Sept 27-28, 2011

STARSimulation of ADC acquisition (CR)

M.J. LeVine 24STAR HFT meeting, Sept 27-28, 2011

STARSimulation of status words in RDO (CR)

M.J. LeVine 25STAR HFT meeting, Sept 27-28, 2011

STARRDO roadmap

• Slave FPGA – code completed• VME FPGA – code completed• Master (TRG/DAQ) FPGA

– In late stages of simulation @ Subatech

• PCB layout expected to start 12/11• Assemble prototypes 3/12

M.J. LeVine 26STAR HFT meeting, Sept 27-28, 2011

STARStatus: documentation

M.J. LeVine 27STAR HFT meeting, Sept 27-28, 2011

STARSummary of present status

• Ladder card– 3 prototypes functional– Analog behavior characterized– Data packing verified– Beginning to look at exported data

• QRDO– Message protocol working in USB– Fiber functional– Still a work in progress

M.J. LeVine 28STAR HFT meeting, Sept 27-28, 2011

STARStatus (cont’d)

• RDO– Layout will begin after

• master FPGA simulation is complete• and QRDO debugging completed

• DAQ PC– Delivered– 2 DRORCs installed

M.J. LeVine 29STAR HFT meeting, Sept 27-28, 2011

STARComments

• We were delayed 6 months due to ladder card layout error and subsequent fix with interposer.

• We have had working ladder cards only since June, 2011

• Amazing progress since then