The Gigabit Link Interface Board (GLIB) Paschalis VICHOUDIS CERN PH-ESE-BE

1xTCA IG, 07-March-2011

The Gigabit Link Interface Board (GLIB)

Paschalis VICHOUDIS

CERN PH-ESE-BE

xTCA Interest Group meeting – 07 March 2011


Sophie Baron

Manoel Barros Marin

Vincent Bobillier

Stefan Haas

Magnus Hansen

Markus Joos

Francois Vasey

Paschalis Vichoudis

The Team

PH-ESE-BE

GLIB project homepage: https://espace.cern.ch/project-GBLIB/public


Outline

1. INTRODUCTION

2. IMPLEMENTATION

3. STATUS & DELIVERABLES


Introduction

THE GLIB IS: an evaluation platform and an easy entry point for users of high speed

optical links

THE GLIB IS TARGETED FOR: optical link evaluation in the laboratory control, triggering and data acquisition from remote modules in beam or irradiation

tests

CONCEPT

GBT

Versatile Link

TIA

LD

PD

Laser

GBT13

TRx

FPGA

Payload

GBT GLIB

E-Link


Introduction

OVERVIEW

Mid-size double-width Advanced Mezzanine Card (AMC)

Serves as an evaluation platform and an easy entry point for users of high speed

optical links.

4 SFP+ transceiver modules

Virtex-6 FPGA with twenty 6.5Gbps transceivers.

I/O capability can be further enhanced with two FPGA Mezzanine Cards (FMC).

Gigabit Ethernet link to PC for bench-top operation.


Introduction

TYPICAL USE CASES (1/6)

BENCH-TOP: beam test setup

= SFP+= TTC FMC


Introduction


BENCH-TOP: front-end module test setup

= SFP+= TTC FMC= E-LINK FMC


Introduction


BENCH-TOP: system test setup

= SFP+= TTC FMC


GLIB

Timing/Trig to FE

FRONT-END

FPGA

GLIB configGbE:

Slow CTRLDAQ

Timing/TrigGBT:

Power Supply

FRONT-ENDFRONT-END

GBT payload10GbE:3.2Gbps/link

GBT(s)

Introduction


BENCH-TOP: system test setup [remote control/readout]

= SFP+= TTC FMC= 10GbE FMC


GLIB

HARDDISK

GBT payloadGLIB config

Timing/Trig to FE

FRONT-END

GbE:

FPGA

Backplane

CrateGLIB

FPGA

GbESlow CTRL

DAQTiming/Trig

GBT:

FRONT-ENDFRONT-END

SRAM SRAM

GbEswitch

ClockDistr.

PCIeswitch

MCHμP

Commercial MCH

Crate Management

GBT(s)

Introduction


CRATE: beam test setup

= SFP+= TTC FMC


GbEswitch

ClockDistr.

PCIeswitch

MCHμP

Commercial MCH

GLIB

CPU StorageMedium

GLIB config

Storage

Timing/Trig to FE

FRONT-END

CPU

FPGA

Backplane

CrateGLIB

FPGA

Slow CTRLDAQ

Timing/TrigGBT:

GBT payload

PCIe:

FRONT-ENDCrate Management

GBT(s)

Introduction


CRATE: system test setup

= SFP+= TTC FMC


AMC edgeconnector

Port [0:1]

Port [4:7] MGT quad

CLK1/TCLKA

JTAG circuitry(CPLD-based)Module Management

Controller (MMC)IPMI

JTAG

FPGA

GbE

Port [12:15]

4x SFP+

4

SRAM

I/O

CLK

JTAG

FMC#2

FMC#1

I2C

ClockDistribution

Circuitry

GbEPHY

20

I/O

CLK

JTAGI2C

TRx

4

160

160

SRAM

4

4

CLK2/TCLKB

CLK3/FCLKA

Port [2:3]

Port [8:11] MGT quad

M-LVDS TRx

Port [17:20]MGT quad

MGT quad

Diff. I/O pairs

ARCHITECTURE

Implementation


First Prototype

CMS xTCA workshop, 01-Feb-2010


First Prototype

CMS xTCA workshop, 01-Feb-2010


INTERFACES (1/2)

Optical

Four SFP+ cages

AMC

Port [0-1]: GbE.

Port [4-7] (Fat Pipe): PCIe x4 GEN2. Possibility to implement other protocols.

Port [8-11] (Extended Fat Pipe): PCIe x4 GEN2. Possibility to implement other

protocols.

Port [2:3]: LVDS I/O pairs. Possibility to implement other differential I/O standards.

Port [12-15]: LVDS I/O pairs. Possibility to implement other differential I/O standards.

Port [17-20]: M-LVDS.

CLK1/TCLKA: M-LVDS clock input.

CLK2/TCLKB: M-LVDS clock input/output.

CLK3/FCLKA: HCSL/M-LVDS clock input.

Implementation


INTERFACES (2/2)

FMC

2 High-pin count (HPC) sockets

160 user-specific I/Os (single-ended or differential pairs)

2 differential clock inputs and 2 differential clock outputs.

The primary FMC is accessible from the front panel

The primary FMC also provides four optional 6.5Gbps transceiver lines.

PC (only in bench-top operation)

GbE RJ45 socket (1000BASE-T).

PCIe 4x GEN2 adapter board.

Possibility to implement additional PC interfaces on the FMC mezzanines.

Implementation


FPGA

XILINX Virtex-6 LXT FPGA ( VLX130T, FF1156 package)

600 I/O that can be configured to various differential or single-ended standards.

4 Ethernet MAC and 2 PCIe Hard-IP blocks.

20 6.5Gbps transceivers (MGTs) organized in 5 quads.

~10Mb of block RAM.

Pin Compatible with higher capacity FPGAs

– VLX195, VLX240, VLX365, VSX315

Implementation


RAM & MMC

On-board memory

Two 72Mb (2M x 36bit) SRAM devices (CY7C1470 by Cypress)

Operating frequency at up to 250MHz.

Upgradeable up to 1.125Gb (once available)

Module Management Controller (MMC)

Implements the Intelligent Platform Management Interface (IPMI) for the AMC initialization and monitoring in μTCA environment.

Mezzanine card based on an ATMEL microcontroller.

Developed by CPPM (microcontroller firmware in collaboration with DESY).

Dimensions: 39mm x 20mm x 10mm.

Implementation


Implementation

CLOCK DISTRIBUTION


Implementation

MODULE MANAGEMENT CONTROLLER

MP

GA0GA1GA2

ENABLE#SCLSDA

PS0#PS1#

MMCmezzanine Board

Temperature #1

FPGAtemperature

FPGA

CPLD Control of regulators & other devices

IPMI

I2C

JTAG

GND

VCCGND

AMC front panel LEDS

I/O

AMC edgeconnector

I/OI/OI/O I/O

FrontPanel

Modulehandle

I2C

BoardTemperature #2

Diode for module detecton

I/O

I/O

I/O

I/O

Note: Not required for bench-top applications


CPLDFPGA

FPGA configuration

EEPROM

MMC

DedicatedJTAG lines

I/O

I/O

DedicatedJTAG lines

I/O I/O

I/O

AMC edgeconnector

DedicatedJTAG lines

I/O

JTAGheader #2

JTAGheader #1

Port0Port1

GbE

I2C I2CIPMI

FPGAconfig

I/O

I/O

Other devicesJTAGchain

JTAG lines

I/O

ADDRDATA

FPGA/EEPROM parallel bus

FPGAconfig

Implementation

JTAG CIRCUITRY

Important for Testing


Implementation

POWERING

LTM4601

LTM4606

LTM4606

LTM4606

12VMP

1V

1V5

3V3

2V5

MP

MAX8556

Ext. PowerConnector

MAX85561V2A

1VA

3V3

JumperSelector

3V3MP

LT3021

LT3021

LTC6902

CLKSYNC1

CLKSYNC2

CLKSYNC3

CLKSYNC4

2V5MP

1V8MP

FPGA MGT VTT

FPGA MGT VCC

MMC, CPLD I/O

CPLD I/O

CPLD core

FPGA core

FPGA I/O, various ICs, FMCs, master tracking voltage for LTMs

FPGA I/O, EEPROM I/O, various ICs, FMCs

AMC edgeconnector

1V8EEPROM coreLT3021

Power budget: Up to 0.5W for Management and 80W for Payload Power


Implementation

POSSIBLE FMC IMPLEMENTATIONS


Implementation

FIRMWARE ARCHITECTURE

SRAMinterface

FPGA EEPROMinterface

PatternGenerator

FPGA EEPROM

SRAM

SRAM

ClockDistribution controller

ClockDistribution

chipset

FPGA LOCAL BUS

GBTIP

Fat Pipe interface

(typically PCIe IP)

GbEinterface

Port[8:11]

Port[4:7]

USER LOGIC

SFP+

Port[0]

Port[1]

GbE PHY

FMC#1interface

FMC#2interface


Implementation

TESTING

- Possibility for boundary scan testing

- Use of commercial hardware for creating loopbacks and providing clock I/O for connectivity and clock distribution testing, respectively.

- Testing of optical parts (loopbacks w/ optical fibres, BER measurements

etc)

- Various testpoints on board

- Special testing software and firmware

AMC edgeconnector

Port [0:1]

Port [4:7] MGT quad

CLK1/TCLKA

JTAG circuitry(CPLD-based)Module Management

Controller (MMC)IPMI

JTAG

FPGA

GbE

Port [12:15]

4x SFP+

4

SRAM

I/O

CLK

JTAG

FMC#2

FMC#1

I2C

ClockDistribution

Circuitry

GbEPHY

20

I/O

CLK

JTAGI2C

TRx

4

160

160

SRAM

4

4

CLK2/TCLKB

CLK3/FCLKA

Port [2:3]

Port [8:11] MGT quad

M-LVDS TRx

Port [17:20]MGT quad

MGT quad

Diff. I/O pairs


** Presented at TWEPP 2010 **

Summary

Specifications: v1.9 available.

Hardware: First prototype available, on-going tests.

Testing tools: Commercial hardware, boundary scan software.

Infrastructure: Crate, MCH available, commercial AMCs available. Getting

familiar with.

Software/firmware: Development on-going.

Latest news: GLIB has entered the open hardware repository.

https://edms.cern.ch/nav/EDA-02180-V1-0

specs from GLIB website

GLIB at OHR


Deliverables

GbEswitch

ClockDistr.

PCIeswitch

MCHμP

Commercial MCH

GLIB

CPU StorageMedium

GLIB config

Storage

Timing/Trig to FE

FRONT-END

CPU

FPGA

Backplane

CrateGLIB

FPGA

Slow CTRLDAQ

Timing/TrigGBT:

GBT payload

PCIe:

FRONT-ENDCrate Management

GBT(s)

The GLIB team envisages to deliver and support software, firmware and

hardware for the following 3 setups:

- Bench-top beam test setup

- Bench-top front-end module test setup

- Crate system test setup

The required FMCs (TTC & E-Link) will also be delivered and supported.

= SFP+= TTC FMC= E-LINK FMC

Bench-top front-end module test setupBench-top beam test setup Crate system test setup

Date post:	06-Jan-2016
Category:	Documents
Upload:	blanca
View:	46 times
Download:	2 times

The Gigabit Link Interface Board (GLIB) Paschalis VICHOUDIS CERN PH-ESE-BE

Documents