gbtstatus.2013.02.01

8/15/2019 gbtStatus.2013.02.01

1/15

GBT Chipset Status and Production Plans

Paulo Moreira

On behalf of the GBT team

CERN

01 February 2013

http://cern.ch/proj‐gbt 1GBT Project Status

8/15/2019 gbtStatus.2013.02.01

2/15

Outline

• Radiation Hard Optical Link Architecture

• The GBT System

• GBLD Status

• GBTIA Status

•

GBTX: – Data Bandwidth

– ASIC Status

– Package Status

–

Testing

Status• GBT‐SCA Status

• GBT Building Blocks Status

• GBT‐FPGA Status

• Experiments Requirements• GBT Project Future

• Activities / Manpower / Budget

http://cern.ch/proj‐gbt GBT Project Status 2

8/15/2019 gbtStatus.2013.02.01

3/15

Radiation Hard Optical Link Architecture


On-DetectorRadiation Hard Electronics

Off-DetectorCommercial Off-The-Shelf (COTS)

GBTX

GBTIA

GBLD

PD

LD

Custom ASICs

Timing & Trigger

DAQ

Slow Control

Timing & Trigger

DAQ

Slow Control

FPGA

GBT GBT

Versatile Link

8/15/2019 gbtStatus.2013.02.01

4/15

The GBT System


FE

Module

FE

Module

P h a s e– A l i g n e r s + S e r / D

e s f o r E – P o r t s

FE

Module

E – P o r t

E –

P o r t

E – P o r t

GBT – SCA

E – P o r t

Phase ‐ Shifter

E – P o r t

E – P o r t

E – P o r t

E – P o r t

C D R

D E C / D S C R

S E R

S C R / E N C

I2C MasterI2C Slave

Control Logic Configuration(e‐Fuses + reg‐Bank)

Clock[7:0]

C L K M a n a g e

r

CLK Reference/xPLL

External clock reference

control

data

One 80 Mb/s port

I2C

Port

I2C (light)

JTAG

JTAG

Port

80, 160 and 320 Mb/s ports

GBTIA

GBLD

GBTXe‐Link

clock

data‐up

data‐down

e P L L T x

e P L L R x

clocks

8/15/2019 gbtStatus.2013.02.01

5/15

GBLD Status

• GBLD V4.0 – Available in small quantities

• Integrated in the VTRx and VTTx

– Fully functional

– Excellent performance

– Radiation hardness proved

– (Almost) production ready (V4.0):• Problem: logic synthesis partially removed

the TMR in V4.0!• This was not the case with V3!

– Version V4.1 corrects this problem:• Submission:

– 19th February 2014

• Only the digital logic will be changed: – TMR logic was successfully tested in the

past with V3

– Minimum risk

• Low power GBLD (LpGBLD) – Reduce the power consumption by 40%

– VCSEL driver only:• Lower modulation current (12 mA max)

– Prototyping:•

Final design

review:

– 1st February

• Submission: – 19th February 2013

– No risk involved, version V4.1 is the baseline!

– If successful the LpGBT will (likely) become the baseline!


4.8 Gb/s, pre‐emphasis on

Total jitter: ≈25 ps

8/15/2019 gbtStatus.2013.02.01

6/15

GBTIA Status

• GBTIA V1.0 – Fabricated in 2008:

– Fully functional

– Excellent performance

– Integrated in the VTRx

• GBTIA V2.0 – Fabricated in 2012:

– Fully functional and production ready• Available in small quantities

• Integrated in the VTRx

– Add features:• New pad layout

• Received Signal Strength Indication (RSSI) – To facilitate optical fiber/PIN‐diode

alignment.

• Higher reverse bias voltage for the PIN‐diode

• Internal voltage regulator

– Jitter performance slightly worse than the first version:

• Tj = 36 ps instead of 30 ps in V1

• But BER performance still very good !

• GBTIA V2.1

– Submitted for fabrication: November 2013

– Targets the improvement of the jitter performance

– Will become the base line solution if successful:

• No risk involved, version V2.0 is still a good

baseline solution


8/15/2019 gbtStatus.2013.02.01

7/15

GBTX Data Bandwidth

• The GBTX supports three frame

types: – “GBT” Frame

– “Wide Bus” Frame

– “8B/10B” Frame

• “GBT” Mode

– User bandwidth: 3.28 Gb/s

• Up/down‐links

• “Wide Bus” and “8B/10B”frames are only supported for the uplink

–

The

downlink

always

uses

the

“GBT” frame.

• “8B/10B” Mode

– Downlink data 8B/10B encoded

– No FEC

– User bandwidth: 3.52 Gb/s• “Wide Bus” Mode:

– Uplink data scrambled

– No FEC

– User bandwidth: 4.48 Gb/s


P h a s e– A l i g n e r s + S e r / D

e s f o r E – P o r t s

Phase ‐ Shifter

E – P o r t

E – P o r t

E – P o r t

E – P o r t

C D R

D E C / D S C R

S E R

S C R / E N C

I2C MasterI2C Slave

Control Logic Configuration(e‐Fuses + reg‐Bank)

C L K M a n a g e

r

CLK Reference/xPLL

JTAG

GBTX

e P L L T x

e P L L R x

8/15/2019 gbtStatus.2013.02.01

8/15

GBTX Status

• GBTX submitted for fabrication on the 6th

of August

• Prototypes (bare die) available: – 160 ASICs (Since December 2012)

– Possible to buy 240 pieces in 2013

• The GBTX in numbers: – ½ million gates

– Approximately:• 300 8‐bit programable registers

(all TMR)

• 300 8‐bit e‐Fuse memory

– Clock tree (chip wide):

• 9 clock trees (all TMR)

•

Frequencies:

40/80/160/320

MHz – 7 PLLs:

• RX: CDR PLL + Reference PLL (2.4 GHz)

• Serializer PLL (4.8 GHz)

• Phase‐Shifter PLL (1.28 GHz)

• xPLL (VCXO based PLL, 80 MHz)

• (2x) ePLL (320 MHz)

– 17 master DLLs:

• 9 for phase alignment of the e‐links

• 8 for clock de‐skewing

– 40 replica delay lines:

• For phase alignment of the e‐links


8/15/2019 gbtStatus.2013.02.01

9/15

GBTX Package Status

• Package: – 20 × 20 ball array

– 0.8 mm pitch

– Size: 17 mm × 17 mm

– 8 layer substrate

– Heat slug

– On package:• Quartz Crystal

• Decoupling capacitors

• Manufactured and designed by ASE: – Design verified an simulated by CERN:

• Interconnectivity

• AC performance

• Electro ‐migration

– Information flow:• CERN ↔ IMEC ↔ASE

• Long “feedback” cycle: 1 to 4 weeks!

• Five iterations were required to agree on the design!

• Status: – Package design complete.

–

About

to

start:• Tooling development

• Substrate manufacturing

– Some export license problems still to be sorted out:

• Capacitors!!!

– Forecast (conservative!?):•

Prototypes

available

for

testing:

May

2013


8/15/2019 gbtStatus.2013.02.01

10/15

GBTX Testing Status

• The Credence Sapphire ASIC

tester: – Characterization

– Production

– Three auxiliary boards are needed:

• Load board:

– Ready

• The GBTX Test Board (TB)

– Schematic ready:

– PCB under development:

» 15th March• The GBTX Link Tester Board (LTB)

– Ready

• The Stand Alone Test board (SAT):

– Dedicated to:

• Field tests

• SEU tests

– Design under development

• Schematic under development


8/15/2019 gbtStatus.2013.02.01

11/15

GBT‐SCA Status

• Analog Circuitry

– ADC block• Design development outsourced to an IP vendor based on the DCU ADC architecture

• Preliminary design database delivered to CERN, simulation results according specifications.

• Final design delivery expected in early February 2013

• Bandgap – Design ready

• DAC – Building blocks are ready from MEDIPIX‐3 project.

• Integration work is needed

• Digital Circuitry

– Core Logic – RTL code extensively redesigned during last year. Level of completeness 80%

• Need to triplicate and synthetize the RTL code.

– Development of a Testbench based on System Verilog

• ePort (with HDLC transmission protocol)

– RTL code 80% ready (Minor effort is needed to finalize triplication) – Functionality checks are O.K. and code is synthesizable

• Chip Assembly and prototype submission – Place & Route work and physical verification to be done

– Target tape out date: MOSIS MPW run in May 2013


8/15/2019 gbtStatus.2013.02.01

12/15

GBT Building Blocks (IP) Status

Available “IP” to facilitate the implementation of

e‐Link transceivers in the frontend ASICs:

• SLVS Receiver

– Wire‐bond, DM metal stack

– C4, LM metal stack

• SLVS Driver

– Wire‐bond, DM metal stack


• SLVS Bi‐directional


• HDLC transceiver

– Synthesizable Verilog

• 7B/8B CODEC

– Synthesizable Verilog

• ePLL‐FM

– Frequency Multiplier PLL

– Radiation Hard

– 130 nm CMOS technology with the DM metal stack (3‐

2‐3).

– Input frequencies: 40/80/160 MHz

– Output frequencies: 160/320 MHz regardless the input

frequency

– Programmable phase of the output clocks with a

resolution of 11.25° for the 160 MHz clock and 22.5°

for the 320 MHz clock

– Programmable charge pump current, loop filter

resistance and capacitance to optimize the loop

dynamics

– Supply voltage: 1.2 V ‐ 1.5 V

– Nominal power consumption: 20 mW @ 1.2 V ‐ 30

mW @1.5 V

– Operating temperature range: ‐30°C to 100°C

• ePLL‐CDR (under development)

– Data rate: 40/80/160/320 Mbit/s

– Output clocks: data clock + 40/80/160/320 MHz with

programmable phase

– Internal or external calibration of the VCO frequency

– Possibility to use it as a frequency multiplier PLL

without applying input data

– Programmable charge pump current, loop filter

resistance and capacitance to optimize the loop

dynamics

– Supply voltage: 1.2 V ‐ 1.5 V

– Operating temperature range: ‐30°C to 100°C

– Prototype fabrication: May 2013


ePLL‐ FM

8/15/2019 gbtStatus.2013.02.01

13/15

GBT‐FPGA Status

• GBT‐SERDES successfully implemented in FPGAs: – Scrambler/ Descrambler + Encoder/ Decoder +

Serializer/CDR• Firmware:

– KITs are available for download for:• Stratix II, IV

• Virtex 4 and 5

• Virtex 6 (with a project on ML605 evaluation card)

– On going work for:• Cyclone

• Kintex7

• Implementation of the GBT‐Wide Bus mode

• Implementation of the IC channel protocol

– The GBT is now implemented in its latency optimized version on the GLIB SVN

– Available soon for:• Stratix IVGx

• Virtex 6LXT

• Optimization studies: – Optimization of use of resources – Low and “deterministic” latency

• Not for ALTERA

• GBT‐FPGA Community: – Users have access to the sharepoint site:

• https://espace.cern.ch/GBT‐Project/GBT‐FPGA/default.aspx

– 56 registered members (most users from collaborating

institutes)• LHC experiments, but also CLIC, PANDA, GBT

• Active users are now part of the development “team”

– To register to the GBT‐FPGA community:• email to [email protected] with

– Full name

– Project

– Experiment


Altera + opto TRx ‐ 4.8 Gb/s

Xilinx ‐ 4.8 Gb/s

8/15/2019 gbtStatus.2013.02.01

14/15

Experiments Requirements


GBT Chipset Production: Q2 2014

Forecast including 20% spares (based on the above numbers):

• 10k GBTIA• 28k GBLD (+10k depending on ATLAS and ALICE)

• 28k GBTX (+10k depending on ATLAS and ALICE)

• 5k GBT‐SCA

Production costs supported by PH to be recovered later from the experiments!

LS1: 2013‐2014

LS2: 2018

LS3: 2022‐2023

Experiment Upgrade ASICs needed ASIC Quantities Systems Notes

ALICE LS2 2014 GBTIA/GBTX/GBLD 4,000 TPC / ITS / m‐Tracker To be confirmed

ATLAS LS2 GBTIA/GBTX/GBLD 288 TGC Trigger & Readout To be confirmed

LS2 GBTX/GBLD 1,280 µmegas

LS2 GBTIA 640 µmegas

LS2 GBTIA/GBTX/GBLD 150 LAr calorimeter, Trigger

LS2 GBTX/GBLD 6,000 LAr calorimeter, Trigger Not confirmed, backup solution only!

LS2 GBTIA/GBTX/GBLD 100 Tracker

CMS (prototyping) Yesterday! GBTX/GBLD 120 Forward HCAL

2015 (XMAS) 2013 GBTX/GBLD 1,500 Forward HCAL

LS2 2014 GBTX/GBLD 4,200 HCAL Barrel and Endcaps

LHCb LS2 2014 GBTIA 3,000 Almost All The vertex detector will likely use a GBT transmitter “IP”

GBTX/GBLD 14,000

GBT‐SCA 3,000

8/15/2019 gbtStatus.2013.02.01

15/15

GBT Project Future

• LpGBT: Low power GBT chip set –

Reduce the GBT chipset power consumption to ~ ¼• GBTX: ~500 mW

– Two ASICs:• Simple SERDES with reduced functionality (LpGBT – SerDes)

– Low pin count and footprint (targeting tracker developments)

– Simple parallel port

• Full GBTX functionality (LpGBTX) – General purpose

– E‐Links

– High bandwidth capability:• Downlink 4.8 Gb/s (as in the GBTX)

• Uplink two modes: 4.8 and 9.6 Gb/s –

E‐Links double the bandwidth in the 10 Gb/s mode – Technology: 65 nm CMOS

• LpGBT – SerDes

• LpGBTX

• LpGBLD (2nd generation, 10G) will be very likely kept in 130 nm CMOS?

• Serious development effort to start Q1 2014 – (Testing and production of the GBTX is the priority for 2013)

– Target:• LpGBT – SerDes prototypes in 2017

• LpGBTX prototypes in 2018

– Developments:

• LpGBLD (4.8 Gb/s): MOSIS MPW, 19th February 2013


gbtstatus.2013.02.01

Documents