pci express analyzer

1

PCI Express Analyzer

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

2

הטכניון - מכון טכנולוגי לישראל

Technion - Israel institute of technology

Final Presentation

Samuel Amir , Danny Volkind

Mr. Orbach Mony

Analyzer Core

3

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Agenda

• PCI - Express – Reminder

• Link and Lane Training - Overview

• Project Goals

• Project features and capabilities

• PCI-Express Generator - Brief

4

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Agenda (cont.)

• Project Block Diagram

• Demonstration

• Future Improvement Procedures

5

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

PCI - Express

Reminder

6

PCI-Express Topology

Ref. Clock Ref. Clock

Dev

ice

A Device B

TX+TX-

RX+RX-

TX+TX-

RX-RX+

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

• Differential

• Dual Simplex

• Speeds of 2.5Gb/sec

7

Packet Formation

Packet formation reflects

layered architecture.

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Transaction

Data Link

Physical

Transaction Layer

DataHeader

Data Link Layer

CRCSequenceNumber

Physical Layer

FrameFrame

8

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

8/10b conversion (4.2.1)

8b/10b Encoded Byte Value

Byte Value 00H

• Convert each byte to a 10bit character according to a pre-

defined table.

• Extra characters are used as control characters or not used.

• Embedded clocking-> No need to add clock traces.

• Error detection (running disparity).

• DC balancing

• Reduces ISI.

9

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

PCI Express scrambling (4.2.3)• Assures no constant pattern is transmitted.

• Spread the energy transmitted in one frequency to

different frequencies-> reduces EMI! (and gets FCC

approval)

• Only data characters are encoded.

• Encoding is done using a linear feedback shift register.

• Decoding is done using the same process at the receiver

side.

10

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Frequency Mismatch (4.2.7)

How does the protocol cope with the fact that each

device is feed by a different clock source?

Ref. Clock Ref. Clock

Dev

ice

A Device B

TX+TX-

RX+RX-

TX+TX-

RX-RX+

11

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Frequency Mismatch (4.2.7)

Input @ 2501MHz Output @ 2500MHz

Input Shift Register is filling up faster than it is emptied!

Input @ 2500MHz Output @ 2501MHz

Input Shift Register is emptied faster than it is filled!

If we take 600ppm difference between clocks, the

transmitter and receiver clocks can shift one clock

every 1666 clocks!

12

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Frequency Mismatch-Solution• The protocol issues a SKIP ordered set that can be

skipped so that the input shift register can be partially

cleared!

• The SKIP ordered sets insertion time as dictated by the

protocol is between 1180 and 1538 symbol times.

• Calculated allowed deviance : ± 300ppm !

• This is known as “clock tolerance compensation

mechanism”.

13

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Link & Lane

Training

14

Link and Lane Training

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Lane - a single set of differential RX TX pairs

Link - a collection of lanes connecting two

PCI-Express Devices.

Dev

ice

A Device B

TX+TX-

RX+RX-

TX+TX-

RX-RX+

x1 Lane wide Link

15

Link and Lane Training (cont)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Training - a process aimed at turning a collection of

available lanes into a properly functioning link.

Elements established during training :• Physical Level

• SERDES lock.• Symbol alignment.

• Link Level link configuration• Link data rate• Link width• Etc.

16

Link and Lane Training (cont)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Configuration

Disable

Polling Loopback

Detect

HotReset

Recovery

L0s

L2

L3

L0(Full On)

L2/L3Ready

L1

Link Training States

Link Power Manaement States

Active Power Management States

To Detect

17

Training Ordered Sets

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

• Group of 16 symbols.• Used During Polling State (TS1,TS2)• Establish alignment• Exchange Physical layer parameters• Not scrambled!

18

Training Ordered Sets (cont)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

TS1

19

Fast Training Sequence

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Used to re-establish bit and symbol lock when transitioning out of the LO power management state.

K28.1K28.1K28.1K28.5

FTS pattern

20

Skipped Ordered Set (CTC)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Compensate for differences in frequencies between bit rates at two devices sharing a mutual Link (Clock Tolerance Compensation)

Skipped Ordered Set pattern

K28.0K28.0K28.0K28.5

21

Electrical Idle

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

• Steady state condition• Transmitter differential pair held at fixed value• Must remain at this state at least 20nsec• Must attempt to detect a receiver within 100msec

22

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

DLLP packet

DLLP ContentsDLLP Contents44bytebyte

CRCCRC2byte2byte

Data Link LayerData Link Layer

FrameFrame1byte1byte

FrameFrame1byte1byte

Physical Layer

23

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

TLP packet

Transaction LayerTransaction Layer

Data0-4Kbyte

Header12/16byte

Data Link LayerData Link Layer

LCRCLCRC4byte4byte

Sequence#Sequence#2byte2byte

Physical Layer

FrameFrame1byte1byte

FrameFrame1byte1byte

24

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Packet Header

FmtFmt

Address/RoutingAddress/Routing

TypeType Requestor IDRequestor ID RsrvRsrv Traffic ClassTraffic Class

LengthLength AttrAttr TagTag

16/1216/12ByteByte

RsrvRsrv Byte EnablesByte Enables

Data PayloadData PayloadIndicator and Indicator and

16/12B header flag16/12B header flag

Memory, I/O, Config,Memory, I/O, Config, Message; RequestMessage; Request

or Completionor Completion

Bus#, Device#, Bus#, Device#, Function#,Function#,

Virtual ChannelVirtual ChannelSupportSupport

TransactionTransactionTagTag

ReservedReservedfor futurefor futureexpansionexpansion

Attributes:Attributes:Snoop,Snoop,

OrderingOrdering

First DW BE;First DW BE;Last DW BELast DW BE

RequestedRequestedLength or Length or

Payload SizePayload Size

Header

Transaction LayerTransaction Layer

Data

Data Link LayerData Link Layer

CRCSequenceNumber

Physical LayerPhysical Layer

FrameFrame

25

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Project Goals

26

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Project Goal Overview• PCI-Express packets capturing at wire speed of 2.5Gbps

• Selective filtering – work modes

• TLP analysis – header based filtering

• Simple register based user interface

• RS232 accessible.

27

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Project Features

&

Capabilities

28

Project features

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

• Receiving PCIe communication at 2.5 Gbps.

• Full line synchronization capability according

to PCIe spec.

• Handling of realignment and clock tolerance

compensation events on the fly.

• Invalid symbol filtering.

29

Project features (cont.)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

• Detection of and field extraction for all PCIe training

sequences.

• Selective descrambling of the received data.

• Accumulative user-controlled counters for statistic

purposes.

• Data stream marking and preliminary analysis.

30

Projects features (cont.)• 6 Available work modes:

• Wire Speed Capture.

• Capture TS1 / TS2

• Capture DLLP

• Capture TLP

• Selective TLP capture

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

31

Projects features (cont.)•TLP filtering versatility:

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

TLP Type field 5 Bit

TLP format field 2 Bit

TLP Traffic Class 3 Bit

TLP length 10 Bit

TLP Attribute 2 Bit

TLP Tag 8 Bit

TLP Byte Enables or Message 8 Bit

TLP Sequence num 12 Bit

TLP Bus num 8 Bit

TLP device number 5 Bit

TLP function number 3 Bit

TLP poisoned packet 1 Bit

TLP Type field Mask 5 Bit

TLP format field Mask 2 Bit

TLP Traffic Class Mask 3 Bit

TLP length Mask 10 Bit

TLP Attribute Mask 2 Bit

TLP Tag Mask 8 Bit

TLP Byte Enables or Message Mask 8 Bit

TLP Sequence num Mask 12 Bit

TLP Bus num Mask 8 Bit

TLP device number Mask 5 Bit

TLP function number Mask 3 Bit

TLP poisoned packet Mask 1 Bit

32

Project features (cont.)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

•Storage

• Fully user controlled dynamically allocated

16K x 32 memory.

• Captured data is saved with preliminary

analysis.

• Bursts and capture events are separated

allowing selective extraction.

33

Project features (cont.)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

• LCD Interface

• Built-in LCD micro-controller implementation.

• 1 Kbyte user memory for display commands.

• Display file can be loaded on the fly.

34

Project features (cont.)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

• User Interface

• RS232 serial link operating at 115.2Kbps.

• Simple comm. protocol implementation

allowing read and write commands accessing

32bit address space with 16bit data.

• All the core features are accessible.

35

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Generator - Overview• Capable of generating all PCIe traffic.

• Fully user-controlled.

• PCIe compatible including clock tolerance

compensation simulation.

• Can be set to continuous or single shot mode.

36

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Generator - Overview

37

Basic Block Diagram

PCIe

MCL PDM

SDMPLA

Modular Single-channel UnitSDRAM

MCSU

AD

DR

DA

TA

CCIU

LQAM MGR

Gigabit Receiver1

1

Decryption Module2

Wrap Filter3

Packet Filter4

MSU Control5

Link Assessment6

2

34

5

6 Memory Controller7

Central Controller8

7

8

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Physical

Physical

Transaction

Physical

Physical

Data Link

Transaction

Physical Data Link

38

Modular Single-channel Unit

MGR

SDM

LQAM

MCSU

PLA

PDM

PCIe

MCL

CCIU

Modular Single-channel Unit#2

PCIe

SDRAM8Mx16

SDRAM8Mx16

Addr

Data

Control

Data

Top Level

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

39

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Project Block Diagram

40

PDM – LFSR

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

8 bit values generated by LFSR for repeated data value of 0

41

PDM – LFSR (cont)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Scrambling spectral power distribution for repeated data value of 0

42

PDM – LFSR (cont)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Reset Value (Comma)

Figure 1

43

PDM – LFSR (cont)

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

LFSR1_out

LFSR2_out

44

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

Demonstration

AnalyzerAnalyzer

Frame

Frame

Dev

ice

AD

evice B

45

Future Developments

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

• Additional functions.

• User interface development.

Open Code flexibility opens the door for development

46

Closing Words

Thank You !

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

47

Additional informationCertain images adopted from PCI-SIG PCI Express™ Architectural Overview Presented at the 2002 PCI-SIG Developers Conference and Intel Developers Forum, Fall 2001.

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

48

Project Development Process

High Speed Communication Fundamentals

Semester A

Final ConceptRequirements Doc

PCI ExpressArchitecture Concepts

market surveyCurrent Available Products

Existing Infrastructure

Constructing Analyzer CoreBuilding Blocks

Semester B

Analyzer CoreDevelopment report

Debugging & TestingEach block

Final Core Integration

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות