המעבדה למערכות ספרתיות מהירות

High speed digital systems laboratoryהמעבדה למערכות ספרתיות מהירות

טכנולוגי - מכון הטכניוןלישראל

חשמל להנדסת הפקולטה

Technion - Israel institute of technologydepartment of Electrical Engineering

Full sniffer system for PCIe

Midterm presentation

Performed by: Omer Blecher , Roy Fridman

Instructor: Boaz Mizrachi


Agenda •Main Goal and Project A Goals•Current status •System characterization•Long Range modifications•First step modifications•completed tasks •Future tasks and current semester plan


Main Goal• Providing a fully operational Sniffer who is able to connect on a PCIe bus ,stream a PCIe packet to a analyzer and perform a complete packet analysis of the signals in an analysis and control PC .• General purpose of project utilizing/modifying existing components of the system and creating missing components for (full system integration).


Project A goals(1/2)• Gaining extensive theoretical knowledge of the systems current

building blocks.• Learning the platforms/protocols and algorithms needed on a

wide perspective – 1. PCI express electrical definitions and protocol

requirements.2. 8/10 bit coding. 3. UART connection.4. RocketIO transceivers.5. s/w and h/w platform design tools such as PPC,HDL

designer, Xilinx platform studio.


Project A goals(2/2)• Creating a fully detailed system characterization (h/w, s/w

and more) including future s/w and hardware modification and algorithms that will be implemented on project B.

• full system operation and “hello world” modification:

1. Hardware: moving the TGA current design from the FF672 evaluation board to the FF1152 and confirming that the design is fully active on the new board (will be elaborated ).

2. software: creating a flow that its main purpose is to display a packet (will be elaborated ).


PCFF672

Crossbar

Board

Passive

Trace

Board

Test

2.5G cables 2.5G Management cable

Current status (General)


Current status (Hardware)• CTG – creates a burst of

Randomized packet transmission with defined parameter of min and max values in the PGP BRAM.

• CTA – received the sent burst after alignment, saves only the damaged packet in the TSS BRAM.

MGT

PGP BRAM

OPB arbitrer

PPCProccesor

block

PCIe

TSS BRAM

UART Lite

OPB bus

CTG

Control

Data

ISBRAM

Host computer

PLB bus

PLB2OPBbridge

PLB BRAM

PLB BRAM

controller

CTA

PCIe

PLB arbitrer

ALG

8/10 bit encoding

Data padding

Depends on min and

max parameters

Time XXStartK char

Stamp

EndK charX

Stamp

X X


Current status (Software)• Pic (microcontroller)- configure M21121- 34x34 Crosspoint switch , write and read data from the EEPROM , Turn on or off LEDs ,enable and disable dipswitches, read status bits from the card and write control bits

• Power PC- turn on LEDs , read status of dipswitches and push buttons, read and write from the TSS and PGP memories, and send command to the pic.

• Experiment GUI- not relevant because it will be recreated to suit different needs


Features and utilization in S/W:

•Displaying packets in logical sectors

•Traffic Summary

•Link Tracker

•Timing calculations

•Bus utilization

•Advanced Search for specific packets

System characterization(Top down)


Displaying packets in logical sectors (1/5)

There are 4 PCIe transaction types:

•Memory transaction.

•I/O transaction.

•Message transaction.

•Configuration transaction.

I/O Write Request I/O Read Request I/O Write Completion

Interrupt signaling Error signaling

Power management

I/O Read Completion

Memory Write Request Memory Read Request

Memory Write Completion Memory Read Completion

Configuration Write Request Configuration Read Request Configuration Write Completion Configuration Read Completion


Packet number

TLP Memory M32Read RequesID Tag AdressLast BE

First BE

Packet number

TLP Memory M64Read RequesID TagAdress Hige

Last BE

First BE

Adress Low

Time stamp

ECRC LCRC Time stamp

ECRC LCRC

Packet number

TLP Memory M32Write RequesID Tag AdressLast BE

First BE

Packet number

TLP Memory M64Write RequesID TagAdress Hige

Last BE

First BE

Adress Low

ECRC LCRCData Time stamp

ECRC LCRCData Time stamp


Basic display method:


• Serial number: every packet gets a number which indicates the order of the arrival.

• TLP: the sequence number which was given to the packet by the DLLP.

• ECRC: Every packet can have ECRC- CRC calculation which created in the TLP.

• LCRC: Every packet has a LCRC- CRC calculation which created in the DLLP.

• Time stamp: Every packet gets a time stamp- indication of the arrival time.


General:


• Type of transaction: Memory, I/O ,Configuration and Message.

• Operation: Read or Write .

• RequestID: The I.D number of the device which create the request.

• Tag: Field to identify the request.

• Byte Enable: Indicates which byte of the DWord is a valid data bytes, and which byte is “Padding” bytes.

• Address: The register which we want to read/write from/to.

• Data: The intended data (optional).


Request Packets:


• Completion I.D: The I.D number of the device which send a complition.

• Completion Status: Indication if the device perform a successful completion.

• Byte Count: Indication of how many packets are left to complete memory read

request.

• RequestID: The I.D number of the device which create the request.

• Tag: Field to identify the origin of the request

• Low Address: Byte address of the first enabled byte of data.


completion Packets:


Traffic Summary (1/2)The s/w will consist of 5 detailed summeries: • General packet summery: Show a summery of of all packets

Group by valid and invalid packets.

• TLP packet summery: Show a summary of all the packets that

origin from the TLP, group by all type of TLP types.

• Request summery: Show a summery of all the request packets, group by the different

requester I.D.

• Completion summery: Show a summery of all the completion packets, group by the

different completer I.D.

• Error summery: Show a summery of all the damaged packets, group by all types of

error.


Traffic Summary (2/2)Display method of 2 root summaries:

Error TypeError Type DescriptionDescription

Bad ECRC’sBad ECRC’s Indication of a bad Indication of a bad transmission from the transmission from the transaction layertransaction layer

Bad LCRC’sBad LCRC’s

Indication of bad Indication of bad transmission from the transmission from the data link layerdata link layer

Alignment ErrorAlignment Error

The packet isn’t The packet isn’t aligned correctlyaligned correctly

Bad packet lengthBad packet length

Invalid length of Invalid length of packetpacket

End of a bad packetEnd of a bad packet

Get the code K30.7 Get the code K30.7 which represent an which represent an invalid packetinvalid packet

TLP TypeTLP Type DescriptionDescription

Memory RequestMemory Request Transfer data from or to a Transfer data from or to a memory mapped locationmemory mapped location

I/O RequestI/O Request

Transfer data from or to an Transfer data from or to an I/O mapped locationI/O mapped location

Configuration requestConfiguration request

Device configuration/setupDevice configuration/setup,,

Read or write a request or a Read or write a request or a completioncompletion

Message requestMessage request

From event signaling From event signaling mechanism to general mechanism to general purpose messagingpurpose messaging

Completion packetCompletion packetA reply packet for request A reply packet for request packetspackets


Link Tracker (1/2) Presenting the arrival of packets according to a time table:

• The first column is for the time since the beginning of sampling packets.

• The second column shows the number of packet.

• The third column shows packets that arrive from the graphic card.

• The forth column shows packets that arrive from the mother-board.


Link Tracker (2/2)

Time

00:00:0100:00:0200:00:0300:00:0400:00:0500:00:0600:00:0700:00:0800:00:0900:00:1000:00:1100:00:1200:00:1300:00:1400:00:15

STP

PackageID

10

11

Upstream DownStream

STP SEQ NUM Header Data

Data

Data

Data

ECRC LCRC

STP SEQ NUM Header Data

Data

Data

END

ECRC LCRC END

Basic display method:


Timing calculations

•Calculate efficient bus width in an interval of time

•Choosing the start time of the calculation:

•Choosing the end time of calculation:

•Presenting the total time:

•Presenting the bit rate of the upstream and the bit rate of the down stream:

*** Time is measured between different intervals that will be defined by

the user.

Start time00:00:05

End time00:00:38

Total time00:00:35

UpstreamDownstream

28 Mb/s

53.7 Mb/s


Bus utilization D

ata

thro

ughp

ut

Time (us)

Presenting a diagram of number of packets every 1 micro second:


Advanced Search for specific packets (1/2)

Configure the analyzer to capture a specific range of packets:

•The configuration is made by sending a mask to the modified TGA: The mask contain a sequence of bits for h/w filtering use. the main idea is that the user defines a search and a mask is created accordingly in the PC. A predefined h/w block added o the TGA will use the mask to filter wanted packets.

•Search display method: after the receiving of the packets, we will show on the screen only chosen packet or range of packets.


Advanced Search for specific packets (2/2)Capturing only the I/O request packets.

Examine the forth byte

Sending the byte “00011111”, when 11111 represent the type field, and 000 represent the bits that we ignore

Is the type filed equal to “00010”, which means I/O

request?

Saving the packaet in the TSS memory, and waiting for the next packet

Ignoring for this packet, and waiting for the next packet

No

Yes

STP (K27.7) R|Fmt|Type Continuation of the headerSequence Number

R|Fmt|Type

Type

00100

00010

Configuration

I/O Request

The chosen byte (first byte of the header)

000 11111


Long range modifications• Classifier – A component with basic

logic abilities of receiving a word of 32 bits (A) and compare a selected stream of bits in A to a known “mask” (B) and perform the following logic operations:

A=B? A<B? A>B? A<>B?

• DRAM controller+buffer - interface with DRAM is needed for minimizing trade off problem with quality of filtering and allowing wanted features that are mapped in the system characterization.

MGT

PPCProcessor

block

TSS BRAM

PLB bus

ALN*

Classifier

* This current block may not be suitable and the alignment can also be done inside the classifier

DRAM controller and Buffer

DRAM DRAM


Software first step modificationCreate a gui of sending read and write to devices on the board

Add a button run a chosen script, that configure the board devices

Create a gui that print the current status of the board

Configure the TGA to capture a single packet

Display the packat in divided to logical sectors of start K-char, time stamp, data and End K-char

Read a single packet from the TGA memory

TimeStartK char

EndK charX X X


Hardware first step modificationFull design relocation

ISE 7.1(Place&Route)

HDL Designer2005.2

(optional - needed for later modifications)

Modelsim(Simulation)

Synplify 8(sythesis)

Edk 7.1.2Full system design


Completed tasks• System characterization - s/w and h/w

• systems current building block extensive review

• PCI express electrical definitions and protocol requirements.

• 8/10 bit coding.

• UART connection.

• RocketIO transceivers.


Future tasks and current semester plan

• weekly minimal hours each –

1. until 6/4/06: 36

(3 days)

2. After 6/4/06: 18

(day and a half)

180

54

220

100

0

50

100

150

200

250

first steph/w

classifier logic char

first steps/w

basicpacket

flow

Detailed distribution


Detailed time distribution S/W:

1. Design environment review (c++) 4 weeks

2. Basic GUI creation 1 week

3. System interface with FF672 and FF1152 2 week

4.Integration (first step s/w modification) 1 week

H/W:

1. Low level system designs 5 weeks

2. first step h/w modification 3 weeks

המעבדה למערכות ספרתיות מהירות

Documents