PCIE auto negotiation in cyclone V

[ARaja4]

Hi,

I'm trying to make gen2 x4 PCIE link using cyclone V 5CGTFD5C5 FPGA. I'm using a custom board and Intel PCIE gen2 X4 FPGA example design. The problem I'm facing is PCIE link always downgrade to either Gen1 or to X1 or X2. With cvp, it downgrades to gen2 X2, and without cvp its either gen2 X1, or gen1 X2. I have tested perstn with both hard reset (U22) and softreset. I captured ltssam state transitions as follows.

0x00 -> 0x01 -> 0x02 -> 0x07 -> 0x08 -> 0x0B -> 0x0C -> 0x0D -> 0x0E-> 0x0F -> 0x0C -> 0x0D -> 0x1A -> 0x07 -> 0x08 -> 0x0B -> 0x0F

State encoding defines available in datasheet as follows

0x00: detect.quiet 

0x01: detect.active

0x02: polling.active

0x03: polling.compliance

0x04: polling.configuration

0x05: polling.speed

0x06: config.linkwidthstart

0x07: config.linkaccept

0x08: config.lanenumaccept

0x09: config.lanenumwait

0x0A: config.complete

0x0B: config.idle

0x0C: recovery.rcvlock

0x0D: recovery.rcvconfig

0x0E: recovery.idle

0x0F: L0

0x10: disable

0x11: loopback.entry

0x12: loopback.active

0x13: loopback.exit

0x14: hot.reset

0x16: L1.entry

0x17: L1.idle

0x18: L2.idle

0x19: L2.transmit.wake

0x1A: speed.recovery

Besides, I have been using a similer custom board with 5CGTFD7D5 fpga and gen2 X4 pcie link is working fine.  I also captured ltssam state transitions for this.

0x00 -> 0x01 -> 0x02 -> 0x07 -> 0x08 -> 0x0B -> 0x0C -> 0x0D -> 0x0E -> 0x0F -> 0x0C -> 0x0D -> 0x1A -> 0x0D -> 0x0E -> 0x0F

Both projects meets timing and both PCBs are almost same which makes hard to conclude that this issue is due to electrical problem in PCB. I'm using the same example design from Intel Gen2 X4 PCIE in two projects by only changing the part number since the both FPGAs are pin compatible.

Any information to solve this issue is highly appreciated.

Thanks,

Aruna

[Nathan_R_Intel]

​Hie,

Your LTSSM states show you are able to achieve L0 state. Hence, only certain lanes cannot link up.

My primary suspect will be related to receiver detection on some of the upper lanes. Either lane 2 or 3. The case of with CvP and without CvP shows can achieve uptill x2 but not consistent.

Hence, please check the following:

i. AC coupling caps is range of 75-200nF

ii. Differential OCT (termination) is 100ohms

iii. Lanes are not swaped (p-n)

Regards,

Nathan

[ARaja4]

Hi,

Thanks for the reply. I checked with your suggestions and all of them are correct. Besides, I found that changing PCIE RX lanes IO-standard from 1.5V PCML to LVDS solved the issue mostly. (TX lanes IO-standard had to remain 1.5 PCML since its cannot be changed). Most of the times device enumerated Gen 2 X4, but sometimes it was Gen 2 X1. Interesting fact is that, when I do CVP again (without cold or worm boot PC), It enumerates successfully with Gen2 X4. Doesn't PCIE link training and speed and width negotiation happen during the PC boot process? Then, any idea that how the lane width could change between different FPGA reconfiguration via CVP?

Thanks,

Aruna

[ARaja4]

By doing the CVP again, I mean disabling the driver in device manager and re-enabling it. It seems re enumerate the PCIE.

Besides, It occurred to me that disabling both CVP and autonomous PCIE enumeration option fixed the problem. Any information about PCIE lane negotiation issues when CVP is enabled?

Thanks,

Aruna

[Nathan_R_Intel]

​Hie Aruna,

Please check my replies to your above questions:

Doesn't PCIE link training and speed and width negotiation happen during the PC boot process? Going back to PCie basics; the link training, speed and width negotiation happens when LTSSM goes through Detect->Polling-->Config-->L0.

This process happens during PC boot process. However, if any properties of the link changes (such as Tx PLL loose lock, Rx signal detect deassertion and others); the link will re-train. It will either go through entire link re-training process (Detect-->Polling-->Config-->L0 or go through recovery for speed and width change.

Then, any idea that how the lane width could change between different FPGA reconfiguration via CVP? As explaine above, when performed CvP, the link re-training will start again from Detect

From you side; it seems enabling CvP causes an issue. I will suggest referring to using CvP in Cyclone V from our user guides below to ensure you are performing CvP correctly.

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_c5_pcie_avst.pdf

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_cvp.pdf

Regards,

Nathan