I'm trying to make gen2 x4 PCIE link using cyclone V 5CGTFD5C5 FPGA. I'm using a custom board and Intel PCIE gen2 X4 FPGA example design. The problem I'm facing is PCIE link always downgrade to either Gen1 or to X1 or X2. With cvp, it downgrades to gen2 X2, and without cvp its either gen2 X1, or gen1 X2. I have tested perstn with both hard reset (U22) and softreset. I captured ltssam state transitions as follows.
0x00 -> 0x01 -> 0x02 -> 0x07 -> 0x08 -> 0x0B -> 0x0C -> 0x0D -> 0x0E-> 0x0F -> 0x0C -> 0x0D -> 0x1A -> 0x07 -> 0x08 -> 0x0B -> 0x0F
State encoding defines available in datasheet as follows
Besides, I have been using a similer custom board with 5CGTFD7D5 fpga and gen2 X4 pcie link is working fine. I also captured ltssam state transitions for this.
0x00 -> 0x01 -> 0x02 -> 0x07 -> 0x08 -> 0x0B -> 0x0C -> 0x0D -> 0x0E -> 0x0F -> 0x0C -> 0x0D -> 0x1A -> 0x0D -> 0x0E -> 0x0F
Both projects meets timing and both PCBs are almost same which makes hard to conclude that this issue is due to electrical problem in PCB. I'm using the same example design from Intel Gen2 X4 PCIE in two projects by only changing the part number since the both FPGAs are pin compatible.
Any information to solve this issue is highly appreciated.
Your LTSSM states show you are able to achieve L0 state. Hence, only certain lanes cannot link up.
My primary suspect will be related to receiver detection on some of the upper lanes. Either lane 2 or 3. The case of with CvP and without CvP shows can achieve uptill x2 but not consistent.
Hence, please check the following:
i. AC coupling caps is range of 75-200nF
ii. Differential OCT (termination) is 100ohms
iii. Lanes are not swaped (p-n)
Thanks for the reply. I checked with your suggestions and all of them are correct. Besides, I found that changing PCIE RX lanes IO-standard from 1.5V PCML to LVDS solved the issue mostly. (TX lanes IO-standard had to remain 1.5 PCML since its cannot be changed). Most of the times device enumerated Gen 2 X4, but sometimes it was Gen 2 X1. Interesting fact is that, when I do CVP again (without cold or worm boot PC), It enumerates successfully with Gen2 X4. Doesn't PCIE link training and speed and width negotiation happen during the PC boot process? Then, any idea that how the lane width could change between different FPGA reconfiguration via CVP?
By doing the CVP again, I mean disabling the driver in device manager and re-enabling it. It seems re enumerate the PCIE.
Besides, It occurred to me that disabling both CVP and autonomous PCIE enumeration option fixed the problem. Any information about PCIE lane negotiation issues when CVP is enabled?
Please check my replies to your above questions:
Doesn't PCIE link training and speed and width negotiation happen during the PC boot process? Going back to PCie basics; the link training, speed and width negotiation happens when LTSSM goes through Detect->Polling-->Config-->L0.
This process happens during PC boot process. However, if any properties of the link changes (such as Tx PLL loose lock, Rx signal detect deassertion and others); the link will re-train. It will either go through entire link re-training process (Detect-->Polling-->Config-->L0 or go through recovery for speed and width change.
Then, any idea that how the lane width could change between different FPGA reconfiguration via CVP? As explaine above, when performed CvP, the link re-training will start again from Detect
From you side; it seems enabling CvP causes an issue. I will suggest referring to using CvP in Cyclone V from our user guides below to ensure you are performing CvP correctly.