We are having trouble bringing up the PCIe Hard IP on our custom board. The hard IP is set up for gen 1 x1. It appears that coreclkout is stuck low and not toggling.
We are using a FPGA project based on
The only changes we made are pin assignments for the general IO to match our custom board, and using Quartus Pro 20.4.0.
The custom board uses the Cyclone 10 GX and the hardware is set up for up to 2 lanes. The TX and RX lanes location have been verified to be correct using pin planner. They are located at bank 1C ch4 (lane0) and 5 (lane1). We also verified that the refclk is 100MHz coming from another custom board generated from a CDCM9102RHBT clock generator.
nPERST is controlled by the root complex and located at the dedicated pin (NPERSTL0) with VCCIO at 1.8, and we can see the signal goes high using signaltap
The pcie RX lane and refclk are set as CML I/O standard, and the TX lane is set at "high speed differential IO"
From our root complex, we can see that the lttsmstates changes from 0x00 > 0x01 > 0x06 > 0x01 > 0x02 > 0x03 and it stays there.
From the Cyclone 10 GX endpoint side using signaltap, the coreclockout/pld_clk, lttsmstates, and currentspeed are all stuck low.
Any suggestions on how we can go about debugging this would be greatly appreciated, or possible reasons why coreclkout is stuck low.
The coreclkout is derived from the PCIe Refclk, below are the posibilities of this problem:
Thank you for your reply, SK!
1. npor is (user_input_nrst & perstn), from signal tap it shows that it goes high. I also tried tying it straight to 1'b1, but the lttsmstates shown on the root complex are still the same
2. I used the PCIe refclk as the sampling clock, and do see serdes_pll_locked goes high. I also added the coreclkout to the signaltap, but doesn't see it switching.
Do you have any suggestions on what else we can look at?
You may not see the "coreclkout_hip" switching as the sampling clock 100Mhz may be running slower. You can create a free-running counter in your design and clock by "coreclkout_hip" if the counter keeps increasing, which means the coreclkout_hip is alive. When the serdes_pll_locked is asserted, you should see the "coreclkout_hip" is running at the expected frequency.
We added a counter for coreclkout_hip, but the counter stayed at 0.
We signaltap these signals using pcie_refclk as the sampling clock:
pld_clk_inuse = 0
serdes_pll_locked = 1
pld_core_ready = 1
reset_status = 1
perstn = 1
npor = 1 (hard tied to 1)
ltssmstate = 0
We also tried setting a signaltap with sampling clock coreclkout_hip, but the signaltap status is "waiting for clock".
When we were watching the pcie trainings on youtube, the powerpoint said to set refclk input to CML or HCSL, but on the device datasheet, it doesn't provide the I/O standards specifications like it does for the other standards (LVDS, RSDS, LVPECL, etc). Would you know the IO standards (like Vcm, VID, etc) for CML and HCSL for Cyclone 10GX?
Since we use the CDCM9102 clock generator to create the pcie refclk, we followed figure 9 (interfacing between LVPECL and HCSL) of the datasheet for the input of the refclk. Do you see any issues with setting the pcie_refclk inputs as HCSL and following that figure to shift LVPECL to HCSL?
Do you have other suggestions that we can look into?
Thank you so much for your help
Most of the use cases of PCIe will share the reference clock with the host via the PCIe edge finger. Do you mean your design requires using a separate clock source between the Host and Endpoint? Besides, as per the C10 pin connection guidance, the DC coupling is allowed for PCIe reference clock if the IO standard is selected is HCSL. Please ensure you are using DC coupling as per requirement.
Does the pld_clk is connected with coreclkout_hip in your design? Next, you probably can create a simple design, and input the 100Mhz to an fPLL, and see if you can get the 62.5Mhz/125Mhz output.
Sorry for the confusion. The refclk does come from the golden fingers for the Cyclone 10GX, but is generated with CDCM9102 on the host side. This clock generator output is set to LVPECL, so we were following the datasheet to convert it to HCSL. We could also set the output to LVDS.
The pld_clk is connected to coreclkout_hip in the design.
For the simple project suggested:
I used the "fPLL Intel Arria 10/Cyclone 10 FPGA IP" selecting "CORE" for FPLL mode, and the pll isn't locking or outputting. The pll uses pcie_refclk (REFCLK_GXBL1C_CHBN) as input, and it's set to output 125MHz.
Could this explain why my coreclkout_hip was low and ltssm states aren't changing?
Just for some clarification, should pll_powerdown be 1 or 0 (I tried both and it still didn't work)? Does 1 mean the fpll is powered down and output = 0?
The signal tap uses the pcie_refclk as sampling clock and it appears to work since I can monitor other signals.
I plan to look into my input logic levels for the refclk, but if there are other suggestions of what to look into, please do let me know
The pll_powerdown signal is high when you wanted to reset the PLL. You may do a reset after power on, for example, hold the fPLL at reset for some time, and then release the reset. Below is the example code you can use, where the clk_lock signal can connect to pll_powerdown.
reg [9:0] count_rst =0;
assign clk_lock = count_rst;
always @(posedge clk_100MHz) begin
if (count_rst!= 1)
count_rst = count_rst +1;
count_rst = count_rst;
Besides, please try to use the IOPLL Intel FPGA IP and see if you also encounter the same problem.
Here is the Cyclone 10 GX Pin connection guideline, please review your custom board to ensure it meets the requirement: