PCIE interface fails in Cyclone IV GX DevKit

[Altera_Forum]

I have configured the FPGA Cyclone IV GX (Cyclone IV GX Dev Kit) with PCIE firmware example from this site: http://www.alterawiki.com/wiki/pci_express_in_qsys_example_designs 

The cpld and flash memory in the devkit were used to implement a bootloader to configure the FPGA.  When the PC is reboot with the FPGA Devkit connected via PCIe, it doesnt pass bios check and the motherboard emits three tons.  Any idea to solve this problem?  Thank you!

[Altera_Forum]

One issue with PC's is if the reboot involves a PCIe slot power cycle, there can be a race condition between the slot power being stable and the PERST, slot reset de-asserted and the FPGA loading. On a Cyclone the load is quite fast but on larger FPGA's it can be an issue ... anyway common workarounds are to try with the slot powered on do a warm restart to see if the PC will restart without power cycling the PCIe slots. Or, you can supply power from the external supply and get the Cyclone configured up ahead of restarting the PC. ... The alternative is to figure out what the PC BIOS is complaining about and work back from that. 

 I have also seen BIOS's that lock out slots PCIe slots as a security measure to avoid the addition of PCIe cards ... you may want to check if this is an issue the BIOS is complaining about.   Best Regards, Bob.

[Altera_Forum]

Hi Bob, thank you for your suggestion. I have solved this problem connecting the devkit in another pc motherboard. The software application is running ok. 

I decided to use another design reference [1] only because ease (it does not employ qsys tool and ddr memory). It uses a DMA to transfer data betwen PCIe endpoint, internal memory and host PC. The test on the DevKit was ok.  After that, I have migrated that example for two custom designed boards with PCIe interface.  The first one uses the CycloneIV GX110 whose PCIe is operating ok (this has already been tested earlier with a DMA controller for PCIe that uses the HardIP PciE Block). The second one uses a CycloneIV GX150 (the same of the cyclone IV GX devkit) and the PCIe doesnt respond. The clock signal named as core_clk_out of 125 MHz) is not being generated by the IP Core.   I have verified the ref_clk signal from host (100 Mhz), perst, external clock for the IP, tx and rx pairs connected to the fpga and all is ok.  Could give me please any suggestion to evolve in solving this problem?  Thank you, Kind regards, Jaime  [1] https://www.altera.com/content/dam/altera-www/global/en_us/pdfs/literature/an/an456.pdf

[Altera_Forum]

--- Quote Start ---  Hi Bob, thank you for your suggestion. I have solved this problem connecting the devkit in another pc motherboard. The software application is running ok. I decided to use another design reference [1] only because ease (it does not employ qsys tool and ddr memory). It uses a DMA to transfer data betwen PCIe endpoint, internal memory and host PC. The test on the DevKit was ok.  After that, I have migrated that example for two custom designed boards with PCIe interface.  The first one uses the CycloneIV GX110 whose PCIe is operating ok (this has already been tested earlier with a DMA controller for PCIe that uses the HardIP PciE Block). The second one uses a CycloneIV GX150 (the same of the cyclone IV GX devkit) and the PCIe doesnt respond. The clock signal named as core_clk_out of 125 MHz) is not being generated by the IP Core.   I have verified the ref_clk signal from host (100 Mhz), perst, external clock for the IP, tx and rx pairs connected to the fpga and all is ok.  Could give me please any suggestion to evolve in solving this problem?  Thank you, Kind regards, Jaime  [1] https://www.altera.com/content/dam/altera-www/global/en_us/pdfs/literature/an/an456.pdf  --- Quote End ---    HI Jaime,  I have not spent much time with this but for CycloneIV GX150, if you have an "equivalent" DevKit as your custom design, it is always helpful if the DevKit works as a reference. I don't know what test equipment you have but a starting point is to make sure all the requisite clocks are running and the resets deasserted. You may need a scope for the clocks. So ... check that the external reset is gets deasserted as expected.  I am assuming the configuration is getting done successfully with by USB or some on board flash ... is there a way to confirm this .  I generally can run the Cyclone IV Transciever Development Kit board on the bench , outside of the PCIe slot and configure via USB or Flash and have the clocks running.  I am sorry I can't suggest much more at this time except to leverage the DevKit which works and is the basis for your custom design. Someone more familiar with the core may be able to say what conditions gate the core_clk_out from being generated.  If there is any uncertainty about pin connectivity , normally JTAG Boundary scan can help debug.   Best Regards, Bob.

[Altera_Forum]

Hi Bob, thank you for your response.  

The external reset is deasserted to high state. The ref_clk signal from PC was analyzed with oscilloscope and is ok too.  The core_clk_out is not being generated from the ip core.  Someone could be suggest how to test if the PCI IP on FPGA is alive?   Thank you

[Altera_Forum]

Jaime, 

 If you re really stuck you can go one of three routes ...   1. Add SignalTap internal analyzer where it's clock is some suitable synchronous or >2x clock to sample other signals of interest. To do this you will need a JTAG access that I assume you have already for the FPGA configurtion.  2. If you have debug LED's , try bringing out internals to LED's that may give you an idea.  3. If 1 and 2 are not successful ... I would run the test Simulation Test Bench that comes with the original design to confirm that the new design Ok.    Can you say some more about how the FPGA is configured ? Is there a sign that the design you have does anything after the the reset is deasserted ?  The Altera reference designs use the DevKit LED's to indicate PCIe state and FPGA alive etc ... do you have these?  Best Regards, Bob.

[Altera_Forum]

Hi Bob, ok I will do it.  

I am reading the PCI Express Hard IP documentation to learn some about these signals that I have to check. The signal tap instance that comes with the reference design has the 'core_clk_out' as the sampling clock and because this clock is not being generated, nothing was sampled. I will replace this clock with a PLL output clock.  Leds on the board are indicating that FPGA is active after the reset is deasserted.  Thank you for your help!