FPGA Intellectual Property
PCI Express*, Networking and Connectivity, Memory Interfaces, DSP IP, and Video IP
6343 Discussions

PCI Express Hard IP on Cyclone 10GX: BARs using I/O space and debug questions

NWein
Novice
1,128 Views

I have a design using the hard PCIe core in a Cyclone 10 GX.  First time we're using it.  Quartus Prime Pro 19.3.

Two questions:

1) It appears to be impossible to set any of the BAR registers to "I/O Address Space", although it is an option in the pull-down.  The user guide says this option was only for legacy endpoints, and that was removed several versions ago.  Is it a PCIe thing that you can't set a BAR to I/O space?  In past old-style PCI designs we usually used I/O space.

2) Although our link seems to be training OK and we can access the config registers and initialize the BARs, I am seeing no transactions coming out of the rxm_bar<n> interfaces.  Are there any good suggestions on how to debug this?  I see there is a hip_status bus you can enable, but haven't found any documentation of what the signals actually do.  If possible I'd like to be able to confirm whether the read transactions from the CPU are actually being seen by my endpoint.  That would be a start at least.

Thanks!

0 Kudos
12 Replies
Rahul_S_Intel1
Employee
1,099 Views

Hi ,

Keeping the post in public.

May I know which PCIEe IP you are using is it Avalon MM Avalon ST.

 

0 Kudos
NWein
Novice
1,090 Views
0 Kudos
Rahul_S_Intel1
Employee
1,057 Views

Kindly find the inline answers for your question.

Question no:1

It is not possible to se the I/O address space in Cyclone 10 GX and  for your info the I/O address space is only supported in Legacy endpoint . The Legacy endpoint is only supported in certain variants of Arria 10 devices Avalon ST configuration.

Below is the error you will get from qsys when you select the I/O address space.

Error: pcie_a10_hip_0: The current value "I/O address space" for parameter "Type" (bar0_type_hwtcl) is invalid. Possible valid values are: "Disabled" "64-bit prefetchable memory" "32-bit non-prefetchable memory".  The parameter value is invalid under these current parameter settings:  "Port type" (port_type_hwtcl)="Native endpoint". Rule(s): bar0_type.

 

Question:2

For debugging the PCIe , we have an step by step documentation, in the below link. If you could not able to access the link kindly let me know.

https://www.intel.com/content/www/us/en/programmable/support/support-resources/support-centers/pcie-support.html

0 Kudos
Rahul_S_Intel1
Employee
1,022 Views

Hi ,

May I know still you need support

0 Kudos
NWein
Novice
1,011 Views

Well, it's still not working.  Unfortunately, my attention has been divided among a bunch of different issues so I haven't been able to concentrate full time on any of them.

I'll provide an update on this tomorrow.

0 Kudos
Rahul_S_Intel1
Employee
984 Views
0 Kudos
NWein
Novice
971 Views

Hi,

Unfortunately my attention has been on other issues and I haven't gotten much done on this.  I do have one update, and then I will be returning to focus more on this in a week or so.  The I/O space issue has been put aside, this is all about just getting PCIe to work now.

I decided to try the Windows PCIe device test software that is included with the example design.  Unfortunately, when I plugged the board in and refreshed the device list, my board never showed up.

I will note that the instructions for that software state that it assumes a particular vendor and device ID, and that the .inf file should be changed if you 

That suggests that the PCIe link was not up at all, but that does not agree with our previous results which suggests that the device was up but user-space accesses were not getting through.  So I'm not sure what was happening there.

I will note that the instructions for that software state that it assumes a particular vendor and device ID, and that the .inf file should be changed if you use something different.  However it is not at all documented how to do that.  It's easy enough to figure out how to change the vendor ID, but the device ID is a mystery.  To avoid problems, I temporarily changed my own IDs to the expected defaults, so I don't *think* that would have been a problem.

Presumably I should put some signal taps on the PCIe block and see if I can figure out the state of the link, although I'm not so sure about how to interpret all the PCIe status signals.  I'm inclined to keep working with the Windows platform, but at some point I'm going to need to confirm (somehow) that it's working properly and not contributing to my problems.  Trying to control the variables.

I will resume this hands-on work the week after next.

 

0 Kudos
Rahul_S_Intel1
Employee
958 Views

From the last description, I can assume that, the PCIe device is getting detected and not properly working in Windows Driver level.

 

If the above is the case , it will be a little challenging to debug through the Signal Tap

0 Kudos
Rahul_S_Intel1
Employee
948 Views

Hi ,

 With the above , may I close the tread. Kindly let me know if you need further assistance.

 

Feel free to open an new thread for any assistance.

 

Regards,

Rahul S

0 Kudos
NWein
Novice
934 Views

Hi,

I have finally been able to put aside other stuff and can get back to this now.

I have created a version of my design that has *only* the PCIe interface and a register set attached.  I would like to get it working with the Interop software.

However, the docs with the Interop software only mention compatibility with Windows 7.  Last time I tried it unsuccessfully it was on Windows 10.  Does the Interop software work with Windows 10?

Also, I inadvertently installed the driver (double-clicked the .inf file) *before* attempting to discover the device on the PCI bus.  Could that have caused a problem?  I would think the order wouldn't matter so much, but not certain.

0 Kudos
Rahul_S_Intel1
Employee
903 Views
0 Kudos
NWein
Novice
886 Views

I have made some further progress.

I built a version of my project that only contained the PCIe interface and a register set connected to BAR0.  My Windows10 machine was able to see the the FPGA in the device list.

I ran the Interop software, and it seemed to succeed at everything except actually accessing the registers on BAR0.  Output from the program is attached.

Subsequently, I felt sure I had figured out the problem: I had por_nreset attached to an active-high reset, so I figured it was being held in reset (although the configuration registers still worked, apparently).  I fixed that, tried again, and got the exact same results.

I had Signaltap monitoring the BAR0 avalon bus. but I was seeing some severe misbehavior from Signaltap, which made it impossible for me to do any real debugging.  As soon as I hit "Run Analysis", Signaltap would trigger immediately even though the trigger conditions were not met.  Further, no waveforms were shown prior to the trigger point, regardless of whether the trigger point was set to pre, center, or post.

I was using the coreclkout as my clock.  The STP file is attached; it shows that half-data I got, nothing before the trigger point.  Also, it was set to trigger on reset high, but you can see in the capture that reset is low.  That reset signal is inverted when passed to the npor_reset input of PCIe, so it's properly active low at that point.

I have used Signaltap many times over many years; this sort of misbehavior is rare and I must admit that when it happens I feel rather helpless to do anything productive.

0 Kudos
Reply