PCIe root port unable to read endpoint RAM

[sglow1]

I've got two boards, both using Cyclone IV FPGAs with hard PCIe IP blocks.  One board is a PCIe endpoint and is working great.   I can use it in a PC and it's been working well for years.

The other board functions as a PCIe root port and is intended to be used to program and test the first board.  I'm having a lot of problems configuring that root port though and so far have been unable to successfully read memory from the endpoint.

I have been able to read and write configuration registers on both the root port (using type 0 config TLPs) and on the endpoint (using type 1 TLPs).  Any time I issue a memory read TLP to the endpoint I get a response with no data and completion status of 1 (Unsupported request).  I've tried both 32-bit and 64-bit reads and both act the same way.

I know I need to configure both the root port and endpoint config registers and I suspect that this is where my problem lies.  Here's what I've done, can anyone tell me what I'm missing:

• Set the root port device control register (0x04) to 6 to enable bus mastering and memory access.  I set the slave port device control register to 2 to allow memory access.  Note that I always read zero back from these registers no matter what I set them to which is odd.
• Set root port register 0x18 to 0x0100.  This sets the secondary bus number to 1 which tells the root port to pass messages addressed to that bus on to the endpoint.
• Set the endpoint base address register at address 0x10.  I've tried various addresses but none seem to work.
• Set the PCIe device capabilities register at 0x88 for both the root port and endpoint to a value of 0x0010.  That disables all error reporting, enables relaxed ordering, sets max payloads and requests to 128 bytes, disables phantom functions, etc.  This is based on the Altera documentation and example code.
• I've tried to set the prefetchable memory base and limit registers in the root port at address 0x24 but for some reason this seems to be read only.  The register reads 0x00010001 and can't be set.  That LSB is read only and according to the PCI-PCI bridge spec it should be zero so I'm not sure why it's set by the IP core.
• I can write to the prefetchable upper base register on the root port (0x28), but the upper limit register (0x2C) also seems to be read only and stuck at zero.  Again, not sure why.  I left the upper base register at 0 which I think should allow access to the lower portion of RAM.

Can anyone see what I'm missing here?  

Thanks,

Steve

[sglow1]

OK, I think I've found my problem. 

When reading the IP core user's manual I didn't pay enough attention to the qword alignment of data.  As a result, when trying to write to a register that was not qword aligned (i.e. address bit 2 was set) I was putting the data in the wrong place in my TLP.

That's why I couldn't write to some configuration registers.  It wasn't that they were read only, it's because I was inadvertently always writing zeros to them.

RTFM I guess, and if you're still having problems, read it again!

[wchiah]

Hi Steve,

Thanks for sharing the resolution with us.

I believe that will be contributed to anyone who facing the same problem.

So do you still have any further question that I can address you ?

Else I will set this forum loop as close.

If your Intel support experience falls below a 9 out of 10, I kindly request the opportunity to rectify it before concluding our interaction. If the issue cannot be resolved, please inform me of the cause so that I can learn from it and strive to enhance the quality of future service experiences.

Regards,

Wincent_Intel

[sglow1]

Hi Wincent,

You can close this topic.  I'm now able to read/write memory to my endpoint successfully.

- Steve