Arria 10 PCIe DMA Bus Mastering

[Aflop]

Background:

I have been working with the Arria 10 GX Devkit and using the PCIe with DMA HIP. I have the general scheme working with 5 BARs of memory with a few minor kinks. 

Problem :

In addition to having 5 BARs of memory, I need to have Bus Mastering enabled as well. My understanding of this is that I have complete access to the Host side memory. I am working closely with software and we have a plan on how to execute an interrupt and addressing scheme such that I do not write where I am not supposed to. Unfortunately, I have yet to find an example of this use case. 

Question :

Could someone point me to an example of this? It does not need to be BARs and Bus Mastering but I need an example or document of how to implement the Bus Mastering so that I can integrate it into my design. An example of both would be superb.  

Thank you,

[Aflop]

Additionally, I am reading the "Arria 10 and Cyclone 10 GX Avalon-MM Interface for PCI Express User Guide". I'm sure that if I spent the next year studying it I might have a viable solution, but what I am looking for is a process. 

Section 6.8.1 Sending a Write TLP - seems like it is part of the process. It looks like this might be where I insert the host memory address but the process does not go into sending the data. 

Does a step-by-step process exist for this somewhere or could anyone help me to connect the dots?

[RongYuan]

Hi,

Doing a BAR read/write is different from doing an interrupt.

When FPGA PCIe is an endpoint, the host written data to a BAR will be sent to on-chip memory. This data transfer requires help from a driver.

If you're looking for an interrupt design, you may check

1. Implementing MSI-X for PCI Express in Altera FPGA Devices - Intel Community

2. https://www.intel.com/content/www/us/en/docs/programmable/683686/20-4/setting-up-and-verifying-msi-interrupts.html

3. https://www.rocketboards.org/foswiki/Projects/A10AVCVPCIeRootPortWithMSILTS

The 1st one is more conceptual, only sim.

The 2nd and 3rd ones are more practical.

Regards,

Rong

[RongYuan]

Any further questions?

[Aflop]

I am not looking for an interrupt design, however I will be utilizing interrupts in the design I am asking about. 

Background:

Previously we had a board using an Intel Cyclone IV FPGA. It utilized 6 bars and 2 direct DMA channels. My current design has the BARs and we are testing them currently. The next task is to instantiate these direct DMA channels.

The way the Endpoint-to-Host-PC channel works:

The host PC utilizes a BAR to write a host memory address space which has been mapped and is free to be used by the endpoint. To be clear this is an address in Host PC memory. The endpoint FPGA board then writes to this address space and triggers an interrupt when finished. The host PC then knows to read from the address space and we repeat. This process has been described to me as Bus Mastering. The key point is that the Endpoint FPGA Board is writing to Host PC memory and it is not done through a BAR. 

Question :

How do I achieve this design?      

[RongYuan]

The DMA example implements such data transfer. You may check the interrupt function in the driver.

Generally a Host PC is based on an OS so an interrupt handle has to be registered with an IRQ first (done by a driver). Once FPGA raises an interrupt, the Host will receive it and feedback to FPGA a transaction completion, then FPGA unmasks the interrupt.

Regards,

Rong

[RongYuan]

If no further questions, I'll transit this case to community support. Thanks.