PCIe external DMA controller example

[dsun01]

Dear Intel Support/Expert

I am building a PCIe DMA with external DMA descriptor controller. 

from user guide

• 10.2. Understanding the External DMA Descriptor Controller
  Using the External DMA Descriptor Controller provides more flexibility. You can either
  modify or replace it to meet your system requirements.You may need to modify the
  DMA Descriptor Controller for the following reasons:
  • To implement multi-channel operation
  • To implement the descriptors as a linked list or to implement a custom DMA
  programming model
  • To store descriptors in a local memory, instead of system (host-side) memory.

is there a valid example to show how to store descriptors in a local memory?

Thank you,

David

[Wincent_Altera]

Hi David,

Which device you are using ? If you are using Arria 10.

Below user guide shows some of example of using the external DMA controller

https://www.intel.com/content/www/us/en/docs/programmable/683425/18-0/understanding-the-external-dma-descriptor.html

Regards,

Wincent_Intel

[dsun01]

I am using Arria 10. I don't think you understand what I am asking.

[dsun01]

the example is using standard way to read descriptor from the host PC. I want to know how to read descriptor from other path like FPGA internal memory. 

• To store descriptors in a local memory, instead of system (host-side) memory.

is it possible to get any detail document or instruction how to implement this feature. 

I attached QAR file which I created using the provided example.  here is the instantiation of the module in the top. this is defined differently to the UG-01145_avmm_dma | 2021.06.03

I attached QAR file which I created using the provided example.  here is the instantiation of the module in the top.

[Wincent_Altera · Accepted solution]

Hi David,

EXPLANATION

The Descriptor Controller is essentially what controls the DMA transfers, and the CPU/control registers are used to interface with the descriptor controller. When an external descriptor controller is used, the use of the control registers becomes optional if you want to initiate locally. Our example design with external descriptor controller maintains the use of control registers as it assumes the host cpu controls the transfers -- but in reality, you have full control on when/how descriptors are sent to the read/write data movers.

In external descriptor controller "mode", avalon streaming interfaces are exposed, which is what you would use to feed the descriptors directly to the read/write data movers. In this mode, the implementation/storage of the descriptor table is up to the user, including when and how to send descriptors to the data movers. The requirement for you would be to send the descriptors in the correct format, and monitor the status buses (to see when tx/rx movers are ready to accept new data/done). You would then use the Txs interface to send status updates to the host.

Of course, this also assumes that your system is able to handle the system memory allocations accordingly and takes into account that the fpga is directly targeting system memory without communicating to the host aside from the 'completions'.

RECOMMENDATIONS

I would recommend going over Section 5 (IP Core Interfaces) of the IP user guide, and specifically Section 5.1.7 (Avalon-ST control interface for external descriptor controller) for details on the AVST interface. Section 10.2 also has details on using an external descriptor controller. Please download the User Guide as the online version is currently missing the core interface section (I will have this updated).

I also recommend reviewing our design example with the external descriptor controller, where you may inspect the example controller code and observe that the dma transfers are initiated by the controller itself (the controller reads the registers written by the host, but given host is bypassed, you can have your controller send descriptors to the dma ip directly based on your own criteria). You may find the example design ep_g3x8_avmm256.qsys, in the <quartus_install_dir>/ip/altera/altera_pcie/altera_pcie_a10_ed/example_design/a10 directory.

I know that was a lot of information! So please feel free to let me know if you need further clarifications.

Regards,

Wincent_Intel

[dsun01]

Hi Wincent

Thank you very much for the detailed and patient explanation. it is very helpful. I can imagine there is a state machine to coordinate the information transfer back and forth. as a mid-level engineer, it is not trivial to figure out a state machine from scratch and verify it. my personal understanding is that Intel provides an example will be the easiest way for a develop like me. of course, Intel couldn't do everything for its developer.  

at least Intel keeps an approach for local DMA initialization. 

Again from the user guide, the purpose of the external DMA is listed very clearly. does Intel have an example of the following applications?

Using the External DMA Descriptor Controller provides more flexibility. You can either
modify or replace it to meet your system requirements.You may need to modify the
DMA Descriptor Controller for the following reasons:
• To implement multi-channel operation
• To implement the descriptors as a linked list or to implement a custom DMA
programming model
• To store descriptors in a local memory, instead of system (host-side) memory
To interface to the DMA logic included in this variant, the custom DMA descriptor
controller must implement the following functions:
• It must communicate with the Write Mover and Read Mover to copy the descriptor
table to local memory.
• The Write Mover and Read Mover must execute the descriptors stored in local
memory.
• The DMA Avalon-MM write (WrDCM_Master) and read (RdDCM_Master) masters
must be able to update status to the TX slave (TXS).

I hope there is a giant shoulder I can rely on, it not, it will be a good opportunity for me to enhance my develop experience. I will start from 5.1.7

appreciate your help.

David

[Wincent_Altera]

Hi David,

Unfortunately, we do not have step-by-step example to run it.

But there is some step-by-step video for DMA example running for Arria 10.

It might not be what you looking at, but hope can give you some ideas to work on it.

If you does not have more question, I would like to have your permission to close this loop.

If you have a new question, feel free to open a new thread to get support from Intel experts.

Otherwise, the community users will continue to help you on this thread.

Thank you If you feel your support experience was less than a 9 or 10, please allow me to correct it before closing or let me know the cause so that I may improve your future support experience.

Regards,

Wincent_Intel

[Wincent_Altera]

https://www.intel.com/content/www/us/en/support/programmable/support-resources/fpga-training/fpga-quick-video-index.html?q=AVMM%20DMA&s=Relevancy

[TejaMB]

Does external descriptor mode support simulation? So far I had no luck in running on VSC and Questa. Both of them are stuck in the initialization state. I'm using Agilex P-tile MC-DMA.