PCIe design reference recommendation (Cyclone 10 GX)

[Derek_Teng]

Hello!

I'm a newbie in PCIe design. Now I want to design a Gen2x1 PCIe board using C10GX (10CX220YU484). Since the chip I'm using can only implement two types of interfaces, Gen2x1 or Gen2x2, and I can't use “avalon - mm with dma”, so I choose to use the “avalon - mm” interface and add a DMA IP core in the platform designer to achieve DMA data transfer.

Do you have any recommended example designs for reference?

I've looked at the design examples provided in /ip/altera/altera_pcie/altera_pcie_a10_ed/example_design/c10, but there is no corresponding documentation.

In addition, can you provide the source code of the Windows driver? I want to modify it on this basis to meet my usage requirements.

[Wincent_Altera]

Hi Derek,

If you using Cyclone 10 GX and you need AVMM - DMA design.

You may refer to Intel® Cyclone® 10 GX FPGA – PCIe* 2.0 x4 DMA Design Example

If your slot is gen2x2, by right the host will auto degrade from x4 to x2 is you slot into x2 slot.

For this design, only Linux version of driver is provided. 

Detail about the design guide you may refer to https://www.intel.com/content/www/us/en/docs/programmable/683554/18-0/installing-the-dma-test-driver-and-running.html

Regards,

Wincent_Altera

[Derek_Teng]

Hi Wincent,

Thank you very much for your reply and help.
When I was reading the document "Arria10 and Cyclone10 GX Avalon Memory-Mapped (Avalon-MM) Interface for PCI Express* User Guide", I have a question to ask.
The document mentions an Avalon-MM-to-PCI Express Address Translation Table. It introduces the control registers of the PCI Express Avalon-MM bridge. Among the registers in the address range of 0x1000 - 0x1FFF, only the Avalon-MM-to-PCI Express address map entry is mentioned. May I ask where this translation table is actually stored?
In addition, does the computer driver need to edit this translation table?
Can I understand it in this way: Before starting the DMA transfer, the computer driver needs to edit this translation table according to the DMA descriptor table first. In this way, during the DMA transfer, the bridge can convert the avalon-mm address passed in through the txs interface into the physical address of the memory applied by the computer driver for DMA.

There is another question I'd like to ask. The board I designed is a Gen2x1 PCIe board. If I generate an IP core of "Avalon-MM with DMA, Gen2x4", can it be used for my board?

[Wincent_Altera · Accepted solution]

Hi Derek,

Can I understand it in this way: Before starting the DMA transfer, the computer driver needs to edit this translation table according to the DMA descriptor table first. In this way, during the DMA transfer, the bridge can convert the avalon-mm address passed in through the txs interface into the physical address of the memory applied by the computer driver for DMA.
>> Yes, your understanding is correct. Before initiating a DMA transfer, the computer driver typically needs to configure the address translation table within the PCI Express Avalon-MM bridge. This configuration ensures that the addresses used in the DMA descriptor table are correctly mapped to the physical addresses in the system memory.

The board I designed is a Gen2x1 PCIe board. If I generate an IP core of "Avalon-MM with DMA, Gen2x4", can it be used for my board?
>> Theoritically, you can use a Gen2x4 IP core with a Gen2x1 board, the system will operate with only one lane active, limiting the bandwidth to that of a Gen2x1 configuration. Make sure to adjust any settings in the IP core or driver to reflect the actual lane count supported by your hardware.

Regards,
Wincent_Altera

[Derek_Teng]

Hi Wincent.

Thank you very much for your reply and help.
I'd like to consult you on a few more questions.
I've read some guidance documents and can understand some design examples provided by Intel Altera. However, these design examples are all internal loops.
That is, when performing DMA read/write or using the BAR interface (RXM) for read/write operations, it's always reading and writing a on-chip memory.
The data is neither output externally nor input from the outside.

In my actual application, the data read by DMA (from the computer to the FPGA) needs to be output to my logic module for processing;
the data source for DMA write (from the FPGA to the computer) is the sensor data on my board.
I used Platform Designer to design a Qsys with an IP core that uses Avalon-MM with DMA(gen2x4, 128bit, 125Mhz).

1. When handling DMA write, I added an Avalon FIFO Memory Intel FPGA IP (configured as avmm-read, avst-sink, Allow backpressure).
The external sensor data is input through avst-sink (I exported the avst-sink interface). The dma_wr_master of the PCIe core is connected to avmm-read to read the sensor data.

2. When handling DMA read, I added an Avalon FIFO Memory Intel FPGA IP (configured as avmm-write, avst-source, Allow backpressure).
The dma_rd_master of the PCIe core is connected to avmm-write to write the data read from the computer into this FIFO,
and then output it to the external logic module through avst-source (I exported the avst-source interface).

May I ask if you can recommend a better way to handle the input of external sensor data into this PCIe system and the output of the data read from the computer by the PCIe system to the external logic module?

3. I hope to use the BAR interface (RXM) to read and write the control registers in my logic module and read and write the operating status of my logic module.
If I directly export the BAR interface, Platform Designer will not allocate an address segment for this BAR, nor will it set the addressing space size of this BAR.
Will this cause the BAR interface to be actually unavailable because the computer fails to allocate space for this BAR during startup and enumeration?
How should I export this BAR interface (RXM) to the outside of Qsys?
Can I first connect this BAR interface to a on-chip memory, and then manually add this BAR interface to the top-level input/output ports in the generated Verilog code?

[Wincent_Altera]

Hi Derek,

1. When handling DMA write, I added an Avalon FIFO Memory Intel FPGA IP (configured as avmm-read, avst-sink, Allow backpressure).
The external sensor data is input through avst-sink (I exported the avst-sink interface). The dma_wr_master of the PCIe core is connected to avmm-read to read the sensor data.

2. When handling DMA read, I added an Avalon FIFO Memory Intel FPGA IP (configured as avmm-write, avst-source, Allow backpressure).
The dma_rd_master of the PCIe core is connected to avmm-write to write the data read from the computer into this FIFO,
and then output it to the external logic module through avst-source (I exported the avst-source interface).
>> Ok.. it sound right to me from logical perspective.

How should I export this BAR interface (RXM) to the outside of Qsys?

>> Qsys needs to see something bolted to the BAR interface before it will allow any allocation of BAR memory space. So if you just export the BAR, Qsys says meah, he does not need any space.
>> You need to connect an Avalon-MM pipelined bridge to Rxm_BARn in Qsys. Then export the Avalon master side of the bridge.

Regards,
Wincent_Altera

[Derek_Teng]

Thank you. I think I've found a better solution.

I use the generic component in the IP column of the platform designer to create my own interface module. It's very convenient to export the RXM interface, the data source interface for DMA write operations, and the data output interface for DMA read operations to my external logic.
At the same time, I can also adjust its address allocation so that the address addressing range meets my design requirements and matches well with the addressing space on the computer side.

Thank you!

[Wincent_Altera]

Hi Derek,

Thanks for sharing with me how you work on this. 
Meanwhile, is there anything else you think I could help you ?

Regards,
Wincent_Altera

[Derek_Teng]

Hi Wincent,

There are no other questions. Thank you for your patient answers. Thanks!