FPGA Intellectual Property
PCI Express*, Networking and Connectivity, Memory Interfaces, DSP IP, and Video IP
6404 Discussions

Poor performance with PCIe and DMA on Linux

Honored Contributor II

Hello all, 


I built a evaluation system on the Arria II dev board that transports two similar datastreams to a linux host so that each stream can be read from a different device file. 


The sopc system uses a dma controller that reads data from a fifo. The fifos are located in a custom component. The fifo is twice the maximum dma transfer length. The fifo generates a waitrequest if it's less than half full. 


Each stream has a datarate of approx. 62.5 Mb/s, so I need 125 Mb/s what should be possible with a PCIe-x1 port. 


If I measure the throughput with `dd`, I reach 40.1 Mb/s reading one stream at a time. 


I realized that waitrequest from the PCIe-Tx-port throttles the data flow significantly.  


My system is homogenous with 32 bit datawidth. 


Does somebody has a idea how I can speed up my data transfers?  


Thanks in advance. 





0 Kudos
1 Reply
Honored Contributor II

Hello again, 


I did more tests. 


I doubled the datawidth from 32 bit to 64 bit. The fifos write doublewords so the throughput is doubled in theory, because the pcie-tx-port ist 64 bit wide. 


In fact the throughput isn't better now. 


If I increase the DMA-block-length, the whole card freezes. The PCIe-interface sets waitrequest and stays in an that state. The card does not react to any command until I reset it. If I put debugging messages into my driver's code, the block length can be bigger, because it takes time until the next dma-request. 


This issue occured with singleword transfers, too. 


Does anybody has an idea? 





0 Kudos