We are developers currently testing our 100G NIC (Mellanox connect X5) on our Intel(R) Xeon(R) Gold 6258R CPU @ 2.70GHz (4 NUMA)running Ubuntu18.04 and have some questions regarding the DMA to be used. We have 2 options:
1) Use DMA available on the NIC (NIC connected via PCIe bus ).
The throughput achieved is around 172420 KBps (1.4Gbps) for a single DMA channel .We also tried using taskset to run on a single core. $ taskset -c 4 sudo insmod ./dmatest.ko channel=dma0chan0 timeout=2000 iterations=100 run=1
but got similar results.
As far as our understanding, the NIC is connected to the memory controller through one DMA(Host) channel. So for evaluating the throughput of one DMA channel, we used the above mentioned method.
Is our understanding correct? If not, can we use multiple host DMA channels for a single device(NIC) ? Since the throughput from one DMA channel is very low(1.4Gbps) compared to our expectations (100Gbps), we raised this query.
1) What is the DMA controller(chip) used in this setup? is it 8237?
2) Is this the throughput to be expected from a single DMA channel? 3) If not, then what's the limit and how to achieve it?