Intel DPDK (Xeon) - how to reserve a core for Linux Kernel

[AYADA9]

Hi,

I am working on a board with Dual Intel Xeon E5-2600L series processors. With our app, I am able to achieve 40Gbps processing at "X" bytes for a few minutes, with 4 LoadBalancer Cores and 16 Worker cores. However, during extended test-runs, I am seeing packet drops in large bursts (burst of a few thousand packets) even at 2X packet size over a 24hr traffic test (constant 40Gbps traffic).

I do not think it is PPS performance issue. The drops are always on the 82599 due to FIFO over-flow i.e. indicate that the Load Balancer is not fast enough to pull it sometimes (so issue is not at Workers). All the Load Balancer threads are running on Socket 0 - which is directly connected to 2 82599 devices. Theoretically the LoadBalancers should be capable of pulling 40Gbps at "X" Bytes and I also see that for a few minutes there are no drops. As I keep increasing the Packet size, the duration for the bursty drops keeps increasing, so much so that at 2X it takes about 24 hrs for the drops to happen - but I still eventually see the drops - which then again go away for next few hours and so on (I am not yet sure if there is a definite periodicity to the drops).

Inorder, to improve the LoadBalancer thread performance, I do not have those threads Hyper-threaded with the Worker threads - i.e. the corresponding HyperThread is not allocated (Initially I had the Workers hyper-thread with LoadBalancer and that was impacting performance of the LoadBalancer - resulting in drops on 82599). I believe, the non-allocated threads can be used by the Linux Kernel. I was therefore wondering if this sudden drop indicates that some Linux Kernel processing is happening on the free cores and holding off the LoadBalancer due to hyper-thereading. To test this, I wanted to fix the Linux cores, to the other 4 cores that I have available (and ensure that there is no hyper-threading with the LoadBalancer threads). Is there a way I can force the Linux Kernel to exclude certain cores?

Any other suggestions as to what maybe causing these drops during an extended traffic run (at line-rate) are welcome. (I do plan to test with the sample LoadBalancer app soon).

Regards

Alam

[Muthurajan_J_Intel]

Dear customer,

Thank you.

Can you kindly let us know your update on this please.

Forcing the Linux kernel to exclude certain core - you must be familiar with ISOLCPU - that isolates the core from Kernel. https://groups.google.com/forum/# !topic/linux.kernel/Txr5lt--Vv8 https://groups.google.com/forum/# !topic/linux.kernel/Txr5lt--Vv8

Please let us know

Thanks,

[DKami1]

Hi Alam,

Another thing you may want to check is the PCI limitation. You might reaching the PCI limitation and not the CPU limitation.

Regards,

Daniel Kaminsky