Embedded Intel® Core™ Processors
Communicate Intel® Core™ Hardware, Software, Firmware, Graphics Concerns

Low DPDK performance

AATLI
Beginner
2,071 Views

Hi

I'm trying to measure the throughput of DPDK for physical ports (using 64-byte udp packets), but it seems very low. My test setup is shown below:

 

+─────────────────────────────+

│ 82599ES 10-Gigabit SFI/SFP+ │

+───+────+──────────+────+────+

│ p0 │ │ p1 │

+────+ +────+

^ ^

| |

v v

+────+ +────+

│ p0 │ │ p1 │

+───+────+──────────+────+────+

│ NAPATECH Adapter - 2 port │

+─────────────────────────────+

 

I isolated cores (master and logical cores on hyper-threading) from Linux kernel according to the output of dpdk/tools/cpu_layout.p, iommu is set to pt, intel_iommu is off and 1G hugepages allocated for numa nodes as follows;

$ cat /sys/devices/system/node/node*/meminfo | fgrep Huge

Node 0 AnonHugePages: 137216 kB

Node 0 HugePages_Total: 3

Node 0 HugePages_Free: 2

Node 0 HugePages_Surp: 0

Node 1 AnonHugePages: 227328 kB

Node 1 HugePages_Total: 2

Node 1 HugePages_Free: 1

Node 1 HugePages_Surp: 0

$ ./cpu_layout.py

cores = [0, 1, 2, 8, 9, 10]

sockets = [0, 1]

Socket 0 Socket 1

-------- --------

Core 0 [0, 12] [6, 18]

Core 1 [1, 13] [7, 19]

Core 2 [2, 14] [8, 20]

Core 8 [3, 15] [9, 21]

<span style="color: # 000000; font-family:...

0 Kudos
5 Replies
CarlosAM_INTEL
Moderator
490 Views

Hello avatli,

Thank you for contacting the Intel Embedded Community.

We suggest you verify that the mentioned test fulfills with the requirements stated at the http://dpdk.org/doc/dts/gsg/index.html Getting Started Guide — DPDK Test Suite documentation.

It is important to let you know that your DPDK consultations will be further answered using the email addresses listed at the http://dpdk.org/ml DPDK Mailing Lists.

Please let us know if this information is useful to you.

Best Regards,

Carlos_A.

0 Kudos
AATLI
Beginner
490 Views

Hello

No one has answered the question at the DPDK Mailing Lists. We still have the performance problem. I could not reach your benchmark results, especially for small packets.

0 Kudos
CarlosAM_INTEL
Moderator
490 Views

Hello avatli,

Thanks for your update.

We have the following suggestions to solve this inconvenience:

1) Please add --numa knob to ./testpmd command line.

2) Please increase the number of descriptors (try default values (no –rxd –txd flags, and rxd=256, txd=512/1024).

3) Also, it would be helpful to see the output of 'lspci –vt'.

Please let us know if this information is useful to you.

Best Regards,

Carlos_A.

 

 

0 Kudos
AATLI
Beginner
490 Views

Hi Carlos

Unfortunately still the same low performance, here is my output;

argela@argela-HP-Z800-Workstation:~/ovs_dpdk/dpdk/app/test-pmd/build/app$ lspci -vt

-+-[0000:7f]-+-00.0 Intel Corporation Xeon 5600 Series QuickPath Architecture Generic Non-core Registers

| +-00.1 Intel Corporation Xeon 5600 Series QuickPath Architecture System Address Decoder

| +-02.0 Intel Corporation Xeon 5600 Series QPI Link 0

| +-02.1 Intel Corporation Xeon 5600 Series QPI Physical 0

| +-02.2 Intel Corporation Xeon 5600 Series Mirror Port Link 0

| +-02.3 Intel Corporation Xeon 5600 Series Mirror Port Link 1

| +-02.4 Intel Corporation Xeon 5600 Series QPI Link 1

| +-02.5 Intel Corporation Xeon 5600 Series QPI Physical 1

| +-03.0 Intel Corporation Xeon 5600 Series Integrated Memory Controller Registers

| +-03.1 Intel Corporation Xeon 5600 Series Integrated Memory Controller Target Address Decoder

| +-03.2 Intel Corporation Xeon 5600 Series Integrated Memory Controller RAS Registers

| +-03.4 Intel Corporation Xeon 5600 Series Integrated Memory Controller Test Registers

| +-04.0 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Control

| +-04.1 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Address

| +-04.2 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Rank

| +-04.3 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Thermal Control

| +-05.0 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Control

| +-05.1 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Address

| +-05.2 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Rank

| +-05.3 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Thermal Control

| +-06.0 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Control

| +-06.1 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Address

| +-06.2 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Rank

| \-06.3 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Thermal Control

+-[0000:7e]-+-00.0 Intel Corporation Xeon 5600 Series QuickPath Architecture Generic Non-core Registers

| +-00.1 Intel Corporation Xeon 5600 Series QuickPath Architecture System Address Decoder

| +-02.0 Intel Corporation Xeon 5600 Series QPI Link 0

| +-02.1 Intel Corporation Xeon 5600 Series QPI Physical 0

| +-02.2 Intel Corporation Xeon 5600 Series Mirror Port Link 0

| +-02.3 Intel Corporation Xeon 5600 Series Mirror Port Link 1

| +-02.4 Intel Corporation Xeon 5600 Series QPI Link 1

| +-02.5 Intel Corporation Xeon 5600 Series QPI Physical 1

| +-03.0 Intel Corporation Xeon 5600 Series Integrated Memory Controller Registers

| +-03.1 Intel Corporation Xeon 5600 Series Integrated Memory Controller Target Address Decoder

| +-03.2 Intel Corporation Xeon 5600 Series Integrated Memory Controller RAS Registers

| +-03.4 Intel Corporation Xeon 5600 Series Integrated Memory Controller Test Registers

| +-04.0 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Control

| +-04.1 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Address

| +-04.2 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Rank

| +-04.3 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 0 Thermal Control

| +-05.0 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Control

| +-05.1 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Address

| +-05.2 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Rank

| +-05.3 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 1 Thermal Control

| +-06.0 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Control

| +-06.1 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Address

| +-06.2 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Rank

| \-06.3 Intel Corporation Xeon 5600 Series Integrated Memory Controller Channel 2 Thermal Control

+-[0000:40]-+-03.0-[42]--

| +-07.0-[60]--+-00.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection

| | \-00.1 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection

| +-09.0-[41]----00.0 LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS

| +-10.0 Intel Corporation 7500/5520/5500/X58 Physical and Link Layer Registers Port 0

| +-10.1 Intel Corporation 7500/5520/5500/X58 Routing and Protocol Layer Registers Port 0

| +-11.0 Intel Corporation 7500/5520/5500 Physical and Link Layer Registers Port 1

| +-11.1 Intel Corporation 7500/5520/5500 Routing & Protocol Layer Register Port 1

| +-14.0 Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers

| +-14.1 Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers

| +-14.2 Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers

| \-15.0 Intel Corporation 7500/5520/5500/X58 Trusted Execution Technology Registers

\-[0000:00]-+-00.0 Intel Corporation 5520 I/O Hub to ESI Port

+-01.0-[03]--

+-03.0-[0f-12]----00.0-[10-12]--+-00.0-[11]--+-00.0 NVIDIA Corporation GF110 [GeForce GTX 590]

| | \-00.1 NVIDIA Corporation GF110 High Definition Audio Controller

| \-02.0-[12]--+-00.0 NVIDIA Corporation GF110 [GeForce GTX 590]

| \-00.1 NVIDIA Corporation GF110 High Definition Audio Controller

+-07.0-[28]----00.0 Napatech A/S NT4E2-4-PTP Network Adapter 4x1Gb

+-10.0 Intel Corporation 7500/5520/5500/X58 Physical and Link Layer Registers Port 0

+-10.1 Intel Corporation 7500/5520/5500/X58 Routing and Protocol Layer Registers Port 0

+-11.0 Intel Corporation 7500/5520/5500 Physical and Link Layer Registers Port 1

+-11.1 Intel Corporation 7500/5520/5500 Routing & Protocol Layer Register Port 1

+-14.0 Intel Corporation 7500/5520/5500/X58 I/O Hub System Management Registers

+-14.1 Intel Corporation 7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers

+-14.2 Intel Corporation 7500/5520/5500/X58 I/O Hub Control Status and RAS Registers

+-15.0 Intel Corporation 7500/5520/5500/X58 Trusted Execution Technology Registers

+-1a.0 Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller # 4

+-1a.1 Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller # 5

+-1a.2 Int...

0 Kudos
CarlosAM_INTEL
Moderator
490 Views

Hello avatli,

Thanks for your update.

In order to better understand the situation, we would like to address the following questions:

Could you please tell us the model of the platform that is being used?

Could you please paste the output of lscpu?

From the lspci output it looks like the NICs are connected to the PCIe ports on socket 1 (out of s0 and s1). If that is the case, it is best to use the cores from socket 1 as well. 2 cores per 10G port should be enough (depending on the platform) could you try the following core mask:

./testpmd –c 0x3c0 –n 2 -- -i --numa --nb-cores=4

And also with more descriptors:

./testpmd –c 0x3c0 –n 2 -- -i --numa --nb-cores=4 --txd=1024 --rxd=512

Also, Could you please confirm if you have 2 memory channels populated? Are the socket 1 channels also populated? What is the model / frequency of the DIMMs?

Please let us know all the requested information to have the complete perspective of the cited situation.

Best Regards,

Carlos_A.

 

0 Kudos
Reply