[Vune- core-utilization-in-dpdk-apps] CPU Time for DPDK Rx thread is not close to 100%

[paresh_agarwal]

Hi

I am new to using VTune profiler.

I am using Intel® VTune™ Profiler (version 2023.1.0) offline.

I am trying to test communication between two modules which uses DPDK for packet forwarding from Module 1 to Module 2

I want to understand how many of rte_eth_rx_burst() calls actually fetch data in Module 1 which receives data from Module 2. 

In Module 1 rte_eth_rx_burst() is running under a while(1) loop on a separate thread which is used exclusively for received data from Module 2.

I am running "Input and Output Analysis" by selection DPDK I/O API

My understanding is even if there is no data then also CPU time for the DPDK Rx thread of Module 1 would be close to 100% (99.6%) once my module 1 is running as it runs in while (1) loop.

But I can see even though the Module 1 (DPDK Rx Thread) is running fine throughout the test run of 60 seconds, it shows CPU time close to 100% only for first few seconds 7~8 seconds.

Whereas DPDK Rx Spin time is shown close to 100% (99.5%) throughout the test run of 60 seconds

Please help me to understand what is missing which is causing this behavior.

[AthiraM_Intel]

Hi,

Thank you for posting in Intel Communities.

Could you please share the following details:

1. OS and Hardware details
2. Sample reproducer code
3. The exact steps you followed to reproduce the same from our end

Thank you

[paresh_agarwal]

Hi

Please find the details

1.OS and Hardware details

Collection and Platform Info
Application Command Line:
Operating System: 4.18.0-348.12.2.rt7.143.el8_5.x86_64 Red Hat Enterprise Linux release 8.4 (Ootpa)
Result Size: 172.4 MB
Collection start time: 14:59:56 18/05/2023 UTC
Collection stop time: 15:00:59 18/05/2023 UTC
Collector Type: Driverless Perf system-wide sampling, User-mode sampling and tracing
Finalization mode: Fast. If the number of collected samples exceeds the threshold, this mode limits the number of processed samples to speed up post-processing.
CPU
Name: Intel(R) Xeon(R) Processor code named Icelake
Frequency: 2.1 GHz
Logical CPU Count: 24
Max DRAM Single-Package Bandwidth: 76.0 GB/s
LLC size: 37.7 MB
Cache Allocation Technology
Level 2 capability: not detected
Level 3 capability: available

2. Sample reproducer code

The code below is a sample code which can give you an idea of how my code looks like, there is a separate thread which runs on a separate core to receive rx data from Module 2

#include <rte_ethdev.h>

#define MAX_RX_PACKETS 32

int main() {
uint16_t port_id = 0; // Port ID to receive packets from
uint16_t nb_rx_packets;
struct rte_mbuf *rx_packets[MAX_RX_PACKETS];

// Initialize DPDK and Ethernet device

// Main loop
while (1) {
nb_rx_packets = rte_eth_rx_burst(port_id, 0, rx_packets, MAX_RX_PACKETS);

if (nb_rx_packets == 0) {
// No packets received
continue;
}

// Process received packets
for (int i = 0; i < nb_rx_packets; i++) {
// Process the packet in rx_packets[i]
// ...
}
cd
// Free the received packets
rte_pktmbuf_free_bulk(rx_packets, nb_rx_packets);
}

// Cleanup and exit
// ...

return 0;
}

3.The exact steps you followed to reproduce the same from our end

a) Starting Vtune for "Input and Output Analysis with DPDK I/O API checked

b) Staring application which runs module 1 and module 2

c) Module 1 starts thread to receive data from module 2, even if there is no data thread is running as soon as module 1 starts (so CPU time should be close to 100%)

d) Run for 60 second or more than 60 seconds

e) Stop vtune capture.

Vtune only shows CPU time close to 100% only for first few seconds, sometimes it is 7~8 seconds, sometimes 15~20 seconds.

[AthiraM_Intel]

Hi,

Thank you for sharing the details. We are checking on this internally.

Could you please reply to the private message which we sent.

Thanks

[paresh_agarwal]

Hi,

Any update on this issue? 

" Could you please reply to the private message which we sent. "

I am not sure which private message are you talking about.

Thanks

[AthiraM_Intel]

Hi,

We have not heard back from you. Could you please give us an update?

Thanks

[paresh_agarwal]

Hi,

Sorry, I am not sure what is expected from my side, I have already shared the details which were asked.

Thanks

[AthiraM_Intel]

Hi,

We are extremely sorry for the confusion. We need your "rte_ethdev.h" file or the executable to reproduce the issue from our side. So that we contacted you privately.

Could you please share your "rte_ethdev.h" file. Or if possible send your executable.

Please attach the result directory also.

Thanks.

[paresh_agarwal]

Hi,

Thanks for your reply, I cannot share the executable with you.
Please find the attached rte_ethdev.h file which is taken from dpdk-20.11.3/lib/librte_ethdev

Thanks

Paresh

[AthiraM_Intel]

Hi,

we are checking on this internally. We will get back to you with an update.

Thanks

[AthiraM_Intel]

Hi,

Could you please share your result folder. We are not able to see the issue from our side.

We are extremely sorry for the inconvenience caused.

Thanks

[paresh_agarwal]

Hi,

Unfortunately, I cannot share the result folder due to confidentiality of the project.
Please let me know if you need any specific information/file from the result folder and the location of that information/file.

Thanks

[AthiraM_Intel]

Hi,

Thank you for the update. We could see that there are so many dependencies (header files) in the code you shared.

Please see the below screenshot:

Could you please share a sample reproducer with all the dependent files to reproduce the issue from our end.

Thanks

[AthiraM_Intel]

Hi,

We have not heard back from you. Could you please share a sample reproducer with all the dependent files to reproduce the issue from our end.

Thanks

[paresh_agarwal]

Hi,

Sorry, I missed the mail.

Regarding dependent files which you have mentioned, I believe all those are from DPDK library only. I shared the "rte_ethdev.h" file from below path dpdk-20.11.3/lib/librte_ethdev,  all related files should be from here.

As mentioned earlier I could not share the reproducer code.

Also, can you please confirm if there is any requirement on the number of resources(core, memory) required to run Vtune ?

Thanks

[AthiraM_Intel]

Hi,

We are extremely sorry for the delay. We are debugging your issue. We are getting errors while replicating your issue.

Please see the below screenshot for the command we used and the error we got:

It will be helpful if you share the the exact steps to reproduce the binary.

We are checking on this internally, we will get back to you.

Thanks