Question About High Throughput Networking

[Alperen_A]

Hello,

I am comparing two different host memory layouts for receiving packet data from an FPGA over PCIe, focusing specifically on CPU-side processing requirements. For my use case, throughput is very important, and I am trying to minimize packet loss as much as possible, ideally down to zero.

In the first approach (chunk-based / stream-like), data is written to host memory in fixed 64-byte blocks. Each block contains its own metadata, such as SOP (start of packet), EOP (end of packet), and possibly a sequence number. A single packet is split across multiple such chunks. On the CPU side, processing involves reading each chunk sequentially, parsing the metadata for every block, detecting packet boundaries using SOP/EOP flags, and reassembling the full packet from multiple chunks before further processing. Also, the packets may be copied to another thread to do the reassembling.

In the second approach, host memory is divided into 2 sections. One is used to hold the packets without any processing, and the other is used as a ring buffer that holds the start of each packet in the first section. This would add another L3 lookup to fetch the address of each 8 packets(the reason for 8 packets is that 8 addresses fit in a cache line). The other overheads of this approach are that it adds a PCIe memory write operation for every 8 packets to write the addresses of the packets, and also it requires another PCIe memory write to change the producer of the ring buffer. 

I would like to get your feedback on these two approaches. I am having some trouble comparing the overheads of the two approaches while optimizing for the throughput, looking from the CPU perspective.

Best Regards,

Alperen Burkay Sevim

[MACM]

Hi Alperen_A,

Greetings of the day.

Hope you are doing well.

Could you please share more about the product that you are using.

So that we can proceed further accordingly.

Best Regards,

Mohammed Ali CM

Intel Customer Support Technician

[MACM]

Hi Alperen_A,

Greetings of the day.

Hope you are doing well.

This is a follow up. Could you please share more about the product that you are using.

So that we can proceed further accordingly.

Best Regards,

Mohammed Ali CM

Intel Customer Support Technician

[Alperen_A]

Dear Mohammed Ali,

Thank you for your response.

We are planning to use an Intel Xeon processor as the host CPU in our system.

Best regards,
Alperen Burkay Sevim

[Sreelakshmi1]

Hi Alperen_A,

Thank you for your response. Kindly confirm the complete model name of the Intel Xeon processor and the full system model so that we can check the details further.

Best Regards,

Sreelakshmi

Intel Customer Support Technician

[Alperen_A]

Hi Sreelakshmi,

We are currently using an Intel Xeon Silver 4510 as the host CPU in our system.

Also, packet data is written directly to host memory via DMA and stored in a PCAP compatible format. Additionally, each packet includes a high-precision timestamp generated using a White Rabbit (WR) synchronization system.

The CPU is responsible for handling continuous high-bandwidth DMA traffic, as well as parsing and processing PCAP formatted packet data together with the associated WR timestamps.

Best regards,
Alperen Burkay Sevim

[pujeeth]

Hi Alperen_A,

Greetings of the day.

Thank you for providing the details. We are currently reviewing the case and will get back to you with an update as soon as possible.

Regards,

Pujeeth_Intel

[Dineshbabu]

Hello Alperen_A,

Greetings!

Thank you for your patience. We have sent an email to you. Kindly review it.

Regards, 

Dinesh

Intel Customer Support Technician