[Intel Corp X540T1BLK] Throughput performance issue.

[Balakrishna]

Hi,

Test scenario: TCP traffic between  NXP 10Gig port to Intel PCI 10G port installed in Linux machine.

Current linux version: 5.8.0.43

Current intel driver verion: 5.10.2

[Issue]

Less than <1mbps throughput observed after few iteration.

[Temp Fix]

When issue is reproduced, making intel 10G interface down and up. No issue seen.

Below is the ethtool output when issue is reproduced. Observed more CRC failures during fail case. Kindly help me out this issue.

fb@fb:~$ ethtool -S enp1s0f0 | grep error
rx_errors: 885406
tx_errors: 0
rx_over_errors: 0
rx_crc_errors: 885398
rx_frame_errors: 0
rx_fifo_errors: 0
rx_missed_errors: 0
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
rx_length_errors: 8
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_csum_offload_errors: 0

Thanks,

[Caguicla_Intel]

Hello Balakrishna,

Thank you for posting in Intel Ethernet Communities. 

Please provide the following information that would help in checking your request.

1.) How many adapters are affected by the issue? 

2.) Can you share more information regarding your setup/configuration? Like Operating System System and motherboard used? 

3.) We've sent an email as we are requesting for SSU logs and photos of the adapter. Please be reminded not to post it here to avoid exposing confidential information.

Looking forward to your reply.

We will follow up after 3 business days if we don't hear from you.

Best regards,

Crisselle C

Intel® Customer Support

[Caguicla_Intel]

Hello Balakrishna,

Appreciate your swift response via email.

We hope you don't mind providing additional details below as this would help us with the investigation.

1.) You mentioned that there are total of 14 dual port adapters affected by this issue, can you confirm that those 14 adapter are all the same model which was Intel® Ethernet Server Adapter X520-T2?

2.) Kindly provide photo of the other side of the adapter. Is there a YottaMark or BradyID sticker like the one shown on below link?

YottaMark* and BradyID* Stickers on Intel® Ethernet Adapters. Please be reminded to send the photos via email. 

https://www.intel.com/content/www/us/en/support/articles/000007074/network-and-i-o/ethernet-products.html

3.) Can you confirm if you have tried the following for the Tuning Throughput Performance for Intel® Ethernet Adapters?

https://www.intel.com/content/www/us/en/support/articles/000005811/network-and-io/ethernet-products.html

Awaiting to hear from you.

Should there be no response, we will follow up after 3 business days.

Best regards,

Crisselle C

Intel® Customer Support

[Balakrishna]

@Caguicla_Intel 

Shared requested additional info through email. Kindly let me know if you need any further info.

Thanks

[Caguicla_Intel]

Hello Balakrishna,

Thank you for the reply.

Unfortunately, we weren't able to receive your reply. Will you be able to re-send your reply via email? We will also send another email so you can reply on it. 

Looking forward to your reply.

We will follow up after 3 business days in case we don't hear from you.

Best regards,

Crisselle C

Intel® Customer Support

[Balakrishna]

Hi @Caguicla_Intel 

Re-sent all the required details through e-mail, kindly verify it.

Thanks  

[Caguicla_Intel]

Hello Balakrishna,

Yes, confirming that we received the email. Your effort in re-sending your answer is highly appreciated.

Please allow us to further check on this issue. We will get back to you as soon as there is any findings but no later than 2-3 business days. 

Hoping for your kind patience. 

Best regards,

Crisselle C

Intel® Customer Support

[Caguicla_Intel]

Hello Balakrishna,

Good day!

Before we further check this request with our engineers, we would like to know the cable type used and switch/router information. Please also let us know if the 14 adapters are all installed on the same network. 

Looking forward to hear from you.

Should there be no response from you, I’ll make sure to reach out after 3 business days.

Best regards,

Crisselle C

Intel® Customer Support

[Balakrishna]

Hi @Caguicla_Intel 

Thankyou for the replay.

I have sent you requested info and setup diagram through email. Kindly verify it.

Thanks

[Caguicla_Intel]

Hello Balakrishna,

Thank you for the swift response.

Yes, we received your reply via email. Please allow us to check this query with our engineers. We will get back to you as soon as possible but no later than 2-3 business days.

Best regards,

Crisselle C

Intel® Customer Support

[Caguicla_Intel]

Hello Balakrishna,

Thank you for the patience on this matter. Kindly refer to below information for the feedback from our higher level.

Can you confirm if you already tried the recommendations provided in the User Guide?

Please refer to the detailed documentation in the Ethernet User Guide below on section called “Optimizing Performance”, page 39.

https://downloadcenter.intel.com/download/28410/Adapter-User-Guide-for-Intel-Ethernet-Adapters

Will you also be able to check if you can capture the issue using Wireshark Trace and provide us the logs. Wireshark trace with .cap file. Once we have the logs, we will continue to further investigate your concern.

Looking forward to your reply.

We will follow up after 3 business days in case we don't hear from you.

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Balakrishna,

Good day!

This is just a follow up to check if you have read our previous message. If yes, kindly provide the additional information requested for us to continue investigating this query.

Looking forward to your reply.

In case we don't receive your reply, we will reach out after 3 business days.

Best regards,

Crisselle C.

Intel® Customer Support

[Balakrishna]

Hi @Caguicla_Intel ,

Sorry for the late response.

Tried different tuning and optimizing parameters, but still facing the same issue.

I will collect and share tcpdump/wireshark logs over the email soon, Kindly verify once you received.

I remember "TCP previous segment lost" message was seeing in wireshark logs when I was debugging the issue.

Please consider below note:

I have Intel 10Gig single port adapters, here I did not see any performance issue.

[Caguicla_Intel]

Hello Balakrishna,

No problem at all, appreciate your effort in sending us a reply.

We are sorry to hear that you are still facing the same issue after trying out different tuning and optimizing parameters.

We will patiently wait for your email and rest assured that we will let you know once received. (We still haven't got the email as of this moment)

Regarding the note you've shared, we hope you don't mind sharing the photos of the adapter via email as well and configuration where the adapter was installed. This would help with the investigation.

Awaiting to hear from you soon. 

We will follow up after 3 business days in case we don't receive an update. 

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Balakrishna,

Good day!

This is just a follow up for the additional information that we requested. We will also re-send our email for your convenience.

We hope to hear from you soon. 

Should there be no response from you, I’ll make sure to follow up after 3 business days.

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Balakrishna,

I hope you're doing great!

Please be informed that we will now close this request since we haven't received any response from our previous follow ups. Just feel free to post a new question if you may have any other inquiry in the future as this thread will no longer be monitored.

Best regards,

Crisselle C.

Intel® Customer Support