What kind of latency to expect from a X710-T2L?

[Fruit]

Hi!

Today I hooked up two X710-T2Ls with a simple ethernet cable in between and ran some pings. I got RTTs in the .7ms range. This surprised me, as even two simple onboard gigabit NICs in the same configuration have a latency of around .15ms.

Before I start experimenting I would really like to know what kind of latencies I should expect from properly configured X710-T2L cards… I figured that 10 gbit/s NICs like these would go below .1ms easily but perhaps my expectations are wrong?

Thanks in advance!

[Caguicla_Intel]

Hello Fruit,

Thank you for posting in Intel Ethernet Communities. 

Before we proceed checking your query, kindly provide the following information.

1. Current driver and firmware version used

2. Operating System used

3. Cable type used

4. PBA of the adapter. You may refer to the link below on where to find the PBA number. It consist of 6-3 digit number located at the last part of the serial number. This would help us identify if you are using an Original Equipment Manufacturer(OEM) or retail version of Intel Ethernet Adapter.

Identify Your Intel® Network Adapter Model Using PBA Number

https://www.intel.com/content/www/us/en/support/articles/000007022/network-and-i-o/ethernet-products.html

Awaiting to hear from you. 

We will follow up after 3 business days in case we don't receive a reply.

Best regards,

Crisselle C.

Intel Customer Support 

[Fruit]

Hi!

Thank you for your quick reply.

Unfortunately it seems my question was not entirely clear from my phrasing.

It is a generic product question: “what is this Intel product capable of”.

It is not about MY system, it's about a hypothetical “perfect” setup. Version numbers of my setup are therefore not relevant.

Apologies for the confusion and thanks in advance,

[Caguicla_Intel]

Hello Fruit,

Thank you for the clarification.

We should be the one apologizing for the confusion as you clearly asked the expected latency for X710-T2L and we assume that you need some troubleshooting on this. 

Please allow us to further check on your query. Rest assured that we will give you an update as soon as possible but no later than 2-3 business days. 

Hoping for your kind patience.

Feel free to let us know should you have additional questions or clarifications. 

Best regards,

Crisselle C.

Intel Customer Support 

[Caguicla_Intel]

Hello Fruit,

Good day!

Please be informed that we already escalated this request to our engineers for further checking and we are currently waiting for their feedback.

We hope you don't mind giving us additional time on this. We will give you an update as soon as we heard from them but no later than 2-3 business days.

Thank you for your kind patience. 

Best regards,

Crisselle C.

Intel Customer Support 

[Fruit]

Hi!
No problem at all, thank you for your effort!
Kind regards

[Caguicla_Intel]

Hello Fruit,

You're welcome, appreciate your kind understanding. 

This is just an update to inform you that we are still checking on this matter. We sincerely apologize for the delay and inconvenience that this has brought you. Rest assured that we will get back to you as soon as possible but no later than 2-3 business days.

Best regards,

Crisselle C.

Intel Customer Support 

[Caguicla_Intel]

Hello Fruit,

Good day!

We'd like to inform you that this request is still escalated to our engineers and we are currently waiting for their feedback. We will give you an update as soon as we heard from them but no later than 2-3 business days.

Thank you for your kind understanding. 

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Fruit,

We sincerely apologize as the answer to your query is taking longer than you expect.

Please be informed that we are still waiting for an update from our engineers. Rest assured that we will give you an update as soon as there is any but no later than 2-3 business days. 

We highly appreciate your kind patience on this matter. 

Best regards,

Crisselle C.

Intel® Customer Support 

[Caguicla_Intel]

Hello Fruit,

Good day!

Please be informed that this request is still escalated to our engineers. Rest assured that we will give you an update as soon as we heard from them but no later than 2-3 business days.

Best regards,

Crisselle C.

Intel Customer Support

[Caguicla_Intel]

Hello Fruit,

We sincerely apologize for the delay on our update. 

Upon series of checking, we regret to inform you that we do not have specific performance information for the X710-T2L. We have internal test/performance information, however to generalize performance there are too many hardware variables to take into consideration.  

What we have available is what is currently published in the Product Brief or Product specification. We also have this article that you can check for additional information. 

Tuning Throughput Performance for Intel® Ethernet Adapters

https://www.intel.com/content/www/us/en/support/articles/000005811/ethernet-products.html  

Intel® Ethernet Network Adapter X710-T2L/T4L Product Brief

https://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/ethernet-x710-t2l-t4l-brief.pdf

Intel® Ethernet Network Adapter X710-T2L Product specification

https://www.intel.com/content/www/us/en/products/sku/189463/intel-ethernet-network-adapter-x710t2l/specifications.html

Feel free to let us know if you have additional questions or clarifications. 

Looking forward to your reply.

Should there be no response, I’ll make sure to reach out after 3 business days. 

Best regards,

Crisselle C.

Intel® Customer Support

[Fruit]

Hello Crisselle,

Thank you for your reply. I've opened the links but none of them seem to mention specific latency estimates.

I'm not sure what you mean by variables, but please be aware that all I need is a ballpark figure. Something like “0.1 ms” or “0.7 ms” would be more than enough for me to decide whether I have a problem and need to set up a lab environment to provide you with the necessary support information. Basically, a simple ‘ping’ is all I'm asking for. Nothing more.

Kind regards

[Caguicla_Intel]

Hello Fruit,

Thank you for the reply. 

Please allow us to forward your reply to our engineers and ask for their insight. We will get back to you as soon as possible but no later than 2-3 business days. 

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Fruit,

Thank you for the patience on this matter. 

Please be informed that this is the only document available. This document may be helpful as it directly discusses Ethernet Latency.

Ethernet Latency & Network Latency Solutions

https://www.intel.com/content/www/us/en/products/docs/network-io/ethernet/achieve-low-ethernet-latency.html

We sincerely apologize as we do not have the Latency Estimates as this is highly environment dependent.

If you have questions or clarifications, please do not hesitate to ask.

Looking forward to your reply.

We will follow up after 3 business days in case we don't hear from you.

Best regards,

Crisselle C.

Intel® Customer Support

[Fruit]

Hi,

I tried to find some information using a search engine and found this:

https://www.cubewerk.de/2020/10/23/ceph-performance-guide-2020-for-ssd-nvme/

It's someone who successfully ran ‘ping’ on two x710 cards with a direct connection. The results for them were:

rtt min/avg/max/mdev = 0.039/0.061/0.105/0.017 ms

These are the kind of numbers I would expect from a 10 Gbit/s NIC. My results are:

rtt min/avg/max/mdev = 0.594/0.642/0.657/0.024 ms

That's a factor 10(!) slower, on average. I do think there's something wrong, here.

Lab setup used:

1. Driver/firmware version used: probably easiest to read from dmesg output:

   i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.16.11
   i40e: Copyright(c) 2013 - 2021 Intel Corporation.
   i40e 0000:02:00.0: fw 8.4.66032 api 1.14 nvm 8.40 0x8000aba9 1.2992.0

2. Operating system used: Ubuntu server 18.04.6 LTS, kernel 4.15.0-156-generic
3. Cable used: R&M Cat.6A 1 meter (tried other cables, made no difference)
   
4. PBA: K37379-005

The operating system and driver/firmware versions are the same on both sides. The systems are fresh installs; all I added was the Intel i40e driver.

Any assistance would be greatly appreciated!

Thanks in advance,

Fruit.

[Caguicla_Intel]

Hello Fruit,

Thank you for providing the details regarding your setup.

We will be glad to further investigate your concern. But before we proceed, we hope you don't mind providing the model of the SFP/transceiver you use. You may consider providing the markings of the transceiver for us to double check on it.

Looking forward to hear from you. 

Should there be no response, we will reach out after 3 business days. 

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Fruit,

Apologies for the confusion.

Please disregard the additional information that we are requesting for. You are using Intel® Ethernet Network Adapter X710-T2L and this doesn't use optical transceiver

We will now proceed further checking your query together with our higher level support. Rest assured that we will give you an update as soon as possible but no later than 2-3 business days. 

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Fruit,

Thank you for the patience on this matter. 

We'd like to set your expectation that we cannot fully compare the adapter's performance from other systems because we have to consider other factors such as system's specifications, configuration and other environment. We cannot guarantee to produce the same output from 2 different systems.

You mentioned that you are using driver version 2.16.11 which is the latest version from our site. Will you be able to try the lower version 2.15.9? Please be informed that it is just for testing purposes, and we would like to check if there will be any changes using the previous driver version.

Intel® Network Adapter Driver for PCIe* 40 Gigabit Ethernet Network Connections under Linux*

https://www.intel.com/content/www/us/en/download/18026/30641/intel-network-adapter-driver-for-pcie-40-gigabit-ethernet-network-connections-under-linux.html

Looking forward to hear from you. 

Should there be no response from you, I’ll make sure to reach out after 3 business days. 

Best regards,

Crisselle C.

Intel® Customer Support 

[Fruit]

Hi Crisselle,

I've just tested 2.15.9 (on both sides) and I'm seeing more or less the same results:

rtt min/avg/max/mdev = 0.585/0.638/0.657/0.032 ms

 For reference, the first few lines of dmesg output:

i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.15.9
i40e: Copyright(c) 2013 - 2021 Intel Corporation.
i40e 0000:02:00.0: enabling device (0000 -> 0002)
i40e 0000:02:00.0: fw 8.4.66032 api 1.14 nvm 8.40 0x8000aba9 1.2992.0
i40e 0000:02:00.0: The driver for the device detected a newer version of the NVM image v1.14 than expected v1.13. Please install the most recent version of the network driver.

(The warning was to be expected from an older driver.)

With regards to hardware differences: please note that I've observed these latencies on (so far) three systems, which all have different motherboards/chipsets (Supermicro, Asus) and different CPUs (Xeon, Core i7).

Also, I really don't think that a slowdown of 10× is something subtle that can be explained by a CPU running at 3.8 GHz or 4.0 GHz.

It should be easily reproducible—by the looks of it, on literally any hardware you have lying around—and if not, the difference in hardware will most likely be a useful aid in pinpointing the problem.

Thanks, and kind regards,

[Caguicla_Intel]

Hello Fruit,

Thank you for the effort in trying out the recommended step and sharing the result with us.

Please allow us to re-escalate this request to our higher level support for further investigation. We will get back to you as soon as possible but no later than 2-3 business days. 

Your kind patience and cooperation on this matter is highly appreciated. 

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Fruit,

Good day!

This is just an update to inform you that we already re-escalated this request to our engineers. We will give you an update once we heard from them but no later than 2-3 business days. 

Our apologies for the delay and inconvenience caused. 

Best regards,

Crisselle C.

Intel® Customer Support