What kind of latency to expect from a X710-T2L?

[Fruit]

Hi!

Today I hooked up two X710-T2Ls with a simple ethernet cable in between and ran some pings. I got RTTs in the .7ms range. This surprised me, as even two simple onboard gigabit NICs in the same configuration have a latency of around .15ms.

Before I start experimenting I would really like to know what kind of latencies I should expect from properly configured X710-T2L cards… I figured that 10 gbit/s NICs like these would go below .1ms easily but perhaps my expectations are wrong?

Thanks in advance!

[Caguicla_Intel]

Hello Fruit,

Thank you for the patience on this matter. 

Please be informed that we have received update from engineering that we have two internal documents regarding performance, including latency for Fortville (not specifically the X710-T2L). This requires CNDA and RDC access so we will transfer this request to proper support group for guidance. Please expect an email within 24 to 48 hours.

Thank you for taking the time in reading this update and have a good day!

Best regards,

Crisselle C.

Intel® Customer Support

[Caguicla_Intel]

Hello Fruit,

Good day!

We are sending this message to share the document information below so you know what to download once you're granted for access.

Content ID: 601404

File Name: intelnda-3742-rhel73-x710-throughput.pdf

Title: Intel® Ethernet Converged Network Adapter X710-DA4 Red Hat* Enterprise Linux 7.3 Ixia* IxChariot* Throughput Data 

Content ID: 608776

File Name: 608776-intelnda-x710da4-latency-linux-4636.pdf

Title: Intel® Ethernet Converged Network Adapter X710-DA4 Latency Performance Red Hat* Enterprise Linux 7.5

Feel free to let us know if you have additional questions or clarifications. 

Best regards,

Crisselle C.

Intel® Customer Support

[Fruit]

Hi Crisselle,

I just got mail that I was granted “basic access” and another mail that “privileged access” was denied. It probably has to do with the fact that I'm a person and not a company. I do work for a company (or, to be precise, the IT department at a university) but I bought these NICs using my own money, as part of a lab setup.

When I use the search functionality of the “Resource & Design Center” I cannot find the documents you mentioned. Do you have a direct URL to them? Or do I really need the “privileged access”?

Kind regards

[Fruit]

A final note for those who found this thread after googling for information about why the latency on your X710-T2Ls is so terrible: Intel refused to help me, apparently because I'm not a company but a live human being. I even tried to register through my employer but to no avail: “the team's decision is final.”

To this day I do not know if they understand that the latency on these cards (and that probably means the entire line of cards, not just the specimens I own) is slow to the point of being defective.

My onboard NICs are faster. My $30 wifi AP is faster. The only device I could find in my household that has worse latency (but only by a little bit) is my old Linksys PAP2T ATA. It runs at 10 Mbit/s. Half-duplex.

That's right, these 10 Gbit/s cards have a latency that's almost as bad as a 15 year old 10 Mbit/s device.

Consider buying from a different manufacturer.

[Centos7]

A bit late to the thread, but here we are 14 months later and nothing has changed. I got the same runaround and was denied access to the documents. I've actually been able to get mean latency down to 28us but this is really unacceptable for a 10G card. I've put in an order for a NIC from a different manufacturer who has plenty of public information available about how to tune their cards.

If this is Intel's status quo, then no wonder they are doing as bad as they are.

[Fruit]

To me 28µs actually sounds quite acceptable for cards like these.

I never posted it here (because I got the information much later and through other channels) but my biggest improvement came from disabling the Energy-Efficient Ethernet (EEE) feature of these cards – from your results I assume you already found that setting.

But for anyone else who might find this useful, I have this in /etc/udev/rules.d/x710.rules:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="i40e", RUN+="/sbin/ethtool --set-eee $name eee off"

[Centos7]

I actually haven't seen that document or tried that setting. Will do and report back. It would be good to get a copy of that document as I think I'm barely touching the capability of this card. If I am, it is disappointing.

[Centos7]

Unfortunately, this setting has no effect (ethtool returns "operation not supported"). It's possible that it's disabled due to BIOS optimizations, though I have seen other threads about Intel NICs simply not responding to the flag and requiring a kernel command line setting to disable it (this was also not effective in my case).

[Fruit]

If you are currently seeing 28µs rtt latency then EEE is almost certainly already inactive. That might have various reasons, such as the other end not supporting it. In my setup I was using X710s on both ends. When I use one of those X710s with an X550 the EEE issue is not present – my X550 does not support EEE.

My original problem was that I was seeing latencies up to 700µs. Disabling EEE brought that down to <100µs. You are already doing way better than that. The udev code snippet was, if anything, intended to be useful to any stragglers who end up here in the future.

[Centos7]

28us is disappointing but I appreciate the help!