Bad Windows 10 Pro networking performance using an X550-T2 converged network adapter

[ma-hnln]

Hi,

I am working with a X550-T2 converged network adapter and would like to receive 10 Gb/s of UDP packets on some amount (e.g. 4) of target ports from the same source IP address. To check if my hardware and driver setup generally works, I am using iperf to generate and receive as much traffic as possible. Everything works perfectly fine (stability and performance wise) when running the receiving part of iperf using Linux, but not when using Windows 10 Pro (both on the same underlying hardware).

As iperf is not optimised for Windows, I additionally wrote a WinSock Registered IO receiver and came to the same conclusion than the iperf test: under Windows 10 Pro, I get very bad networking performance.

From there I simplified the test setup and tried single socket performance and only got to around 180k packets (@ size of 1k) per seconds without major drops. Beyond this number, drops get so overwhelming that is basically impossible to do anything with the incoming data. I also tried simple, blocking IO and came to the same conclusion: performance and stability is very bad under Windows 10 Pro.

To ensure that my code was not the cause of the observed behaviour on Win 10, I downloaded test setups from this series of blog posts . The results were equally bad. Additionally, the posts show that the expected amounts of packets per second, should be indeed much larger (i.e. >= 450k). Most interestingly, these numbers are from 2012, therefore I would expect to beat them with today's hardware and not be slower by a factor of almost 3.

I have configured the network adapter appropriately, e.g.: turning all UDP offload features on and off, maxing out the receive buffer, maxing out interrupt moderation, activating DMA coalescing, ...). Also, the receiving machine does not use SMT/HyperThreading. There is no active firewall and sender and receiver are connected back to back via a link that has proven to be capable of 10 Gb/s (via Linux).

Why is this test under perfect lab conditions working out well on Linux and basically not at all on Win10 Pro?

[ma-hnln]

Thanks for your continued efforts!

[Mike_Intel]

Hello ma-hnln,

We are now looking in to this issue, while doing that please check if you already tried the recommendations

provided in the user Guide? Please refer to the detailed documentation in the Ethernet User Guide on the link below:

Kindly go to section called “Optimizing Performance"

https://downloadcenter.intel.com/download/28410/Adapter-User-Guide-for-Intel-Ethernet-Adapters

And also, did you capture the issue using Wireshark Trace? If yes please provide us the logs. Wireshark trace with .cap file.

This is for us to further check this issue.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[ma-hnln]

Hi Michael,

first, there is no "Optimizing Performance" section, or I am just unable to find it. However, I have gone through all of the options listed under "Device Features". I have used different setups (e.g. everything off-loaded vs. nothing off-loaded, high and low interrupt moderation rates, ...) and I can say, that they are indeed helpful to at least get to 180k packets per second without major loss, meaning that I would not even get what I currently have without tweaking these options. Curiously, when I did my initial Linux based tests, I never touched any option of the NIC, it just worked...

With respect to WireShark, I want  to refer you to the thread I already linked in a previous post: https://community.intel.com/t5/Ethernet-Products/Why-I-have-loss-during-capture-data/m-p/1254800/highlight/true#M23837

There, the user reports that WireShark is not able to keep up with the test. I therefore suspect that its the same in my case and that WireShark will just miss represent the situation completely. Further, I have had WireShark on other Windows systems also doing a bad job in such situations. However, if you insist on a WireShark trace I will spin up the test system, install it and capture a trace.

And please, don't refer me to some other network (performance) testing tools like iperf and NTttcp (in UDP mode) - they are performing even worse (on Windows at least).

[Mike_Intel]

Hello ma-hnln,

Thank you for letting us know. For me to document this response from our last recommendations. Do you have a screenshot of "Device Features" since you mentioned that you cannot find the "Optimizing Performance".

And since you don't want to use wireshark, can you share some screenshots of your test for us to check. We need some data for us to further investigate the the issue that's why we are hoping you can try wireshark.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[ma-hnln]

Hi Michael,

I have attached a picture of the contents of the PDF you have linked me. The only section which comes close to what you wanted me to look at is the Device Features chapter. If I just downloaded the wrong PDF, please provide me with a direct link and not a link to a web site where I have to select the right one on my own.

I didn't say that I would not use WireShark, I just wanted to state that I don't see any value in the trace. Please state how the trace should look like:

• Should any filters be active?
• How long should the trace be?
• What information are you looking for?
• Where should I upload the trace (I am sure that it gets quite large)?

On the other hand, I could give you a test application that just works out of the box. I'm pretty sure that its not my adapter (as it worked just fine with Linux), so you are surely able to reproduce the issue on your own. I have attached the code (RIO.zip) - it is taken from the series of blog posts I have linked already: http://www.serverframework.com/asynchronousevents/rio/ The zip file contains everything necessary to build both sender and receiver, either via Visual Studio projects or cmake.

[Mike_Intel]

Hello ma-hnln,

Thank you for the update, we will try to check these data that you just provided and I will get back to you once I have an update on our own. Please give us 2 to 3 workings days to provide an update.

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[Mike_Intel]

Hello ma-hnln,

I hope you had a great weekend. While reviewing the SSU, we noticed that you are using driver version 4.1.219 which is 25.6. 

I know that you already told us in your previous post that you are using 26.0 however, it seems that it was not installed properly.

Can you try to Completely uninstall the previous driver and Proset first then install the latest driver below:

https://downloadcenter.intel.com/de/download/25016/Intel-Netzwerkadaptertreiber-f-r-Windows-10

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[Mike_Intel]

Hello ma-hnln,

I just want to check if you tried our last recommendation. Kindly update us once done for us to proceed with further investigation.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days.

Thank you.

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[Mike_Intel]

Hello ma-hnln,

I am sending another follow up on the information that I requested. 

Since we have not heard back from you, I need to close this inquiry. 

If you need further assistance, please post a new question. 

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[ma-hnln]

I was on vacation and just forgot to state this here...

Even if its not relevant any longer: I have already reinstalled the driver (as it was requested earlier). The SSU send to you was from an earlier date and indeed there could have been something wrong. However, as I have already stated: reinstalling everything did not solve the issue.

I'm just going to try another manufacturer and see if their hardware/software works better in conjunction with Windows.

[Mike_Intel]

Hello ma-hnln,

We understand and thank you for updating us that we will try other hardware.

If you need further assistance again in the future, please post a new question. 

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician