Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
4970 Discussions

Cannot Enable RDMA for Intel E810-CAM2 on Windows 11 Pro for Workstations

mkuhler
Beginner
4,439 Views

I am getting low transmission speeds (max 10Gbit/s) with the Intel E810-CAM2 NIC, and suspect the issue might be RDMA is not properly enabled, as I have not been able to set it properly. I am using Windows 11 Pro for Workstations, and am getting no results using the Get and Set-NetAdapterRDMA commands in powershell (screenshot attached). I wonder if this is due to an incompatibility between the drivers and Windows 11 Pro for Workstations, but cannot be sure. Both network cards show that they're auto-negotiated link speed is 100Gbit/s.

 

My configuration for troubleshooting is two identical PCs connected via fiber with 100G compatible transceivers directly to the network cards. I've also tested transfer speeds through a 100G FS switch, with no discernable difference. To test I've been using iperf3 and observing file transfers between ~4GB files (screenshots of results below). For reference, here is some additional configuration information:

 

I've tried modifying the network settings on both ends with negligible impact on speed as follows:

  • jumbo packet size: 9014 bytes
  • speed & duplex: auto negotiate
  • RDMA VF resource profile: enabled
  • RDMA maximum VFs enabled: 4
  • NetworkDirect Technology: both iWARP and ROCEv2 modes

 

As I have so far been unable to get/set RDMA settings for this NIC, my suspicion is this is what's causing the low speeds, but I cannot be entirely sure.

My questions are as follows:

  • Do I have the correct drivers for Windows 11 Pro for Workstations? Has this NIC/drivers been tested with Windows 11 Pro for Workstations?
  • How do I enable RDMA on Windows 11 Pro for Workstations for this NIC?
  • Are there any other possible reasons for this bottleneck?

Thank you.

Labels (1)
0 Kudos
12 Replies
B_Y
Employee
4,357 Views

Hello @mkuhler,

Thank you for posting in Intel Ethernet Communities. 

In order to aid in our inquiry, we would first like to ask for your assistance in providing the following information, thank you in advance:

1. Please provide us a photo of the markings both front and back of your network adapter for us to validate the network adapter. 

2. Can you share with us on the system report and adapter configuration? This will allow us to check your Adapter details and configuration (other than the one you have mentioned). Guide for the Intel® System Support Utility as following:

a- Go to https://downloadcenter.intel.com/download/25293/Intel-System-Support-Utility-for-Windows-?product=91600 and download the software.

b- Open SSU.exe

c- Mark the box "Everything" and then click "Scan".

d- When finished scanning, click "Next".

e- Click on "Save" and attach the file to a post.

 

3. I also just want to know if you're having any connection problems. 

 

Looking forward to your response.

 

Best regards,

BY_Intel

Intel Customer Support

 

0 Kudos
mkuhler
Beginner
4,320 Views

Hello @B_Y ,

Thank you for the response. As for validating the NIC, the exact product that I purchased can be found at this link: https://www.fs.com/products/141788.html See attached for photos of the NIC (NIC_Front.jpg, NIC_Back.jpg) and the file generated by Intel's system support utility (Intel_System_Report.txt).

 

Also no, I am not having any connection problems, the only issue is with transmission speed. Please let me know what you suggest.

Thank you.

0 Kudos
B_Y
Employee
4,252 Views

Hello @mkuhler,

Thank you for your patience.


For RDMA please refer Adapter User Guide for Intel® Ethernet Adapters below, page# 66 "4.26 Remote Direct Memory Access (RDMA)":

https://www.intel.com/content/www/us/en/download/19373/adapter-user-guide-for-intel-ethernet-adapters.html

To avoid performance degradation from dropped packets, enable link level flow control or priority flow

control on all network interfaces and switches


From the the Intel System report, it is showing Intel(R) Ethernet Network Adapter E810-C-Q2 instead of E810-CAM2. Do you see anything unusual from the device manager at Network adapters?

 

Best regards,

BY_Intel

Intel Customer Support


0 Kudos
mkuhler
Beginner
4,225 Views

Hello @B_Y ,

Thank you for the additional information. I am out of office and currently unable to test this, but I will modify the settings you mentioned and look at the drivers sometime next week.

 

Thank you,

Madison

0 Kudos
B_Y
Employee
4,182 Views

Hello @mkuhler,

Thank you for let us know. In case we do not hear from you, I will make a follow up on July 25th 2023.

 

Best regards,

BY_Intel

Intel Customer Support


0 Kudos
B_Y
Employee
4,093 Views

Hello @mkuhler,


I hope this message finds you well. I am just sending a follow up if you have any questions from the previous post. If so, please let us know. Thank you.

 

Best regards,

BY_Intel

Intel Customer Support


0 Kudos
mkuhler
Beginner
4,037 Views

Hello @B_Y ,

Thanks for following up. To respond to your earlier post following up about RDMA, 

That is correct, the device manager lists the network adapters as Intel(R) Ethernet Network Adapter E810-C-Q2. I installed the drivers from the Intel Complete Driver Pack on the E810-CAM2 product page (https://www.intel.com/content/www/us/en/products/sku/187410/intel-ethernet-controller-e810cam2/downloads.html). While I see drivers specific to the device, they appear to be for Linux OS. To your knowledge is that device name incorrect, and I need different drivers? If so, where would they be?

As for enabling link level flow control or priority flow control, I don't see those exact parameters but I have the following settings:

  • Flow Control = Rx & Tx enabled
  • NetworkDirect Technology = iWARP

Is the Flow Control setting the specific setting that enables priority flow control? If so, I already have it enabled to no effect. 

0 Kudos
B_Y
Employee
3,998 Views

Hello @mkuhler,


My apologize, please disregard previous comment on device manager. It is because the Intel(R) Ethernet Network Adapter E810-C-Q2 works as expected and E810-CAM2 is the controller.

1) Please refer Adapter User Guide for Intel® Ethernet Adapters below,

https://www.intel.com/content/www/us/en/download/19373/adapter-user-guide-for-intel-ethernet-adapters.html

- Page# 66, Subject: "4.26 Remote Direct Memory Access (RDMA)":

On devices with RDMA capabilities, RDMA is supported on the following operating systems (refer to

"Supported Operating Systems" on page 2 for currently supported versions):

- Linux

- FreeBSD

- ESXi

- Windows Server


Even though Windows 11 supports RDMA, the Intel E810 series' RDMA capabilities are only supported by that OS's 4 versions.


For Original Equipment Manufacturer(OEM) version of Ethernet adapters (since the purchase was from FS and the adapter has its label) , we'd like to set your expectation that we have a limited support and it would be best if you can get in touch with manufacturer (FS) for first opinion since they may have customized drivers, firmware and software to enable or alter features on your card. The software drivers linked from our page are generic versions, and can be used for general purposes. To avoid any potential installation incompatibilities on your OEM card, Intel recommends that you check with your OEM and use the software they provide so you don’t lose features or customizations of the Ethernet adapter. 

It seems FS may included drivers with the package. We would recommend to update the E810-CAM2's firmware to the latest version. We want to avoid using the Non-Volatile Memory (NVM) Update Utility for Intel® Ethernet Network Adapters E810 Series first because it might brick non-Intel adapter.


You may also follow the "Optimizing Performance" guide on page# 55 in Adapter User Guide for Intel® Ethernet Adapters.


Best regards,

BY_Intel

Intel Customer Support


0 Kudos
mkuhler
Beginner
3,969 Views

Hello @B_Y ,
I did see that section on page 66 regarding supported operating systems. In that section you quoted it also says "(refer to "Supported Operating Systems" on page 2 for currently supported versions)". And in that list of supported operating systems on page 2 Microsoft Windows 11 22H2 is listed. Is that list on page 2 incorrect?

 

I have used the optimizing performance guide on page 55 to tune settings, and I am still seeing <10G speeds.  None of the settings listed provide a performance boost dramatic enough to explain why I'm still seeing such low speeds. 

 

I am also currently in contact with a representative at FS and they have not provided any specific software or drivers. I haven't used the Non-Volatile Memory (NVM) Update Utility either.

 

Is this purely an incompatibility issue with the intel drivers and Windows 11 Pro for Workstations?

 

Thank you

0 Kudos
B_Y
Employee
3,937 Views

Hello @mkuhler,


Thank you for the highlight. While the information page:2 may highlight on the OS support for an adapter (the release notes from the the drivers from the Intel Complete Driver Pack on the E810-CAM2 product page also didn't enclose the RDMA support for Windows 11).

- Could you please let me know if you have conducted any further tests utilizing a different system with a comparable or superior setup (PCIe 4.0 (16 GT/s), Data Rate Per Port:100Gb/s, Fiber)?

- Any difference if you perform the copying direct from switch, different switch port or fiber module?

- Is the speed duplex regardless full duplex or auto negotiation applied with the same policy at this system: sender, linked device (switch), destination: receiver?


Best regards,

BY_Intel

Intel Customer Support



0 Kudos
B_Y
Employee
3,769 Views

Hello @mkuhler,


I hope this message finds you well. I am just sending a follow up if you have any questions from the previous post. If so, please let us know. Thank you.


Best regards,

BY_Intel

Intel Customer Support


0 Kudos
B_Y
Employee
3,640 Views

Hello @mkuhler,


Please be informed that we will now close this request since we haven't received any response from our previous follow ups. Just feel free to post a new question if you may have any other inquiry in the future as this thread will no longer be monitored.


Best regards,

BY_Intel

Intel Customer Support


0 Kudos
Reply