Intel E810 - Unable to change channel count

[ronagy]

We are currently looking into changing our Supermicro NIC configuration to use E810 instead of X710. We are using a AMD EPYC 7543 32-Core Processor.

As part of our configuration we isolate two cpus and assign 2 channels/queue to these cpus (SMP/XPS affinity). This works fine on X710. However, on E810 it doesn't seem possible to change the channel count:

sos-gfx.local:~# ethtool -L eth0 combined 2 
netlink error: Resource busy 
sos-gfx.local:~# ethtool -L eth0 combined 40 
netlink error: requested channel count exceeds maximum (offset 36) 
netlink error: Invalid argument 
sos-gfx.local:~# ethtool -L eth0 combined 32 && echo OK 
OK

Is there a way around this or do we need to look at a different card?

The reason we need to do this is because our machines are quite heavily loaded and we have other devices that cause IRQ storms (e.g. GPUs) which can make the NIC drop RX packets. By isolating all NIC processing to 2 dedicated cores we can guarantee minimal latency without packet loss. Something that is not possible (as far as we know) if we are unable to configure the number of channels.

tv2m-srv1.local:~# ethtool -i eth2
driver: ice
version: 6.6.89-0-lts
firmware-version: 3.20 0x8000d853 1.3146.0
expansion-rom-version: 
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
tv2m-srv1.local:~# ethtool eth2
Settings for eth2:
	Supported ports: [ FIBRE ]
	Supported link modes:   1000baseT/Full
	                        25000baseCR/Full
	                        25000baseSR/Full
	                        1000baseX/Full
	                        10000baseCR/Full
	                        10000baseSR/Full
	                        10000baseLR/Full
	Supported pause frame use: Symmetric
	Supports auto-negotiation: No
	Supported FEC modes: None
	Advertised link modes:  25000baseSR/Full
	                        10000baseSR/Full
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	Advertised FEC modes: None
	Speed: 25000Mb/s
	Duplex: Full
	Auto-negotiation: off
	Port: FIBRE
	PHYAD: 0
	Transceiver: internal
	Supports Wake-on: d
	Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
	Link detected: yes

[IntelSupport]

Hello Ronagy,

Thank you for posting in the Intel Community Forum.

Based on your inquiry, we understand that you are unable to change the channel count on the E810 adapter, which is installed in a Supermicro server.

To assist you further, please provide the following details:

• Full name/model of the NIC adapter
• Was the adapter shipped with the server or purchased separately?
• Has it worked correctly before?
• A clear picture of the NIC adapter showing all visible markings

Best regards,

Shankith K P

Intel Customer Support Technician

[ronagy]

- Intel® Ethernet Network Adapter E810-XXVDA2

- Purchased and installed separately
- No, but it does work with our current Intel X710 NICs as well as Broadcom and other NICs we use.
- I don't have a picture of the adapter and I'm not sure how/whether that is relevant.

[TejasMohan]

Hello Ronagy,

Greetings from Intel!

Thank you for writing back and sharing the details.

We kindly request an image showing the label of the adapter to validate the card details.

As you mentioned that the server board is from Supermicro,  we recommend reaching out to the OEM to check if they can assist with the configuration and support of this card with the board.

Please don’t hesitate to contact us for any further assistance.

Thank you for using Intel products and services.

Best regards,

Tejas

Intel Customer Support Technician

[ronagy]

> As you mentioned that the server board is from Supermicro,  we recommend reaching out to the OEM to check if they can assist with the configuration and support of this card with the board.

I just need to you to confirm whether or this specific card supports the feature and if yes, what might cause your drivers to refuse changing the channel count. 

I've referenced you driver code: https://github.com/intel/ethernet-linux-ice/blob/765663d4bc8dd8fc8b78f2e54e2fd3ddd1af6c75/src/ice_ethtool.c#L5663

However, the failure seem to occur in either ice_vsi_recfg_qs or netif_is_rxfh_configured whose implementation is not visible.

[ronagy]

Here are the pictures

[TejasMohan]

Hello Ronagy,

Greetings from Intel!

Thank you for writing back and sharing the details.

Please find below the links to the datasheet and user guide for the adapter, which provide additional information:

Intel Ethernet Controller E810 Datasheet

https://www.intel.com/content/www/us/en/content-details/613875/intel-ethernet-controller-e810-datasheet.html?wapkw=E810%20it%20doesn%27t%20seem%20possible%20to%20change%20the%20channel%20count&DocID=613875

Intel Ethernet Network Adapter E810-XXVDA4T User Guide

https://www.intel.com/content/www/us/en/content-details/646265/intel-ethernet-network-adapter-e810-xxvda4t-user-guide.html?wapkw=E810%20it%20doesn%27t%20seem%20possible%20to%20change%20the%20channel%20count&DocID=646265

Additionally, you may consider updating the firmware and drivers for this adapter to the latest versions to see if that helps resolve the issue.

As mentioned previously, please check with the OEMs regarding card compatibility and configuration-related information.

Please don’t hesitate to contact us for any further assistance.

Thank you for using Intel products and services.

Best regards,

Tejas

Intel Customer Support Technician

[ronagy]

I take it that you don't know whether or not this feature is supported? If you can't figure it out then I'm unsure how you expect me to? 
If you can't tell me then we'll just look for another card.

Again, all I need to know is whether or not this feature is supported. I would prefer not wasting time trying to debug something that is expected behavior.

[TejasMohan]

Hello Ronagy,

Greetings from Intel!

Thank you for writing back and sharing the details.

We’ve taken note of your question regarding the feature support, and we’re currently working to confirm this internally .

This may take a little time, but we’ll make sure to get back to you as soon as we have a definitive answer.

Appreciate your patience, and we’ll keep you posted with our findings.

Best regards,

Tejas

Intel Customer Support Technician

[Fikri_Intel]

Hi ronagy,

Good day to you.

Upon checking, please proceed to install the latest Driver version 2.3.10 and NVM version 4.90 for your Intel® Ethernet E810 adapter. This will help ensure optimal performance and compatibility.

Additionally, kindly verify whether your current Operating System (OS) is listed in the supported OS section for the E810. You may refer to Section 1.3.1 of the attached document for details.

https://cdrdv2-public.intel.com/864646/Intel%20Ethernet%20Controller%20Products_Release%20Notes_30.4_v2.pdf

You can download the complete driver pack from the following link:

https://www.intel.com/content/www/us/en/download/15084/intel-ethernet-adapter-complete-driver-pack.html

Once downloaded, please use the folders below to install the respective components:

== Driver Folder: \Release_30.4.2\PROCGB\Linux

== NVM Folder: \Release_30.4.2\NVMUpdatePackage\E810

Please share the installation results once completed, so we can proceed with further verification if needed.

Regards,

Fikri O.

[ronagy]

Hello Fikri

Does that mean that you confirm that the feature should be supported and you are trying to help us figure out why it doesn't work?

[Amina_Sadiya]

Hi ronagy,

Thank you for your response. To assist you with your query, please confirm whether you have performed the steps mentioned above and share the installation results once completed, so we can proceed with further verification if needed.

Best regards,

Amina

Intel Customer Support Technician

[ronagy]

Thank you for your responses. Unfortunately, since you don’t appear to have sufficient knowledge of your own product and have been unable to answer straightforward questions, we’ve decided to look for another manufacturer.

If you later determine whether your product supports changing the channel count, please feel free to reach out — we will be happy to reconsider.

For the time being, we will not be responding to further verification inquiries.

[Amina_Sadiya]

Hi ronagy,

Apologies for the inconvenience caused. We are doing our best to find a resolution to your query. Kindly allow me some time to check this internally and get back to you with an update.

Best regards,

Amina

Intel Customer Support Technician

[ronagy]

If you have clear answers please do provide them. We still have a server with an Intel 800 Series NIC installed for testing and will probably keep it for another week or two.

[Simon-Intel]

Hi Filip,

Thank you for your patience.

We wish to inform you that we tested the command and its working. Please see below screenshot. Driver version 1.15.4 and NVM 4.6 on Ubuntu 24.04.4 LTS.

Regards,

Simon

Intel Customer Support Technician

[blockdev]

Hello,

Building and using the drivers from the package mentioned above gave us a bit more information in the dmesg log.

It turns out that it's not possible to change channels when RDMA is active. The issue was simply resolved by unloading the irdma module:

rmmod irdma

While I appreciate that the Intel staff here cannot possibly know the specifics of our systems and environment, I still think one could expect that upon raising the question, it could simply have been pointed out that this feature is unavailable when RDMA is active.

It is solved, thanks.

[Akshaya1]

Hi blockdev

Thank you for the update and for sharing the findings. Glad to hear the issue has been resolved. Your feedback is appreciated.

We will proceed to close the case as the issue is resolved.

Please let us know if you need any further assistance.

Regards,

Akshaya

Intel Customer support Technician