How to Increase the number of RSS queues on the E810 above 64 in new firmware

[pkielech]

Hello, 

I have problem with maximum single queues for Intel E810. I get max only 64 single queues. But I get new one adapter with 1.17.8 ice driver I get max 252 Combined queues (for dual AMD Epic 7713 256 cores) . 

root@agr35:/etc/bm/AGRyPrzeladowanie# cat /proc/interrupts | grep eth2- | wc -l 
128

root@agr35:/etc/bm/AGRyPrzeladowanie# ethtool -l eth2
Channel parameters for eth2:
Pre-set maximums:
RX: 252
TX: 252
Other: 1
Combined: 252
Current hardware settings:
RX: 0
TX: 0
Other: 1
Combined: 128

and it's works   I have for example 128 single queues.

I Try it on dual E810 CQDA2 NICs

01) Intel(R) Ethernet Network Adapter 4.96(4.60) 1592 00:065 E810-C-Q2
02) Intel(R) Ethernet Network Adapter 4.96(4.60) 1592 00:161 E810-C-Q2 

ice-eth2-TxRx-0
...
ice-eth2-TxRx-127

The same hardware, the same kernel  but firmware after upgrade or downgrade : 

root@agr53:/home/pkielech# ethtool -l eth2 
Channel parameters for eth2:
Pre-set maximums:
RX: 256
TX: 256
Other: 1
Combined: 256
Current hardware settings:
RX: 0
TX: 0
Other: 1
Combined: 128

and configured queues double (2 on one IRQ) 

root@agr53:/home/pkielech# grep ice-eth2-TxRx /proc/interrupts | awk '{print $NF}'|sed -E 's/[-0-9]+$//' |sort |uniq -c
64 ice-eth2-TxRx

information about network firmware and driver version 

root@agr55:/# ethtool -i eth2 
driver: ice
version: 1.17.2
firmware-version: 4.40 0x8001c96b 1.3534.0
expansion-rom-version: 
bus-info: 0000:41:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

CPU information

root@agr53:/home/pkielech# lscpu 
Architektura: x86_64
Tryb(y) pracy CPU: 32-bit, 64-bit
Rozmiary adresów: 48 bits physical, 48 bits virtual
Kolejność bajtów: Little Endian
CPU: 256
Lista aktywnych CPU: 0-255
ID producenta: AuthenticAMD
Nazwa modelu: AMD EPYC 7713 64-Core Processor
Rodzina CPU: 25
Model: 1
Wątków na rdzeń: 2
Rdzeni na gniazdo: 64
Gniazd: 2
Wersja: 1
Zwiększenie częstotliwości: włączone
Skalowanie MHz CPU: 155%
CPU max MHz: 2000,0000
CPU min MHz: 1500,0000
BogoMIPS: 4000,14
Flagi: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm exta
pic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_
llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin brs arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm
Właściwości wirtualizacji: 
Wirtualizacja: AMD-V
Pamięci podręczne (w sumie): 
L1d: 4 MiB (128 instancji)
L1i: 4 MiB (128 instancji)
L2: 64 MiB (128 instancji)
L3: 512 MiB (16 instancji)
NUMA: 
Węzłów NUMA: 2
Procesory węzła NUMA 0: 0-63,128-191
Procesory węzła NUMA 1: 64-127,192-255
Podatności: 
Gather data sampling: Not affected
Indirect target selection: Not affected
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Not affected
Reg file data sampling: Not affected
Retbleed: Not affected
Spec rstack overflow: Vulnerable, no microcode
Spec store bypass: Vulnerable
Spectre v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Spectre v2: Vulnerable; IBPB: disabled; STIBP: disabled; PBRSB-eIBRS: Not affected; BHI: Not affected
Srbds: Not affected
Tsx async abort: Not affected

NIC informations: 

- "Intel Corporation Ethernet Controller E810-C for QSFP (rev 02)"
Supports auto-negotiation:"Yes"
Advertised auto-negotiation:"Yes"
Link partner advertised auto-negotiation:"Yes"
Auto-negotiation:"on"
Availability:"eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500"
Capabilities:"[40] Power Management version 3"
Capabilities:"[50] MSI: Enable- Count=1/1 Maskable+ 64bit+"
Capabilities:"[70] MSI-X: Enable+ Count=2048 Masked-"
Capabilities:"[a0] Express Endpoint, MSI 00"
Capabilities:"[e0] Vital Product Data"
Capabilities:"[100] Advanced Error Reporting"
Capabilities:"[148] Alternative Routing-ID Interpretation (ARI)"
Capabilities:"[150] Device Serial Number 40-a6-b7-ff-ff-78-21-78"
Capabilities:"[160] Single Root I/O Virtualization (SR-IOV)"
Capabilities:"[1a0] Transaction Processing Hints"
Capabilities:"[1b0] Access Control Services"
Capabilities:"[1d0] Secondary PCI Express"
Capabilities:"[200] Data Link Feature <?>"
Capabilities:"[210] Physical Layer 16.0 GT/s <?>"
Capabilities:"[250] Lane Margining at the Receiver <?>"
Caption:"Not Available"
Default IP Gateway:"Not Available"
DHCP Enabled:"Yes"
DHCP Lease Expires:"Not Available"
DHCP Lease Obtained:"Not Available"
DHCP Server:"Not Available"
Driver:"ice"
Driver Path:"/lib/modules/6.6.90mplay1.0agrbgp-microcode-xapic-iommu/updates/drivers/net/ethernet/intel/ice/ice.ko"
Driver Provider:"Intel Corporation, <linux.nics@intel.com>"
Driver Version:""
Duplex:"Full"
Firmware Version:"4.40 0x8001c96b 1.3534.0"
Interface:"eth2"
I/O Ports:"Not Available"
IP Address:"Not Available"
IP Subnet:"Not Available"
MAC Address:"40:a6:b7:78:21:78"
Location:"43:00.0"
Manufacturer:"Intel Corporation"
Net Connection ID:"Not Available"
Port:"Direct Attach Copper"
Power Management:"Supports Wake-on: d"
Power Management:"Wake-on: d"
Speed:"100000Mb/s"
Supported link modes:"25000baseCR/Full, 25000baseSR/Full, 50000baseCR2/Full"
Advertised link modes:"25000baseCR/Full, 50000baseCR2/Full, 100000baseCR4/Full"
Link partner advertised link modes:"Not reported, Link partner advertised pause frame use: No"

The funniest thing is that the same kernel with kernel drivers allows using the full number of queues for each IRQ separately

root@agr53:/usr/src/ethernet-linux-ice/src# ethtool -i eth2
driver: ice
version: 6.6.90mplay1.0agrbgp-microcode-
firmware-version: 4.40 0x8001c96b 1.3534.0
expansion-rom-version: 
bus-info: 0000:41:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

and we have 128 IRQ and separately queues : 

root@agr53:/usr/src/ethernet-linux-ice/src# grep ice-eth2-TxRx /proc/interrupts | awk '{print $NF}'|sed -E 's/[-0-9]+$//' |sort |uniq -c
128 ice-eth2-TxRx

Can You help us to run it the same on newest driver ?

[IntelSupport]

Hello pkielech,

Thank you for posting in the Intel Community Forum.

To better understand your issue, could you please provide the following information:

1. Where is the adapter installed?
2. Please share the complete system details.
3. Was the adapter shipped with the system or purchased separately?

Best regards,

Shankith K P

Intel Customer Support Technician

[pkielech]

Hello, 

1) NIC was installed on Supermicro  Mainstream A+ Server AS -2024S-TR 

2)  2 x AMD EPYC 7713 and 2 x E810 CQDA2 NIC
 
3) W have multiple NIC witch chip Intel E810. Witch this servers (we have 8 the same and two AS-1024) we was bough  Supermicro NIC AOC-S100GC-i2C (Intel E810-CAM2 Ethernet Controller).  

We can't upgrade AOC-S100GC-i2C with NVM Update Utility 

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.43.20.0
Copyright(C) 2013 - 2025 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [|.........]


Num Description Ver.(hex) DevId S:B Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Controller N/A(N/A) 1592 00:065 Update not 
E810-C for QSFP available 
02) Intel(R) Ethernet Controller N/A(N/A) 1592 00:161 Update not 
E810-C for QSFP available

Tool execution completed with the following status: Device not found.

but we purchased separately new NIC on Ebay (also E810 CQDA2) and without upgrade on this nic we don't have limit 64 single queues. After upgrade (or later downgrade) this NIC with NVM Upgrade Utility new NIC work like another one - max 64 single queues. 

On older kernel/driver I don't have this limitation. Now I don't know what Can I do to get 128 or max 256 single queues. 

[Simon-Intel]

Hi pkielech,

Thank you for your response.

To review this further, I kindly request you to share front and back pictures of the network adapter with the product markings clearly visible. This will allow us to troubleshoot this issue more effectively.

Your prompt response will help us assist you better.

Best regards,

Simon

Intel Customer Support Technician

[pkielech]

Hello,

Below pictures of examples our NIC

more than 64 single queues: 

max64 single queues: 

Regards
Piotr Kielech

[Simon-Intel]

Hi pkielech,

Thank you for your response.

Please allow me sometime to review this internally. We will get back to you as soon as we have an update.

Best regards,

Simon

Intel Customer Support Technician

[Fikri_Intel]

Hi pkielech,

Good day to you.

We recommend updating both the driver and NVM firmware to the latest available versions to ensure optimal functionality and compatibility.

As this is a Dell-branded adapter, we suggest contacting Dell Support directly to obtain the correct and validated NVM and driver package for your specific adapter model. This will help ensure the updates are fully compatible with your system configuration.

Regards,

Fikri O.

[pkielech]

Hello, 

I make new Debian 13 install on new kernel 6.12.52. 

New OEM NIC : E810 2CQDA2
New driver : 2.3.10 
New firmware : 4.90 

root@agr35:/home/pkielech# ethtool -i enp65s0np0 
driver: ice
version: 2.3.10
firmware-version: 4.90 0x80020ef7 1.3863.0
expansion-rom-version: 
bus-info: 0000:41:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes

supports-priv-flags: yes

root@agr35:/home/pkielech# ethtool -l enp65s0np0
Channel parameters for enp65s0np0:
Pre-set maximums:
RX: 64
TX: 64
Other: 1
Combined: 64
Current hardware settings:
RX: 0
TX: 0
Other: 1
Combined: 64

root@agr35:/home/pkielech# uname -r 
6.12.52mplay1.1agrbgp-microcode-xapic-iommu

root@agr35:/home/pkielech# lspci | grep -i eth
41:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02)
43:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02)

...and stil max 64 queues. 

Also now I have new problem. When I use driver 2.3.10 I have problem with bonding on this network (on standard driver from kernel works fine) - LACP works fine but some sending packages was "desapere" 

Have You any other idea to use more than 64 single queues on E810 NIC ?