E810-XXVDA4

[FingerlessGloves]

I have 3 servers with a E810-XXVDA4, connected to a MikroTik CRS520-4XS-16XQ-RM, via QSFP28 to SFP28 breakout cable. The links come up at 25gbit as expected, and traffic flows.

The problem I have is, when the server reboots, it causes the QSFP28 port on the switch to reset all the other lanes. So QSFP28 port 1 is broke out in to 4, each one connected to each server, leaving the 4th breakout not being connected.  When one of the servers reboot, it causes the other two servers to loose their link temporarily. Sometimes it happens twice during the reboot, very occasionally it doesn't happen but it seems to happen most of the time. If I unplug the SFP28 manually, and plug it back in the issue doesn't happen. I've only had it happen during the POST stage, so I believe it's during the time the card initialises. I've spoken to MikroTik support and they believe the issue is with how the card initialises, since it doesn't happen when you manually unplug and replug the SFP28 cables.

I've tried different FEC modes, etc trying to sus out the issue, but the problem always happens. Disabling the OPTION ROM didn't help either.

HPE Gen 10 Servers, with NVM 4.90

OS: Proxmox 9 (Debian 13) with ICE driver from the provided Proxmox Linux Kernel

Is this a bug within the Intel card?

[vish1]

Hello FingerlessGloves,

Greetings for the day!!

To help us further investigate the issue you are experiencing, we kindly request you to share the following details:

1. Please confirm whether the Intel® Ethernet Network Adapter E810-XXVDA4 was purchased separately or if it came pre-installed with the system.
2. Kindly let us know the current firmware (NVM) version of the NIC.
3. Please confirm whether, during the issue, the entire QSFP port resets on the switch or if only a single breakout lane resets while others remain up.
4. If possible, we request you to perform a test by moving one server to:

• a different QSFP port on the switch, or
• a native SFP28 port (non-breakout),
  • and then reboot that server.

After performing the above test, please observe and confirm whether the other servers stop experiencing link flaps, which would help determine if the issue is related to QSFP breakout behavior.

Your inputs will help us narrow down the root cause and assist you more effectively. We appreciate your cooperation and look forward to your response.

Best Regards,

Vishal Shet P

Intel Customer Support Technician

[FingerlessGloves]

Hi Vishal,

1. The card was purchased with the HPE Gen 10 DL360 Server

2. Current NVM version is 4.90

3. All lanes within the QSFP28 port reset, no other QSFP28 ports are effected.

4. I have two CRS520-4XS-16XQ-RM, it's happening on both of those switches, no matter which QSFP28 port.

5. Not tested, but the non SFP28 ports aren't in a group, so there's no other lanes for it to reset. 

Only happens when the server restarts, unplugging the SFP28 from the E810 and plugging it back in, the problem doesn't happen. 

[IntelSupport]

Hello FingerlessGloves,

Thank you for your response.

Based on the details you have shared, we understand that this is an OEM (HPE) branded adapter since it was purchased along with the HPE system.

This issue occurs during the E810 initialization behavior in breakout mode. We recommend contacting the OEM (HPE) for further assistance with the adapter, as this configuration is not supported by Intel. This is important because OEM NVM firmware contains vendor‑specific modifications that Intel does not directly support.

Regards,

Shankith K P

​​​​​​​Intel Customer Support Technician