Cannot upgrade firmware to 2.42 on E810-C

[karlatec]

I cannot upgrade firmware to version 2.42 or 2.50 on E810-CQDA2 network adapter.

FW packages downloaded from: 

https://downloadcenter.intel.com/download/29736/Non-Volatile-Memory-NVM-Update-Utility-for-Intel-Ethernet-Network-Adapter-E810-Series

My current driver and FW versions are:

$ sudo ethtool -i enp216s0f0
driver: ice
version: 1.5.8
firmware-version: 2.30 0x80005d1b 1.2877.0

NIC lspci info:

3b:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02)
        Subsystem: Intel Corporation Ethernet Network Adapter E810-C-Q2

                Product Name: Intel(R) Ethernet Network Adapter E810-CQDA2
                Read-only fields:
                        [V1] Vendor specific: Intel(R) Ethernet Network Adapter E810-CQDA2
                        [PN] Part number: K91258-006
                        [SN] Serial number: B49691A40750

When I try to update FW to version 2.42 or 2.50 I get following error message:

$ sudo ./nvmupdate64e

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.35.62.7
Copyright(C) 2013 - 2021 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [***+......]
Tool execution completed with the following status: The configuration file could not be opened/read, or a syntax error was discovered in the file.
Press any key to exit.

sudo ./nvmupdate64e -u -l -o update.xml -b -c nvmupdate.cfg

        Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.37.5.6
Copyright(C) 2013 - 2021 Intel Corporation.WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [+.........]
Tool execution completed with the following status: The configuration file could not be opened/read, or a syntax error was discovered in the file.
Press any key to exit.
[sys_sgci@spdk-WFP-05 Linux_x64]$ sudo ./nvmupdate64e -u -l -o update.xml -b -c nvmupdate.cfgIntel(R) Ethernet NVM Update Tool
NVMUpdate version 1.37.5.6
Copyright(C) 2013 - 2021 Intel Corporation.Config file read.
Error:   Config file GFID doesn't match NVM image version [config: Current: 0x01571590, Original: 0x01571590]
                                                          [image:  Current: 0x00000000, Original: 0x00000000].

My system is Fedora 33 5.8.15-300.fc33.x86_64.

Looking forward for any advice how can I upgrade FW to latest version, thanks!

[Mike_Intel]

Hello karlatec,

Thank you for posting in Intel Ethernet Communities. 

For us to further check and investigate the issue, please provide the following details.

1. Are you using a PCIe E810-C or this is onboard?
2. If this is a PCIe card, can you provide some picture of the card on both sides focusing on the markings?
3. What is the brand and model of your system?

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[karlatec]

Hey Michael!

This is a PCIe E810-C card. Unfortunately I can't provide any pictures at the moment as I'm working remotely. I may be able to provide some tomorrow or early next week.

The system I use is a Supermicro SYS-2029U-TN24R4T with X11DPU motherboard.

This problem is not exclusive to a single system or a single card. I have 4 E810 NICs in total which I cannot upgrade and some of them are installed in Intel systems: R2208WFTZSR with S2600WFT motherboard (same OS distro and kernel)

K.

[Mike_Intel]

Hello karlatec,

Thank you for the quick reply. I understand, I will wait for your reply with photos of the cards for us to further check and validate.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[karlatec]

Hello Michael,

I managed to take photos of one of the cards. Please let me know if it contains relevant information.

K.

[Mike_Intel]

Hello karlatec,

Thank you for providing the photos that we requested. We need to check this and also the issue that you encounter. Please give us 2 to 3 days to check this and provide an update.

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[Mike_Intel]

Hello karlatec,

Thank you for being patient on this thread. We are currently checking this issue. While doing that, you can also provide the following details.

1. Is this the first time that you tried to update the firmware?
2. Is this the first time that you encounter this issue?
3. Please help generate the SSU of the affected system, please download the utility below:

https://downloadcenter.intel.com/download/26735/Intel-System-Support-Utility-for-the-Linux-Operating-System?wapkw=ssu

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[karlatec]

Hello Michael,

No, it's not the first time FW update was attempted. FW on my NICs was heavily outdated so I had to go up the ladder with versions starting from the oldest: 2.02 -> 2.15 -> 2.22 -> 2.32 and then finally 2.32 -> 2.42 (which failed).

There was reboot done after each FW update.

Yes, that's the first time I encounter a FW update issue. Updates for previous versions were successful.

SSU outputs for two different systems where the problem occurs attached.

[Mike_Intel]

Hello karlatec,

Thank you for providing the quick response. We need to check these logs and get back to you once we find something. Please give us 2 to 3 days to check this and provide an update.

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[Mike_Intel · Accepted solution]

Hello karlatec,

Thank you for patiently waiting for our update. You may try the following steps to update the firmware:

1. Look into the nvmupdate.cfg file for the 2.4 update. You will need to edit the config file.

https://downloadcenter.intel.com/download/30451/Non-Volatile-Memory-NVM-Update-Utility-for-Intel-Ethernet-Network-Adapters-E810-Series-EFI

\E810_NVMUpdatePackage_v2_42_EFI\E810\EFI2x64\nvmupdate.cfg

2. Base on SSU, your Ethernet adapter's ETRACK ID is 80005D1B. Search it in the config file. you would find the section with 80005D1B in the REPLACES line. Similar below:

NVM IMAGE: E810_CQDA2_O_SEC_FW_1p5p4p5_NVM_2p40_PLDMoMCTP_80007062_pldm_fixed_signed.bin

EEPID: 80007062

SKIP NETLIST: FALSE

IMAGE DOWNGRADE: TRUE

RESET TYPE: REBOOT

CURRENT GFID: 0157-1590

ORIGINAL GFID: 0157-1590

REVISION: 02

; REPLACES: 80003D96 800049C3 80004FB3 80005D1B

END DEVICE

3. Remove these two lines in that section (Current and Original GFID):  

CURRENT GFID: 0157-1590

ORIGINAL GFID: 0157-1590

It will look like this then save. Try the NVM update again. 

NVM IMAGE: E810_CQDA2_O_SEC_FW_1p5p4p5_NVM_2p40_PLDMoMCTP_80007062_pldm_fixed_signed.bin

EEPID: 80007062

SKIP NETLIST: FALSE

IMAGE DOWNGRADE: TRUE

RESET TYPE: REBOOT

REVISION: 02

; REPLACES: 80003D96 800049C3 80004FB3 80005D1B

END DEVICE

4. If the above does not work. Please share log outputs with us.

Please run these commands in order to collect logs

export QV_DEBUG_LOG=0xFFFFFFFF

export NUL_DEBUGLOG=1

./nvmupdate64e -l -c nvmupdate.cfg

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[karlatec]

Hi Michael, thanks for the info.

I was able to finally upgrade FW on all systems and NICs to 2.42 and rebooted all of the systems.

However after upgrading to 2.50 and rebooting this issue occurs and the systems fail to boot:

[Mike_Intel]

Hello karlatec,

Thank you for the update and I am sorry to hear about the boot issue. Let me clarify some details first before we proceed.

1. Did you follow our steps when you updated the firmware to 2.4?
2. Can you still boot the system when the firmware is 2.4?
3. What step did you follow in updating the firmware to 2.5?
4. Any other troubleshooting steps that you tried so far?
5. Can you share s screenshot on where the system stops booting.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[Mike_Intel]

Hello karlatec,

I hope this message finds you well. I am just sending a follow up on the information that we requested below for us to check the issue.

1. Did you follow our steps when you updated the firmware to 2.4?
2. Can you still boot the system when the firmware is 2.4?
3. What step did you follow in updating the firmware to 2.5?
4. Any other troubleshooting steps that you tried so far?
5. Can you share s screenshot on where the system stops booting.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[Mike_Intel]

Hello karlatec,

I am just sending another follow up on the information that I requested. 

Since we have not heard back from you, I need to close this inquiry. 

If you need further assistance, please post a new question. 

Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician