Unable to update firmware for x710 update not available

nkoconno · ‎05-01-2024

Hello, I'm running ESX8

Here's my current firmware and drivers;

esxcli network nic get -n vmnic0
Advertised Auto Negotiation: true
Advertised Link Modes: Auto, 100BaseT/Full, 1000BaseT/Full, 10000BaseT/Full
Auto Negotiation: true
Backing DPUId: N/A
Cable Type: Twisted Pair
Current Message Level: 1
Driver Info:
Bus Info: 0000:a1:00:0
Driver: i40en
Firmware Version: 8.30 0x8000a9b7 1.2926.0
Version: 1.11.3.10
Link Detected: true
Link Status: Up
Name: vmnic0
PHYAddress: 0
Pause Autonegotiate: true
Pause RX: false
Pause TX: false
Supported Ports: TP
Supports Auto Negotiation: true
Supports Pause: true
Supports Wakeon: true
Transceiver: internal
Virtual Address: 00:50:56:58:bb:5d
Wakeon: MagicPacket(tm)



esxcli network nic get -n vmnic5
Advertised Auto Negotiation: true
Advertised Link Modes: Auto, 100BaseT/Full, 1000BaseT/Full, 2500BaseT/Full, 5000BaseT/Full, 10000BaseT/Full
Auto Negotiation: true
Backing DPUId: N/A
Cable Type: Twisted Pair
Current Message Level: 1
Driver Info:
Bus Info: 0000:44:00:1
Driver: i40en
Firmware Version: 8.50 0x8000be1e 1.3082.0
Version: 1.11.3.10
Link Detected: true
Link Status: Up
Name: vmnic5
PHYAddress: 0
Pause Autonegotiate: true
Pause RX: false
Pause TX: false
Supported Ports: TP
Supports Auto Negotiation: true
Supports Pause: true
Supports Wakeon: true
Transceiver: internal
Virtual Address: 00:50:56:5d:ec:26
Wakeon: MagicPacket(tm)

Attempting to use the 9.40 firmware from here: https://www.intel.com/content/www/us/en/download/18190/non-volatile-memory-nvm-update-utility-for-intel-ethernet-network-adapter-700-series.html

I copied the tar file up to a vmfs datastore and extracted;

700Series_NVMUpdatePackage_v9_40_ESX.tar

I ran the following; 

[root@esx02:/vmfs/volumes/64680e35-8b8e8790-91a8-3ceceffcf592/intel/700Series/ESXi_x64] ./nvmupdaten64e

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.40.5.5
Copyright(C) 2013 - 2023 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [|.........]


Num Description Ver.(hex) DevId S:B Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Controller X710 8.80(8.50) 15FF 00:068 Update not
for 10GBASE-T available
02) Intel(R) Ethernet Controller 8.48(8.30) 1589 00:161 Update not
X710/X557-AT 10GBASE-T available


Tool execution completed with the following status: Device not found.
Press any key to exit.

The same process works fine on esx7, but on gigabyte hardware vs supermicro

How can I upgrade the firmware on my cards while running esx8?

Fikri_Intel · ‎05-01-2024

Hi nkoconno,

Thank you for reaching Intel Community.

Based on the issue, we can see there are mismatch with Device Driver and Firmware Version in ESXI 8.0 U2.

Your current version:

Driver: i40en

Firmware Version: 8.30 0x8000a9b7 1.2926.0

Version: 1.11.3.10

Driver: i40en

Firmware Version: 8.50 0x8000be1e 1.3082.0

Version: 1.11.3.10

Kindly refer to the below link from VMware Compatibility Guide for upgrade the firmware while running ESXi 8.0 U2 :

https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=io&productid=37994

Regards,

Fikri_Intel

nkoconno · ‎05-02-2024

Thanks, but that is the link to acquire the certified drivers from vmware for the intel nic. My issue is with firmware upgrades.

Here's the drivers/firmware on my esx7 host - fw 8.10 / driver 1.11.1.31

[root@esxi2:~] esxcli network nic get -n vmnic2
   Advertised Auto Negotiation: true
   Advertised Link Modes: Auto, 100BaseT/Full, 1000BaseT/Full, 2500BaseT/Full, 5000BaseT/Full, 10000BaseT/Full
   Auto Negotiation: true
   Cable Type: Twisted Pair
   Current Message Level: 1
   Driver Info:
         Bus Info: 0000:41:00:0
         Driver: i40en
         Firmware Version: 8.10 0x800093ea 1.2829.0
         Version: 1.11.1.31
   Link Detected: true
   Link Status: Up
   Name: vmnic2
   PHYAddress: 0
   Pause Autonegotiate: true
   Pause RX: false
   Pause TX: false
   Supported Ports: TP
   Supports Auto Negotiation: true
   Supports Pause: true
   Supports Wakeon: false
   Transceiver:
   Virtual Address: 00:50:56:59:4c:63
   Wakeon: None

Firmware update successful 8.10 to 9.4

[root@esxi2:/vmfs/volumes/661148bc-c11d7ae0-14ce-d85ed369312c/intel/700Series/ESXi_x64] ./nvmupdaten64e

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.40.5.5
Copyright(C) 2013 - 2023 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [***|......]


Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Network Adapter   8.16(8.10)   15FF 00:065 Update
    X710-T2L                                                     available

Options: Adapter Index List (comma-separated), [A]ll, e[X]it
Enter selection: a
Would you like to back up the NVM images? [Y]es/[N]o: y
Update in progress. This operation may take several minutes.
[***.......]


Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Network Adapter   9.64(9.40)   15FF 00:065 Update
    X710-T2L                                                     successful

A reboot is required to complete the update process.

Tool execution completed with the following status: All operations completed successfully.
Press any key to exit.

After reboot driver 1.11.1.31 / firmware 9.40

[root@esxi2:~] esxcli network nic get -n vmnic2
   Advertised Auto Negotiation: true
   Advertised Link Modes: Auto, 100BaseT/Full, 1000BaseT/Full, 2500BaseT/Full, 5000BaseT/Full, 10000BaseT/Full
   Auto Negotiation: true
   Cable Type: Twisted Pair
   Current Message Level: 1
   Driver Info:
         Bus Info: 0000:41:00:0
         Driver: i40en
         Firmware Version: 9.40 0x8000ecc3 1.3429.0
         Version: 1.11.1.31
   Link Detected: true
   Link Status: Up
   Name: vmnic2
   PHYAddress: 0
   Pause Autonegotiate: true
   Pause RX: false
   Pause TX: false
   Supported Ports: TP
   Supports Auto Negotiation: true
   Supports Pause: true
   Supports Wakeon: false
   Transceiver:
   Virtual Address: 00:50:56:59:4c:63
   Wakeon: None

While not relevant for this issue; Upgrade drivers, reboot

[root@esxi2:/vmfs/volumes/661148bc-c11d7ae0-14ce-d85ed369312c/intel] esxcli software component apply -d /vmfs/volumes/hddds00/intel/Intel-i40en_2.7.2.0-1OEM.700.1.0.15843807_22938502.zip
Installation Result
   Components Installed: Intel-i40en_2.7.2.0-1OEM.700.1.0.15843807
   Components Removed: Intel-i40en_1.11.1.31-1vmw.703.0.20.19193900
   Components Skipped:
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true

all upgraded to vmware spec; firmware 9.4 / driver 2.7.2.0

[root@esxi2:~]  esxcli network nic get -n vmnic2
   Advertised Auto Negotiation: true
   Advertised Link Modes: Auto, 100BaseT/Full, 1000BaseT/Full, 2500BaseT/Full, 5000BaseT/Full, 10000BaseT/Full
   Auto Negotiation: true
   Cable Type: Twisted Pair
   Current Message Level: 0
   Driver Info:
         Bus Info: 0000:41:00:0
         Driver: i40en
         Firmware Version: 9.40 0x8000ecc3 1.3429.0
         Version: 2.7.2.0
   Link Detected: true
   Link Status: Up
   Name: vmnic2
   PHYAddress: 0
   Pause Autonegotiate: true
   Pause RX: false
   Pause TX: false
   Supported Ports: TP
   Supports Auto Negotiation: true
   Supports Pause: true
   Supports Wakeon: false
   Transceiver:
   Virtual Address: 00:50:56:59:4c:63
   Wakeon: None

The ESX v7 hosts are using the same DevID card (15FF) as the v8 hosts. Intel's own instructions say to upgrade the firmware first, then the driver.

Outside of ESX, I booted to a debian live CD and had the same problem (see attached). this does not appear to be an issue with esx.

Outside of Debian, I ran this on a TrueNAS Scale machine with the same issue

root@truenas01[/tmp/intel/700Series/Linux_x64]# ./nvmupdate64e

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.40.5.5
Copyright(C) 2013 - 2023 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [|.........]


Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Controller X710   8.80(8.50)   15FF 00:025 Update not
    for 10GBASE-T                                                available
02) Intel(R) Ethernet Controller        8.48(8.30)   1589 00:202 Update not
    X710/X557-AT 10GBASE-T                                       available


Tool execution completed with the following status: Device not found.
Press any key to exit.

Here's the information on the x710 NICs in my TrueNAS server;

root@truenas01[/tmp/intel/700Series/Linux_x64]# ethtool -i enp25s0f0
driver: i40e
version: 6.1.55-production+truenas
firmware-version: 8.50 0x8000be1e 1.3082.0
expansion-rom-version:
bus-info: 0000:19:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
root@truenas01[/tmp/intel/700Series/Linux_x64]# ethtool -i enp202s0f3
driver: i40e
version: 6.1.55-production+truenas
firmware-version: 8.30 0x8000a9b7 1.2926.0
expansion-rom-version:
bus-info: 0000:ca:00.3
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Given that this problem exists for us across operating systems, I don't think this is an issue with Debian Live CD, TrueNAS and ESX.

What other options do we have to get the Intel firmware updated to 9.4?

Fikri_Intel · ‎05-02-2024

Hi nkoconno,

Thank you for your response.

Based on the issue, would you be able to confirm with us the below inquiries:

1) Has ESXi 7 - Running on Gigabyte Hardware Board (successful upgrade)?

2) Has ESXi 8 - Running on SuperMicro Hardware Board (Failed, Device not found)?

3) Debian, TrueNAS is running on the same SuperMicro Hardware?

Along with that, please share the PBA of the adapter. You may refer to the link below on where to find the PBA number. Providing photos of the adapter focusing on the markings (white sticker) found on the physical card will be highly appreciated for us to double-check on it. The PBA consists 6-3 digit number located at the last part of the serial number. This would help us identify if you are using an Original Equipment Manufacturer(OEM) or retail version of Intel Ethernet Adapter.

Identify Your Intel® Network Adapter Model Using PBA Number

https://www.intel.com/content/www/us/en/support/articles/000007022/network-and-i-o/ethernet-products.html

Kindly advise.

Regards,

Fikri_Intel

Fikri_Intel · ‎05-06-2024

Hi nkoconno,

Hope you are doing well.

Just wanted to follow up, we have not heard your response to the inquiry in the previous post.

Kindly let us know for further analysis.

Regards,

Fikri_Intel

Fikri_Intel · ‎05-09-2024

Hi nkoconno,

Hope you are doing well.

This is regarding your enquiry above.

Since we have not seen any updates, we will be closing this forum case from our end.

If you need any additional information, please submit a new question as this thread will no longer be monitored.

Regards,

Fikri_Intel