Intel S2600WF server unable to flash X722 onboard io module

[jamendola]

Good Day.

I work at a manufacturing plant where I have run into an issue that I have encountered before awhile ago where when the S2600WF server was shipped to us it seems to have been programmed with a different IO Module than what is installed.

There are 2 IO Module types that are shipped to us with our servers.  One which is a Fiber Type and one which is a Copper type.

The one that is installed in the server is a Fiber type.

But when viewing the Ethernet devices via lspci command it shows the onboard IO Module different from what is installed.

This is what we should see:

banco-1-11:~ # lspci -nn | grep ^3d
3d:00.0 Ethernet controller [0200]: Intel Corporation Device [8086:37d2] (rev 09)
3d:00.1 Ethernet controller [0200]: Intel Corporation Device [8086:37d2] (rev 09)
3d:00.2 Ethernet controller [0200]: Intel Corporation Device [8086:37d3] (rev 09)
3d:00.3 Ethernet controller [0200]: Intel Corporation Device [8086:37d3] (rev 09)

As you see from above it sees the 2 37d2 onboard  copper ports along with the 2 37d3 onboard fiber ports.

However, this is what shows on the bad server:

s2600wf-99-8:/var/opt/teradata/teradata-server-firmware-S2600WF/X722_NVM_v5.30 # lspci | grep ^3d
3d:00.0 Ethernet controller: Intel Corporation Device 37d2 (rev 09)
3d:00.1 Ethernet controller: Intel Corporation Device 37d2 (rev 09)
3d:00.2 Ethernet controller: Intel Corporation Device 37d2 (rev 09)
3d:00.3 Ethernet controller: Intel Corporation Device 37d2 (rev 09)

The command above acts like all of the onboard ports are copper.

I have seen this previously on one other server that was shipped to us.

That was solved by reflashing the onboard network with the X722 firmware.

However, on the current server I am not able to do this.

I am able to start the flash but when it gets to 85% then it fails saying "update failed":

There is no other message as to why and I cannot find any log file to lead me in another direction.

I have tried any/every option that the nvmupdate64e script has via the help option.

Note that we have tried a different IO Module to see if this resolved the issue.

To me this is probably just a programing issue so with some help I might be able to fix it.

The plan is to ship the bad server back to the supplier but if there is anyone that can assist me to avoid this that would be great.

[jamendola]

Here is the output from the flash.

s2600wf-99-8:/var/opt/teradata/teradata-server-firmware-S2600WF/X722_NVM_v5.30 # $NVMUPDATE64E -rd -l -o $IVENTORY_LOG_FILE_AFTER -c $NVMUPDATE64E_CFG

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.37.1.1
Copyright(C) 2013 - 2021 Intel Corporation.

WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Config file read.
Inventory
[00:061:00:00]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Flash inventory started.
Shadow RAM inventory started.
Alternate MAC address is not set.
Shadow RAM verification started.
|>.....................[ 0%]......................|
Difference found at offset 0x100 [Device=0xA2, Buffer=0x40] - update required.
NVM update is required.
Shadow RAM verification finished.
Flash inventory finished.
OROM inventory started.
OROM inventory finished.
PHY NVM inventory started.
PHY NVM inventory finished.
[00:061:00:01]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Device already inventoried.
[00:061:00:02]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Device already inventoried.
[00:061:00:03]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Device already inventoried.

Num Description Ver.(hex) DevId S:B Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Connection X722 5.48(5.30) 37D2 00:061 Update
for 10GBASE-T available

Options: Adapter Index List (comma-separated), [A]ll, e[X]it
Enter selection: A
Would you like to back up the NVM images? [Y]es/[N]o: N
Update
[00:061:00:00]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Flash update started.
|======================[ 85%]=============>........|
Error: Flash update failed.

Num Description Ver.(hex) DevId S:B Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Connection X722 5.48(5.30) 37D2 00:061 Update failed
for 10GBASE-T

Post update inventory
[00:061:00:00]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Flash inventory started.
Alternate MAC address is not set.
Flash inventory finished.
OROM inventory started.
OROM inventory finished.
No PHY NVM update taken - skipping inventory.
[00:061:00:01]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Device already inventoried.
[00:061:00:02]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Device already inventoried.
[00:061:00:03]: Intel(R) Ethernet Connection X722 for 10GBASE-T
Device already inventoried.

Tool execution completed with the following status: An error occurred when updating a firmware module.
Press any key to exit.
s2600wf-99-8:/var/opt/teradata/teradata-server-firmware-S2600WF/X722_NVM_v5.30 #

[jamendola]

Thanks for the response.

Note that we have already replaced the motherboard in the node to resolve issue.

Thanks,

Jason Amendola

[Dineshbabu]

Hello jamendola,

Greetings for the day!

Please allow us some time to review the details and we will get back with an update shortly.

Regards,

Dinesh S

[Azeem_Intel]

Hi jamendola,

Greeting for the Day!

1) we request you please share physical Ethernet module and a picture of the product marking label on the module

2) As per you wording "it seems to have been programmed with a different IO Module than what is installed", did you buy the IO module as part of a S2600WF L9 system?

If Yes bought the module as part of a L9, please share a picture of the product marking label of the L9 system,

If you did not buy a L9 system, please share for the invoice of the module and the L6 S2600WF system, also collect a picture of the product marking label of the S2600WF L6 system.

Please do not hesitate in contact us, we are more than glad to assist you.

Best Regards,
Azeem_Intel.

[jamendola]

Azeem_Intel,

Note I no longer have access to the hardware but all of the hardware in the server come preinstalled when shipped from the factory.

Here is the description of the hardware:

Server, Chassis Assy, Intel [R1]R1208WFTYSR, (2) 2.2 Ghz Cascade Lake 5220R CPU, 768GB, (2) 1.6TB SSD, OCP X527-DA2, 1300W PSU

Thanks,

Jason

[Azeem_Intel]

Hi jamendola,

Greeting for the Day!

Hope you are doing well. Based on the information provided, it seems that it could be either an L9 or OEM system. Could you please share a picture of the product marking label of the L9 system? Alternatively, if available, please provide the invoice for the module and the L6 S2600WF system.,

Please do not hesitate in contact us, we are more than glad to assist you.

Best Regards,

Azeem_Intel.

[Azeem_Intel]

Hi jamendola,

Greetings for the Day!

We are following up to find out if you were able to find the information we provided. Please reply to confirm, so we can continue helping on a resolution. Looking forward to receiving your reply.

Best Regards,

Azeem_Intel

[jamendola]

Azeem_Intel,

This issue can be closed.

The motherboard has already been replaced so the issue can be considered as resolved.

If we have another server that has this issue I can always reopen.

Thanks,

Jason Amendola

[Azeem_Intel]

Hi jamendola,

Greeting for the day!

I hope you are doing well. Thank you for providing details regarding the case. We will close the case as of now.

Please do not hesitate in contact us, we are more than glad to assist you.

Best Regards,

Azeem_Intel.