firmware update failed on modular server

[idata]

Hi,

I tried to upgrade the firmware on my Intel modular server from version 6.8 to 6.10. I shut down all servers and put the SCMs in maintenance mode. Unfortunatly, the MMC seems to have crashed during the upgrade process - the MMC didn't came back online for 2 hours. By puling the MMC and reinserting it the MMC came finally back and seems to run under the new firmware. The GUI keeps telling me now that there's still an upgrade pending for the SCMs and CMs (MFS5520VI) but nothing happens. I connected to the serial ports of the SCMs and it looks like they try to get the new firmware from the internal tftp server (1.1.1.254) but the file doesn't exist: "Can't load 'cbay_all_3_10_0140_02.img' Error transferrig file over tftp!". Via "execs" command they boot the old firmware without problems. The serial ouput of the CMM also indicates some missing/corrupted files: "GetFileSize: stat error (No such file or directory), can_t get file size for /sd0/tftpboot/fw/CBM-VI/vifw012701.ima". Looks like, the internal SD Card on the backplane is somehow corrupted.

Is there a way to recover the system? I've already tried to upload the same firmware without any success. Would it be possible to restore the missing files on the SD card? Can the SD card be formated (jffs2) and the firmware transfered manually?

Best regards,

slarty

[Edward_Z_Intel]

Seems you'll need an Emergency Firmware Recovery. http://www.intel.com/support/feedback.htm?group=server# server Contact Intel Customer Support for the procedure.

[idata]

Thanks, Edward. I've already contacted customer support some days ago - but no answer yet... ;-(

I've managed to pull an image off the backplane sdcard and to mount it on a linux machine. I transfered the image to another sdcard and the IMS boots from it - but still isn't able to finish the firmware upgrade. Seems, the sdcard itself is not the problem. The firmware of the SCM seems to be there, but the SCMs dont't find it - maybe it's in the wrong directory. The new SCM firmware is 2 MB smaller than the old one - maybe it was truncated? The other missing file seems to be the BMC firmware for the compute modules. Flashing the new BIOS on the CMs works via EFI shell, but I've no idea how to update the BMC firmware. The missing BMC firmware is obviously the reason for the CMM not "seeing" the CMs.

Do you know what happens, if I delete the faulty SCM firmware image off the sdcard? Is the CMM extracting it again automatically from the upgrade package?

Best regards,

Andreas

[idata]

Still no word from intel support... :-(

I solved the problem myself now anyway... I copied the sdcard to an image (dd), converted the image to little endian (jffs2dump) and mounted it as jffs2 on an mtd unter linux. Then I basically deleted any firmware files under /tftpboot and /fw while leaving the directory structure intact. After that I dd'ed the mtd back to an image, converted it back to big endian again and flashed it on another sdcard. The CMM booted to EBF and I transferd the CMM_Core.zip firmware package over by ftp. After the upload was complete (which took ages...) the firmware upgrade worked like a charm.