S2600STBR CPU2 D1 E1 DIMM slots not working

[MShepard]

S2600STBR server board s/n FWSP01222069 J17012-552
BIOS SE5C620.86B.02.01.0012.070720200218
BMC FW 2.48.89b32e0d
ME FW 04.01.04.381
FRU Ver 1.76

2x Intel Silver 4214 CPU
8x Micron 32GB DDR4-2400 ECC SDRAM Part# 36ASF4G72PZ-2G3B1
P4304XXMUXX chassis, 2x 750W PS, 2x 4x35 HSBP

New system with ESXi 6.5 has been running flawlessly for several weeks with 4x 32GB DIMM = 128GB in CPU1 A1,B1 and CPU2 A1,B1.

Today installing another 4x 32GB DIMM to populate 4 memory channels on each CPU.

CPU1 reports 32GB in A1,B1,D1,E1.  CPU2 reports 32GB in A1,B1, reports D1,E1 uninstalled.

I have swapped DIMMS at least 4 different ways, all DIMMS reported in the 6 working slots, all DIMMs report "not installed" in CPU2 D1,E1 slots.

Is there something else I need to do to enable these slots or something I should try?
Is this a known problem with this board?

SysInfo log files are attached.  Config has 8 DIMMs in A1,B1,D1,E1 of each CPU.

Best regards,

Mark Shepard

[JoseH_Intel]

Hello MShepard,

Thank you for joining the Intel community

Let me understand well. Is the total amount of memory per CPU (32GB) reported correctly in BIOS even when slots D1, E1 show up as uninstalled? In the sysinfo log it shows 6 DIMMs populated, 4 in CPU1 and 2 in CPU2.

By any chance have you tested 1 single DIMM in slots D1 or E1? Could you try a minimal configuration like 1 DIMM in CPU1 and 1 DIMM (either slot D1 or E1) in CPU2. 

What manufacturer and model are your memory modules? I am assuming all of them are same model, size and properly validated.

In case you want to take a look I am forwarding you the Technical Product Specification document so you can check the memory population rules https://www.intel.com/content/dam/support/us/en/documents/server-products/Intel_S2600ST_TPS.pdf#page=48

I will look forward for your updates

Regards

Jose A.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit:

https://intel.com/support/serverbios

[MShepard]

Greetings, Jose,

The amount of memory reported in BIOS is the sum of the memory that shows as installed.  With 8 DIMMs as described, 6 show as installed and the amount of memory is shown as 192GB.

With a single DIMM in CPU1-A1 and a single DIMM in CPU2-D1 or E1 or F1 the system boots but BIOS shows only 32GB of RAM and only a single DIMM in CPU1-A1.  The same DIMM in CPU2-A1, B1 or C1 is reported correctly.

All 8 modules are identical Micron 32GB DDR4-2400 ECC SDRAM Part# 36ASF4G72PZ-2G3B1 as reflected in the sysinfo_log, which includes serial numbers.  The Micron 36ASF4G72PZ is a validated memory module for the S2600STBR.  The configuration was designed in conformance to the TPS.

I have swapped CPU1 and CPU2 and the problem remains exactly the same, it stays with the board and does not move with the CPU.  I examined both CPU sockets under magnification and do not see any damage to the pins.

Best regards,

Mark

[JoseH_Intel]

Hello MShepard,

It indeed looks like a possible hardware failure on the actual DIMM slot. Let me check for the warranty possibilities and I will let you know as soon as I have an update.

Regards

Jose A.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit:

https://intel.com/support/serverbios

[JoseH_Intel]

Hello MShepard,

Our senior team has determined that your issue indeed looks like a possible board malfunction and we will be able to process a warranty replacement for you. I will send you an email so you can reply with the information needed so we can generate the warranty order

Regards

Jose A.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit:

https://intel.com/support/serverbios

[JoseH_Intel]

Hello MShepard,

A warranty order has been created. I shared the details with you through your personal email. I will mark this thread as closed now. If you have further issues or questions just go ahead and submit a new topic.

Regards

Jose A.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit:

https://intel.com/support/serverbios