S2600GZ with more than 256GB of memory

[СКаще]

Hi!

We are trying to get R2312GZ4GC4 server platform working with more than 256GB of RAM

Prod. code: R2312GZ4GC4

Serial: FHBF244003B

TA. NO: G63765-002

Mainboard PBA: G11481-352

Mainboard serial: QSGR23100049

We have latest firmware installed from this download - http://downloadmirror.intel.com/22512/eng/S2600GZ_GL_EFI_BIOS01060002_ME20107112_BMC1174207_FRUSDR109.zip http://downloadmirror.intel.com/22512/eng/S2600GZ_GL_EFI_BIOS01060002_ME20107112_BMC1174207_FRUSDR109.zip

Dual Xeon E5-2680 installed.

Memory modules - 32GB Hynix HMT84GR7MMR4A-H9D3

Maximum working amount of memory - 256GB (8 DIMMs installed in blue memory slots)

If we add more DIMMs - system does not start at all or start with some memory slots marked as fault by LEDs on the board (depends on combination of used slots)

I can't understand the cause of the problem. Server configurator tool list this memory modules as supported and allow to configure up to 24 modules for this platform.

We have 2 identical server platforms on hands and situation with memory idetical on both of them.

[Edward_Z_Intel]

As per https://www.skhynix.com/products/computing/view.jsp?info.ramKind=20&info.serialNo=HMT84GR7MMR4A https://www.skhynix.com/products/computing/view.jsp?info.ramKind=20&info.serialNo=HMT84GR7MMR4A, your DIMM is quad-rank. S2600GZ supports up to 8 ranks per memory channel, so in your case two DIMMs per channel.

For more details, refer to the board http://www.intel.com/support/motherboards/server/sb/CS-033134.htm Technical Product Specification (TPS), section 3.2.4.

[СКаще]

S2600GZ - 24 DIMM slots – 3 DIMMs/Channel – 4 memory channels per processor

8 memory channels total with dual CPU configuration

we have only one DIMM per channel (all DIMMs in blue slots) - this is 4 ranks per channel

at least it must work with 2 quad-rank DIMMs per channel if 8 ranks per channel is limitation

[Edward_Z_Intel]

Ahh sorry for the misunderstanding - I thought you were talking about each processor...

Have you tried to install 8 DIMMs on one processor and see if it works?

[СКаще]

Yes we try to remove all memory modules from channels E-H

System does not start correctly with more than 4 modules in this configuration

Also we try to start system using only channels A & B - system start correctly only with 2 DIMMs installed in A1 & B1.

If modules installed in A1, A2, A3, B1 then system starts, but modules A1, A2, A3 marked as fault by LEDs and BIOS displays only 32GB of RAM (B1 module).

If modules installed in A1, A2, B1, B2 then system does not start.

[Edward_Z_Intel]

So it confirms that the issue is not about total capacity, but each memory channel accepts only one DIMM... Not sure if it's related to the DIMM using DDP (Dual Die Packate) and having "8 internal banks"...

[Daniel_O_Intel]

You can also try going into the BIOS and changing

Advanced > System Acoustic and Performance Configuration > DCLTT

See if that helps the boot any.

[СКаще]

It's a miracle, it works