Xeon E5/E7 memory power management documentation

[Alex_K_1]

I am looking for the CPU datasheet or other form of documentation that would describe the DRAM power management configuration in E5/E7 class CPUs. I was able to find the document named "Intel® Xeon® Processor E5 v2 Product Family Datasheet- Volume Two: Registers", which is very useful, but seemingly incomplete. In particular, in Section 7.1.2, the description of MEM_ACCUMULATED_BW_CH_ register references the PM_CMD_PWR register which does not appear anywhere in the datasheet.

In addition, I am looking for the description of MCHBAR in those processors. Any information I could find refers to desktop and Xeon E3 CPUs only.

Any help is appreciated.

Regards,

Alexey

[Patrick_F_Intel1]

Hello Alexey,

Yes, I see that the PM_CMD_PWR is not defined in volume 2 but, at least for the system I ran on, the value of the register is non-zero. If the uncore lets one measure the CAS, RAS and precharge events then you could work out the values of the PM_CMD_PWR registers.

Vol 2 also doesn't mention MCHBAR or how to get read it (if it is even used on this platform).

What information are you looking for?

Pat

[Alex_K_1]

Thanks Pat,

I am looking for a way to measure DRAM power consumption and, more importantly, to control how memory controller puts memory ranks into lower-power states. In desktop processors, this is done through MCHBAR registers like PM_PDWN_config. I have found no indication of a similar configuration for higher-end server CPUs.

This is a research project.

Thanks.

regards,

Alexey

[Patrick_F_Intel1]

Have you looked at the DRAM_ENERGY_STATUS and DRAM_ENERGY_STATUS_CH[0:3] registers? These registers appear to show the DRAM energy used.

DRAM_POWER_INFO: Spec DRAM Power (DRAM_TDP) seems to show the max expected DRAM power usage.

There are various fields that look like they might impact DRAM power but I don't know enough about it to really be able to say. What sort of fields are you able to change on other platforms? Are you able to change DRAM power options in the BIOS?

Pat

[Alex_K_1]

Thanks Pat.

Yes, I have looked at DRAM_ENERGY_STATUS* and I am getting reasonable data from them. I believe data in those registers comes from the RAPL domain. This is the best I could get at the moment but I have some concerns about accuracy of that data. RAPL reports are estimations based on internal counters and the model that is used to produce them need calibration. I see no way the system can accurately calibrate DRAM power consumption model when modules with different power profiles may be installed by a user. I guess the reported values represent how much DRAM composed of "typical" modules is likely to consume.

Anyway, I was looking for the kind of control that seems to be available though PM_PDWN_config and PM_CMD_PWR MCHBAR registers. It appears that E5/E7 Xeons do not contain the PCI ID for the memory controller part that allows me to determine the location of MCHBAR. However, the DRAM power configuration is a basic concept and I would expect the server IMC to behave largely similarly. I wondered if any documentation is available on that topic.

I have not experimented with other platforms yet and the BIOS setting are quite enable/disable type of thing.

Regards,

Alexey