RAPL values on Ivy Bridge

[BGoel]

I am trying to measure the power consumption of my Ivy Bridge chip using RAPL. When the CPU is Idle, I get following values from RAPL:

PKG Energy/s = 8.1 W

PP0 Energy/s = 2.4W

PP1 Energy/s = 0W (Probably not supported)

If I run a benchmark, I get following values

PKG Energy/s = 60.1 W

PP0 Energy/s = 53.06 W

However, if I try to verify these values using current measurement at ATX 8-pin CPU connector, I get 3.6W at idle and 55.2W when running the same benchmark as above. From what I understand, PKG energy values from RAPL are for the entire chip and PP0 values are for CPU cores. Hence, I would expect PKG values to be slightly less than what I measure at ATX connector (because of loss at voltage regulator) but not more. Is there something wrong in the way I have understood the RAPL measurements?

[McCalpinJohn]

RAPL energy estimates are not measurements -- they modeled based on processor activity.   They are accurate enough to use for p-state and thermal control, but are not intended to replace precision measurements.

There are also some RAPL-related errata in various processor versions, for which you may or may not have BIOS workarounds.  These BIOS workarounds are intended to remedy certain specific systematic errors, but don't make the fundamental mechanism any more accurate.

Given the approximate nature of the RAPL energy models, the 10% difference between RAPL and measured power is not surprising.

[Bernard]

Should not CPU internal thermal measurement be more accurate when compared to current measuremnt at ATX 8-pin connector?

[BGoel]

I understand what you are saying John. But from the figures I posted, the error is more than 100% at idle CPU(8.1W for RAPL vs 3.6W at ATX). Plus, when I do the same verification for Haswell, the RAPL values are amazingly accurate across the entire STAMP benchmark suite. 

[McCalpinJohn]

I can't speak for Intel, but I would imagine that being relatively accurate under high power consumption is a lot more important than being relatively accurate under low power consumption.

 I did not notice your measured "idle" value before, but it is interesting to note that RAPL is 4.5 Watts high for the idle case and 4.9 Watts high for the loaded case.  

It is important to recognize that there is significant variability in energy consumption across chips, even of the same processor model.  Folks in the overclocking world are used to this, but the same principles apply to the server chips.   The variation in power consumption across chips is an important contributor to the ultimate limit on frequency in overclocking experiments, but that variation is still there on all products.    It seems unlikely that Intel would go to the expense of "fine-tuning" the RAPL model for each individual piece of silicon -- the model is much more likely to be based on "generic" values for the implementation.

Your Haswell results might be more accurate because of luck, because of improvements to the RAPL models, or because of the ability of the Haswell chip to vary the voltage levels independent of the p-states (as described in document 330785 "Intel Xeon Processor E5 v3 Product Families Specification Update).

[McCalpinJohn]

In addition to the ability of the Haswell processors to dynamically adjust voltage levels based on temperature and or current draw, I just noticed that the Xeon E5 v3 data sheet (volume 1, document 330783-001) states that:

Invividual processor VID values may be calibrated during manufacturing such that two processor units with the same core frequency may have different default VID settings.

This ability to "tweak" the default voltages at manufacturing time may have the side effect of making the RAPL models more accurate.  Presumably the primary intent is to give Intel better yield and flexibility -- with tunable VID, more parts should be able to meet a frequency target while not exceeding the specified power budget.

It is not clear whether any of the coefficients of the RAPL model are also "tweaked" in manufacturing.  I would guess not (primarily because there should be a lot more coefficients in the RAPL model than in the VID tables), but it is possible that some of the more important coefficients -- such as those related to to leakage current vs temperature -- might need to be tuned for RAPL to remain accurate enough for power management.   It does not really matter very much --- if you need high-accuracy numbers you are going to need to need an external measurement anyway (even if only to prove to yourself that the RAPL numbers are accurate on a particular platform).

[BGoel]

Thanks for the insight John. I guess I overestimated the RAPL accuracy since I got carried away by the results on Haswell and as per Intel's slides (http://www.hotchips.org/wp-content/uploads/hc_archives/hc23/HC23.19.9-Desktop-CPUs/HC23.19.921.SandyBridge_Power_10-Rotem-Intel.pdf, slide 9), RAPL values are almost as good as actual measurement.