Power Consumption varies on Sandy-Bridge with SMT enabled/disabled

[Leonard_P_]

I've found a strange phenomenon with Sandy-Bridge processors and am curious if anyone can answer why this occurs.

If you enable SMT in the bios, power consumption when idle (running sleep) on the system increases by .3%-1% (depending on system).  These measurements come from running on dual-socketed Sandy-Bridge stand-alone servers and rack-based servers using a Watts-Up Pro device.  (Turbo is disabled in both cases.)  Are additional resources being powered when SMT is enabled in the bios?

When running parallel benchmarks (NAS Parallel OpenMP), but only using as many threads as physical cores (16 in this case, 2 sockets, each 8-core Sandy-Bridge), I see a power increase of 2-3% with SMT enabled (in bios) over SMT disabled (in bios).  Using Intel's performance monitoring tools (pcm.x), I see L2 and L3 misses increase with SMT enabled over SMT disabled for some benchmakrs.  Using a custom tool for reading performance counters using PAPI, I find icache misses also increase sizably for a number of the parallel benchmarks.  Although these additional misses impact power, they rarely impact performance (max negative impact is ~.5% for the NPBs). The increase in icache misses (and subsequently L2/L3 misses) leads me to suspect that enabling SMT in the bios causes some form of icache resource partitionining between threads.  (This seems reasonable given the literature on resource partitioning to improve SMT performance.)

My questions are:  1) Is resource partitiioning occurring (am I on the right track)?  2) Why would resource partitioning occur even if there is not active second thread? and 3)  What is being enabled/disabled to cause idle power consumption to increase when SMT is enabled in the bios?

Thank you in advance,

Leo

[TimP]

SMT involves resource partitioning.  Some of the relevant resource are partioned dynamically, on demand, others (such as TLB) are split evenly.  If you didn't set affinity, your threads will swap resources frequently (incurring additional cache misses), and the vacated resources will take enough time to power down that your figures make sense.  I don't know that any expert on these questions is likely to be reading your post.

[Leonard_P_]

Thanks Tim for the response.  I am pinning the threads to specific cores and have used top to check the thread pinnings.  Is it possible that elements in addition to the TLB are being statically partitioned when SMT is enabled?  

Also, any hunches on why there's more power consumption when just running "sleep"?  (sleep isn't being pinned but I don't think it needs to be?)

Lastly, you said experts on SMT might not be monitoring these forums, is there a better place to ask the question?

Thanks,

Leo

[Bernard]

>>>If you enable SMT in the bios, power consumption when idle (running sleep) on the system increases by .3%-1% (depending on system)>>>

Do you mean per socket or per core?