how to measure energy consumption on the coprocessor?

[King_Crimson]

Question: how to determine the amount of energy consumed in Joules for my MPI-OpenMP program running on the coprocessor?

Details: I need to perform power analysis using vtune amplifier, but have just found that what the utility actually allows is "sleep states" and "frequency" profiling. How is the result related to the power dissipation? (P=C*V^2*f? How may the values of C and V be obtained?)

Thanks for your help!

[Frances_R_Intel]

Good question. You could use the output from micsmc to determine the watts used by the coprocessor at various points while your program is running and estimate from that but I'm suspect you already know that. I will ask around and see if anyone knows how to measure this for real.

[King_Crimson]

Frances Roth (Intel) wrote:

Good question. You could use the output from micsmc to determine the watts used by the coprocessor at various points while your program is running and estimate from that but I'm suspect you already know that. I will ask around and see if anyone knows how to measure this for real.

Dear Frances,

Many thanks to your reply. Using micsmc is an effective workaround. From the admin manual and the command-line help information, however, it is unclear to me how to synchronize the utility with my program (ideally, with specific functions executed in parallel). What I've done so far is something like:

running my program &

while ps | grep $!

do
    micsmc --freq >> ${RESULT_FILE}
    sleep ${SAMPLING_INTERVAL}s
done

I'm wondering if there is any canonical way you would recommend of using micsmc.

[McCalpinJohn]

An alternative to running micsmc on the host is to run a process on the coprocessor that periodically reads the contents of the special file "/sys/class/micras/power".    I launch the following script in the background immediately before launching my job, then after the job is over I send a kill signal to the script.

[bash] #!/bin/sh
# set -vx
# dumps MIC power and temperature data every 5 seconds until killed

tbase=`date +%s`
t=0
while true
do
    echo "ELAPSED TIME = $t seconds"
    cat /sys/class/micras/power
    cat /sys/class/micras/temp
    sleep 5
    tnow=`date +%s`
    t=`expr $tnow - $tbase`
done
[/bash]

[Frances_R_Intel]

I haven't come up with a good solution. You might collect both power and core utilization from micsmc at the same time. At least then you could see when your program is parallel. I have also been wondering if there isn't some way to use the ganglia monitor daemon to collect what you want but I have never used ganglia and don't know exactly what it is capable of. It would save you the communication back to the host each time you check the power (which Dr. McCalpin's solution does as well.) Perhaps someone else will come up with a better solution, but I'm afraid that is all I have for you.

[jiri]

To make it even trickier, it seems to me that if the card enters a low-power state, running micsmc wakes it up.

[King_Crimson]

John D. McCalpin wrote:

An alternative to running micsmc on the host is to run a process on the coprocessor that periodically reads the contents of the special file "/sys/class/micras/power".    I launch the following script in the background immediately before launching my job, then after the job is over I send a kill signal to the script.

 #!/bin/sh
# set -vx
# dumps MIC power and temperature data every 5 seconds until killed
tbase=`date +%s`
t=0
while true
do
    echo "ELAPSED TIME = $t seconds"
    cat /sys/class/micras/power
    cat /sys/class/micras/temp
    sleep 5
    tnow=`date +%s`
    t=`expr $tnow - $tbase`
done

/sys/class/micras/power returns:

92000000
93000000
94000000
155000000
41000000
0
53000000
22000000 0 1002000
31000000 0 1000000
33000000 0 1501000

Is there any official Intel documentation that tells how to interpret these data? I haven't found the related information on the developer's guide, which as a side note, mentions another way "MicAccessAPI".

[Alireza_Majidi]

Hi,

I think it will be helpful if you look at PAPI source code for measuring power on MIC:

http://icl.cs.utk.edu/papi/docs/d7/d04/linux-micpower_8c.html

There you can find meaning of power file

[TaylorIoTKidd]

Alireza,

Thank you for your PAPI pointer. You took the wind out of my sails.   :-)

In my next blog series, I will look at measuring power on the coprocessor, including reading and interpreting /sys/class/micras/power.

That series will start after I finish up my turbo and configuration series (in about a month or two).

Regards
---
Taylor
 

[Klaus-Dieter_O_Intel · Accepted solution]

You will find information about power measurements using the MicMgmtAPI (aka MicAccessAPI) of MPSS 3.1 in /usr/share/doc/micmgmt (see examples and libmicmgmt.7.html), for MPSS 2.1 in /opt/intel/mic/mgmt/sdk.

The readme.txt and examples.c sources from the examples sub-directory show how to do power measurements, see the call to do_power_examples(). However, I had to comment other calls to do_pci_config_examples() and do_turbo_util_examples() which did not succeed on my system.

[Qingpeng_N_]

http://software.intel.com/en-us/forums/topic/507670

Here is the measurement results with the method provided in this post with /sys/class/micras/power. The GFLOPS/Watt result does not match the result reported on intel documents for DGEMM call. 

Intel MIC 5110P : 59.00 Joules  in 459.843 milliseconds with 156.57 GFLOPS. So the GFLOPS/Watt=156.57/(59.00/(459.843/1000))=1.2202GFLOPS/Watt

Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz:  60.89 Joules in 798.345 milliseconds with 271.831 GFLOPS. So the GFLOPS/Watt = 271.831/(60.89/(798.345/1000))=3.564 GFLOPS/Watt. Intel(R) Xeon(R) CPU power is measured by PAPI RAPL PACKAGE_ENERGY:PACKAGE0 event.

http://www.intel.com/content/dam/www/public/us/en/documents/performance-... The performance per watt reported on above document on DGEMM call is for MIC is above 4 while for CPU is less than 1.5 GFLOPS/Watt. 

[Mark_L_1]

I found useful "documentation" on the /sys/class/micras file structure here:

  https://github.com/pathscale/intel_xeon_phi_kernel_driver/blob/master/ras/micras_main.c

I'm still looking for a C API for scraping this information from the MIC side. The MicMgmt library is only for host side.

[King_Crimson]

old post got dug out again ...

Mark,

That source code is a very interesting finding. As to implementation in C, I just used popen() to create a pipe, invoked the ``cat pseudo-file'' command, and extracted the power information from the resulting stream. I guess that's what you're looking for.