ittnotify in xeon phi offload regions

[james_B_8]

I'm trying to call the functions __itt_pause() and __itt_resume() from within offload regions so that I can only sample specific sections of my code in a knc-bandwidth collection. However I'm having problems with linking that I can't figure out. I am using Intel(R) VTune(TM) Amplifier XE 2015 Update 1.

My code looks like:

#pragma offload_attribute(push,target(mic))
#include <ittnotify.h>
#pragma offload_attribute(pop)

...

#pragma offload target(mic:offload_target)
{
    __itt_resume();
}

My make options are:

INCLUDE=-I/opt/intel/vtune_amplifier_xe_2015/include/
LIB=-offload-options,mic,ld," -Wl,-Bstatic -L/opt/intel/vtune_amplifier_xe_2015/bin64/k1om/ -littnotify -Wl,-Bdynamic -lpthread

I have also tried:

LIB=-offload-options,mic,ld,"-L/opt/intel/vtune_amplifier_xe_2015/bin64/k1om/libittnotify.a -lpthread

I get the following error at link time:

icc: remark #10397: optimization reports are generated in *.optrpt files in the output location
/tmp/iccMICvCgQqq:(.data+0x338): undefined reference to `__itt_resume_ptr__3_0'
/tmp/iccMICvCgQqq: In function `main':
/nfs/home/users/dc-brig1/tmp/modal_v9/Code/gamma_mpi.c:575: undefined reference to `__itt_resume_ptr__3_0'
make: *** [rmpi] Error 1

How can this be done?

Cheers,

James

[David_A_Intel1]

I suspect this is not supported.  It would require the compiler to generate the calls to the library in the offload regions and I don't believe the compiler is that tightly integrated with the VTune Amplifier.  (can you call any libraries in offload regions?)

I will double-check and get back to this thread, but I seriously doubt it is supported.

[Peter_W_Intel]

@ James B

1. Usually you need to use Composer XE 2015 to build offload Xeon Phi program, and note that MIC library should be used when linking.

2. VTune Amplifier XE 2015 changed analysis type's name from knc-hotspots to advance-hotspots, for example.

Please reference this article, and try example. Hope it helps.

[james_B_8]

@Mr Anderson You can certainly call external libraries from offload regions e.g MKL, You just have to link to the libaries and they have to be compiled with -mmic.

@Peter Thanks for that article. It links without error now.

FYI the variables in my Makefile now look like this:

CFLAGS=-g -offload-option,mic,compiler,"-I/opt/intel/vtune_amplifier_xe_2015/include/ /opt/intel/vtune_amplifier_xe_2015/bin64/k1om/libittnotify.a"
LIBS=-offload-option,mic,ld,"-lpthread"

I needed to modify the code like this also:

#pragma offload_attribute(push,target(mic))
#if defined(__MIC__) || (_MIC_)
#include <ittnotify.h>
#endif
#pragma offload_attribute(pop)

...

#pragma offload target(mic:offload_target)
{
    #if defined(__MIC__) || (_MIC_)
    __itt_resume();
    #endif
}

However it fails when it hits the __itt_resume() function with this error:

offload error: unexpected number of variable descriptors
offload error: process on the device 0 unexpectedly exited with code 1

Does the same if it's a _itt_pause() too, (My actual offload line in my code does do data moving BTW.)

Any idea what this means?

[Peter_W_Intel]

It sounds great that you have no compiling issue, for now.

In my case, I put "#pragma offload target (mic)" before the function which should run on MIC, and this function will call __itt_resume() & __itt_pause(). In my opinion, you might remove "#pragma offload target(mic:offload_target)" before calling _itt_resume(). I got a little bit confusion since your function already runs on MIC, why did you use pragma again?

BTW, you need to check if MPSS is running on MIC & vtune driver on MIC is loaded, see "step 0" of the article.