cannot load library to the device 0 (error code 5)

[Vincent_L_]

hi， I recently met a error while using Xeon Phi. When the program run into the #pragma offload area, it occur error “offload error: cannot load library to the device 0 (error code 5)” and then exit. I wonder what the "error code 5" mean? Can anyone provide me the error code list of the offload error? I can not find such thing through the Internet. Thanks

[Kevin_D_Intel]

We need more details to be of assistance. Knowing the error will likely not be enough. The error code 5 (COI_DOES_NOT_EXIST) is an indication that the specified object was not found which might mean functions used within offload sections are not declared properly in the program.

Depending on the installed version of MPSS, the error codes derive from /usr/include/intel-coi/common/COIResult_common.h (MPSS 3.x) or /opt/intel/mic/coi/include/common/COIResult_common.h (MPSS 2.x).

Can you provide more details about the offload area added to the program?   Or provide the program itself, a code snippet?
Also, which version of MPSS and the compiler (icc/icpc/ifort/icl) are you using?

[Vincent_L_]

Kevin Davis (Intel) wrote:

We need more details to be of assistance. Knowing the error will likely not be enough. The error code 5 (COI_DOES_NOT_EXIST) is an indication that the specified object was not found which might mean functions used within offload sections are not declared properly in the program.

Depending on the installed version of MPSS, the error codes derive from /usr/include/intel-coi/common/COIResult_common.h (MPSS 3.x) or /opt/intel/mic/coi/include/common/COIResult_common.h (MPSS 2.x).

Can you provide more details about the offload area added to the program?   Or provide the program itself, a code snippet?
Also, which version of MPSS and the compiler (icc/icpc/ifort/icl) are you using?

My code is like below：

//in main.c 
foowrapper();


//in foo.c
#pragma offload_attribute (push,target(mic))
void foo()
	{
#ifdef __MIC__
			__m512i x....;
			__m512d y.....;
			__mmask8 mask;
	kmp_set_defaults("KMP_AFFINITY=scatter,granularity=thread");
#pragma omp parallel for
			for(i=0;i<m;i++)
			{
				
					int rem=end-idx;
					mask=_mm512_int2mask(...);
					x=_mm512_mask_load_epi64(...);
					y=_mm512_mask_load_pd(...);
					x=_mm512_mask_i32logather_pd(...);
					y=_mm512_mask_fmadd_pd(...);
				}
				y+=_mm512_reduce_add_pd(...);
			}
			
#endif
	}
#pragma offload_attribute (pop)

void foowrapper()
{
#pragma offload target(mic:0)
     foo();
}

the program can not enter the offload area and throw error mentioned in the title.

[Kevin_D_Intel]

Thanks for the code outline. Unfortunately, try as I did to induce the failure, I can’t, thus I’m either not replicating your environment settings, your source and/or how you are compiling and linking everything.

While it’s not typical to wrap the routine using the push/pop in the way you showed, and it doesn’t seem to be a factor, you could replace those with just the __attribute__ ((target(mic))).  [ e.g.  __attribute__((target(mic))) void foo() ]

I’m guessing you probably are using a 14.0 (or maybe earlier) compiler so something else to try and help identify a possible missing routine is to add to the compilation/link for the offloaded source file the compiler option: -offload-option,mic,ld,”-z defs”

Another run-time method for debugging this is to set the environment variable OFFLOAD_REPORT=3 and see what information is reported about what is loaded or not.

Also, can you check and post the setting of the environment variable: MIC_LD_LIBRARY_PATH

If you can offer any insight into how you are compiling/linking that might help but even better would be if you could provide a complete reproducer. That would certainly be most helpful.

[Vincent_L_]

Thanks for the reply, my icc version is 15.0.0.090 and I compile the program with "icc -openmp  *.c" in a CMAKE framework.I think my environment settings have no problem because I have tested and passed some other simple offload programs. The problem in my project seem to be very weird.  Could I send you the whole source code If you don't mind ? 

[Kevin_D_Intel]

Yes, you can send me the complete source. You can attached that to a reply here (publicly for all to access) or for secure access (Intel only) send me a private reply (via Send Author A Message link) and attach the file to the private reply.

[Vincent_L_]

Thanks, I lucky solved the problem after I compile the program with static link:) 

But I am now facing a new problem: In the offload area like below

int *a=(int *)_mm_malloc(100*sizeof(double),64);

#pragma offload target(mic:0) in(a:length(100))
{
...
_mm512_mask_load_epi64(.. , writemask , &(a[3]));
}

it produces error" process on the device 0 was terminated by signal 11 (SIGSEGV)"

I looked up in the Intel doc and found  that the 3th argument of _mm512_mask_load_epi64 function should be a  64-byte-aligned address. But I have already make pointer a aligned in the _mm_malloc function. 

[James_C_Intel2]

But I have already make pointer a aligned in the _mm_malloc function.

If a is 64 bytes aligned, then &a[3] (where a is of type double, so each element occupies eight bytes) is guaranteed to be at a 24 byte offset from 64 byte alignment...

You need an offload expert (which I am not) to tell you whether the alignment that you specified on the host is guaranteed to be implemented on the target, though.