topic Hi David, in Intel® oneAPI Math Kernel Library

Error compiling MKL for Xeon Phi (MIC) with Compiler assisted Offload mode in Visual Studio

David_F_8 — Tue, 26 Apr 2016 00:15:40 GMT

Hello all,

I am new to Xeon Phi and MKL and I am trying to compile the Compiler assisted Offload for the sgemm example in C++ in Windows using Visual Studio (VS) Prof. R12.

I have set the properties in VS project to include MKL libraries, parallel for ilp64 (32bit ints) and set "mandatory" in the offload mode.

The tests I am running are on a 7200 KNC MIC. I have tried other simple applications all successful including reductions with compiler assisted offloading, implicit memory model and native execution. The driver is working fine and I can SSH to the MIC and mount a shared NFS directory. But calling an MKL function from VS using the compiled assisted offloading has not worked.

The error I get follows:

"error : function "sgemm" called in offload region must have been declared with compatible "target" attribute sgemm(&transA, &transB, &n, &n, &n, &alpha, A, &n, B, &n, &beta, C, &n);"

It would seem that the cross compilation is simply not happening because the compiler is not informed of it and there is no forward declaration instructing the compiler to do it. I would assume that this gets done in the "mkl.h" header file or some of the other ones called therein.

I have also added the MKL include and lib paths in the VS paths entry in my solution's properties to no avail.

David

Hi David,

Ying_H_Intel — Tue, 26 Apr 2016 06:03:00 GMT

Hi David,

Are you compiling the sgemm.c Compiler assisted Offload example in MKL install forder, examples_mic.zip, right?

If yes, could you please try with the Command line way to compile it and see if it work first?

for example, go to startup menu

C:\Users\yhu5\Desktop\MKL MIC\examples_mic\mic_offload\blasc>nmake intel64

icl /Qopenmp /MT /Qstd=c99 /Fo_results\_lp64_parallel_intel64_lib\ /Fe
_results\_lp64_parallel_intel64_lib\ /Qoffload-attribute-target=mic /Qoffload-o
ption,mic,compiler," -I""C:\Program Files (x86)\Intel\Composer XE 2013 SP1\mkl""
\include" /Qoffload-option,mic,link," --start-group ""C:\Program Files (x86)\Int
el\Composer XE 2013 SP1\mkl""\lib\mic\libmkl_intel_lp64.a ""C:\Program Files (x8
6)\Intel\Composer XE 2013 SP1\mkl""\lib\mic\libmkl_core.a ""C:\Program Files (x8
6)\Intel\Composer XE 2013 SP1\mkl""\lib\mic\libmkl_intel_thread.a --end-group"
source\sgemm.c mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib

then let's try MSVC 2012, could you attach the Code and the MSVC proporty setting screenshot to us?

Best Regards,

Ying

#pragma offload target(mic) \
   in(transa, transb, N, alpha, beta) \
   in(A:length(matrix_elements)) \
   in(B:length(matrix_elements)) \
   in(C:length(matrix_elements)) \
   out(C:length(matrix_elements) alloc_if(0))
   {
       sgemm(&transa, &transb, &N, &N, &N, &alpha, A, &N, B, &N,
               &beta, C, &N);
   }

Thank you for the prompt

David_F_8 — Tue, 26 Apr 2016 18:45:04 GMT

Thank you for the prompt response Ying!

I tried your command with a few modifications to adapt it to my machine and now I am able to compile in the "Intel Compiler 16.0 Intel 64 Visual Studio 2012" command line.

Just for reference my Intel MKL library is installed at:

C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl

and I am setting environmental variables with: mklvars.bat intel64 ilp64

The precise versions of the Intel software I have installed follows:

And the command line I used was:

icl /Qopenmp /MT /Qstd=c99 /Qoffload-attribute-target=mic /Qoffload-option,mic,compiler," -I""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\include" /Qoffload-option,mic,link," --start-group ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_intel_lp64.a ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_core.a ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_intel_thread.a --end-group" source\sgemm.c mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib

No I am trying with a slightly modified version of the sample code

502600

/MT /Qoffload-attribute-target=mic /Qoffload-option,mic,compiler, /Qoffload-option,mic,link," --start-group ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_intel_lp64.a ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_core.a ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_intel_thread.a --end-group"

502600

"mkl_intel_lp64.lib" "mkl_intel_thread.lib" "mkl_core.lib"

This leads to several errors:

1>------ Build started: Project: Test01 (Intel C++ 16.0), Configuration: Release x64 ------
1>icl: : warning #10121: overriding '/MD' with '/MT'
1> test_MKL.c
1>src/test_MKL.c(71): warning #266: *MIC* function "min" declared implicitly
1>     for (i=0; i<min(N,6); i++) {
1>     ^
1>
1>src/test_MKL.c(71): warning #2570: *MIC* function has not been declared with compatible "target" attribute
1>     for (i=0; i<min(N,6); i++) {
1>     ^
1>
1>src/test_MKL.c(73): warning #2570: *MIC* function has not been declared with compatible "target" attribute
1>         for (j=0; j<min(N,6); j++) {
1>         ^
1>
1>src/test_MKL.c(80): warning #266: *MIC* function "min" declared implicitly
1>     for (i=0; i<min(N,6); i++) {
1>     ^
1>
1>src/test_MKL.c(80): warning #2570: *MIC* function has not been declared with compatible "target" attribute
1>     for (i=0; i<min(N,6); i++) {
1>     ^
1>
1>src/test_MKL.c(82): warning #2570: *MIC* function has not been declared with compatible "target" attribute
1>         for (j=0; j<min(N,6); j++) {
1>         ^
1>
1>src/test_MKL.c(89): warning #266: *MIC* function "min" declared implicitly
1>     for (i=0; i<min(N,6); i++) {
1>     ^
1>
1>src/test_MKL.c(89): warning #2570: *MIC* function has not been declared with compatible "target" attribute
1>     for (i=0; i<min(N,6); i++) {
1>     ^
1>
1>src/test_MKL.c(91): warning #2570: *MIC* function has not been declared with compatible "target" attribute
1>         for (j=0; j<min(N,6); j++) {
1>         ^
1>
1>src/test_MKL.c(149): warning #167: *MIC* argument of type "const long *" is incompatible with parameter of type "const long long *"
1>         sgemm(&transA, &transB, &n, &n, &n, &alpha, A, &n, B, &n, &beta, C, &n);
1>         ^
1>
1>src/test_MKL.c(149): warning #167: *MIC* argument of type "const long *" is incompatible with parameter of type "const long long *"
1>         sgemm(&transA, &transB, &n, &n, &n, &alpha, A, &n, B, &n, &beta, C, &n);
1>         ^
1>
1>src/test_MKL.c(149): warning #167: *MIC* argument of type "const long *" is incompatible with parameter of type "const long long *"
1>         sgemm(&transA, &transB, &n, &n, &n, &alpha, A, &n, B, &n, &beta, C, &n);
1>         ^
1>
1>src/test_MKL.c(149): warning #167: *MIC* argument of type "const long *" is incompatible with parameter of type "const long long *"
1>         sgemm(&transA, &transB, &n, &n, &n, &alpha, A, &n, B, &n, &beta, C, &n);
1>         ^
1>
1>src/test_MKL.c(149): warning #167: *MIC* argument of type "const long *" is incompatible with parameter of type "const long long *"
1>         sgemm(&transA, &transB, &n, &n, &n, &alpha, A, &n, B, &n, &beta, C, &n);
1>         ^
1>
1>src/test_MKL.c(149): warning #167: *MIC* argument of type "const long *" is incompatible with parameter of type "const long long *"
1>         sgemm(&transA, &transB, &n, &n, &n, &alpha, A, &n, B, &n, &beta, C, &n);
1>         ^
1>
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `free_arrays':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:60: undefined reference to `MKL_free'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:61: undefined reference to `MKL_free'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:62: undefined reference to `MKL_free'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `print_arrays_snap_shot':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:71: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:73: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:73: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:71: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:80: undefined reference to `min'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o:C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:82: more undefined references to `min' follow
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `sgemm_serial':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:103: undefined reference to `cblas_sgemm'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `__offload_entry_test_MKL_c_142sgemm_offload_MKL1492001044302':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:149: undefined reference to `sgemm'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `init_arrays':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:24: undefined reference to `MKL_malloc'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:25: undefined reference to `MKL_malloc'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:26: undefined reference to `MKL_malloc'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `sgemm_offload_MKL':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:123: undefined reference to `start_timer_T64'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `sgemm_serial':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:103: undefined reference to `cblas_sgemm'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `sgemm_offload_MKL':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:125: undefined reference to `elapsed_time_T64'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `print_arrays_snap_shot':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:71: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:73: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:73: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:71: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:80: undefined reference to `min'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o:C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:82: more undefined references to `min' follow
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `sgemm_offload_MKL':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:131: undefined reference to `start_timer_T64'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:149: undefined reference to `sgemm'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:158: undefined reference to `elapsed_time_T64'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `print_arrays_snap_shot':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:71: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:73: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:73: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:71: undefined reference to `min'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:80: undefined reference to `min'
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o:C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:82: more undefined references to `min' follow
1> C:\Users\dfernand\AppData\Local\Temp\2\714077MIC37.o: In function `init_arrays':
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:31: undefined reference to `MKL_free'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:32: undefined reference to `MKL_free'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:33: undefined reference to `MKL_free'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:24: undefined reference to `MKL_malloc'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:25: undefined reference to `MKL_malloc'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:26: undefined reference to `MKL_malloc'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:31: undefined reference to `MKL_free'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:32: undefined reference to `MKL_free'
1> C:\Users\dfernand\Documents\IntelXeonPhi\mic_samples\Test01/src/test_MKL.c:33: undefined reference to `MKL_free'
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Platforms\x64\PlatformToolsets\Intel C++ Compiler 16.0\Microsoft.Cpp.x64.Intel C++ Compiler 16.0.targets(1081,5): error MSB6006: "xilink.exe" exited with code 1.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

I hope you have some clues to what is going on.

Again thank you!

David

Hi David,

Ying_H_Intel — Fri, 29 Apr 2016 01:56:48 GMT

Hi David,

I saw your question. Please give some time, we will check and get back to you if any result.

Thanks

Ying

Hi David,

Ying_H_Intel — Mon, 09 May 2016 07:21:14 GMT

Hi David,

Regarding the compilation, How about comment

#define min(x,y) (((x) < (y)) ? (x) : (y))

Then the command line will work, right?

icl /Qopenmp /MT /Qstd=c99 /Qoffload-attribute-target=mic /Qoffload-option,mic,compiler," -I""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\include" /Qoffload-option,mic,link," --start-group ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_intel_ilp64.a ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_core.a ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_intel_thread.a --end-group" test_MKL.c mkl_intel_ilp64.lib mkl_intel_thread.lib mkl_core.lib

As i haven't found the Windows Host MIC environment, So just did a try in MSVS 2012 environment . If We just put the command into

Project Property => Linker => Additional Options:

xilink: : warning #10006: ignoring unknown option '/Qoffload-option,mic,link, --start-group "C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl"\lib\mic\libmkl_intel_ilp64.a "C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl"\lib\mic\libmkl_core.a "C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl"\lib\mic\libmkl_intel_thread.a --end-group'

I can see similiar error as yours.

Please try to put the link option to

Project Property => Linker=> General [Intel C++] =>Additional Options for MIC linker

"""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_intel_ilp64.a ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_core.a ""C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl""\lib\mic\libmkl_intel_thread.a"

and add Project Property => Linker=>Input=> Additional Dipendencies

mkl_intel_ilp64.lib;mkl_intel_thread.lib;mkl_core.lib

Project Property => Linker=>General=> Additional library Dipendencies

C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\lib\intel64_win

Best Regards,

Ying

As i haven't correct environment, I get the below error, please see if you can make it run.

icc: : warning #10006: ignoring unknown option '-fstart-group'
1>icc: : warning #10006: ignoring unknown option '-fend-group'
1>icc: : warning #10362: Environment configuration problem encountered. Please check for proper MPSS installation and environment setup.
1> k1om-mpss-linux-ld.exe: C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\lib\mic\libmkl_core.a(mkl_memory_patched.o): undefined reference to symbol 'pthread_key_delete@@GLIBC_2.2.5'
1> k1om-mpss-linux-ld.exe: note: 'pthread_key_delete@@GLIBC_2.2.5' is defined in DSO C:\Program Files\Intel\MPSS\/k1om-mpss-linux/lib64/libpthread.so.0 so try adding it to the linker command line
1> C:\Program Files\Intel\MPSS\/k1om-mpss-linux/lib64/libpthread.so.0: could not read symbols: Invalid operation
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Platforms\x64\PlatformToolsets\Intel C++ Compiler 16.0\Microsoft.Cpp.x64.Intel C++ Compiler 16.0.targets(1087,5): error MSB6006: "xilink.exe" exited with code 1.

Intel C++ has an excellent

TimP — Mon, 09 May 2016 10:17:01 GMT

Intel C++ has an excellent implementation of std::min, so you should avoid macro redefinition, as Ying advised. It's annoying to write code to interoperate with other compilers; the macro redefinition still seems to be a usual tactic with MSVC or Intel C, and fmin seems best with aggressive gcc/g++ options. It might be helpful if Intel could also optimize the code targeted for gcc/g++. So I am using an annoying group of macros which invokes std::min for Intel C++. Recent Intel releases are reasonably capable of optimizing the macros like the one quoted.

Those ld directives are spelled --start-group ..... --end-group. As MKL needs them for static linking, please refer to

https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

I removed the macro

David_F_8 — Mon, 09 May 2016 21:51:28 GMT

I removed the macro definition for "min" and I am using the one from the compiler libraries.

On the other hand, I have spent now some time testing the compilation process with MSVS 2012 following the exact instructions given by Ying and I am getting this compilation error:

1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Platforms\x64\PlatformToolsets\Intel C++ Compiler 16.0\Microsoft.Cpp.x64.Intel C++ Compiler 16.0.targets(1081,5): error MSB6006: "xilink.exe" exited with code 1.

The command line works fine, this is what I was using before, but no luck still with MSVS.

Bests,

David

Hi David,

Ying_H_Intel — Wed, 11 May 2016 08:34:53 GMT

Hi David,

Yes, it seems the integration of MSVC and MIC support has some problems. I had escalated the problem to Intel Compiler team

I upload one MSVC project. You may try if it can run at your sides.

Best Regards,
Ying

attach the project. It have

Ying_H_Intel — Wed, 11 May 2016 08:37:55 GMT

attach the project. It have some option in Project Property.

Thanks

Ying

I tried the project and I get

David_F_8 — Tue, 24 May 2016 17:20:44 GMT

I tried the project and I get the following error:

1>------ Build started: Project: Project1 (Intel C++ 16.0), Configuration: Debug x64 ------
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Platforms\x64\PlatformToolsets\Intel C++ Compiler 16.0\Microsoft.Cpp.x64.Intel C++ Compiler 16.0.targets(368,5): error : Element <OffloadOpenMP> has an invalid value of "MICTarget".
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

Best regards,

David