mkl_intel_lp64 vs mkl_gf_lp64 and MKL advisor

[lejeczek]

I wonder if the Advisor get it wrong when suggests:

 -Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -ldl -lpthread -lm

this bit: -lmkl_intel_lp64

when GNU's bits are chosen when applicable instead of Intel's

reason I'm asking is because I get undefined references to GOMP.

many thanks.

[Ying_H_Intel]

Hello, 

The layered structure allow developers to select suitable compiler, threading combination. If you choose GNU  for OpenMP library, which mean you'd like to use libgomp as threading runtime library. 

Could you  try to add the compiler option and see it remove the undefined reference?

-fopenmp -m64 -I${MKLROOT}/include

The advisor is on-line too  https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/. It give  Link option

-Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -lmkl_gf_lp64 -lmkl_core -lmkl_gnu_thread -ldl -lpthread -lm

and compile option: 

 -fopenmp -m64 -I${MKLROOT}/include

If you are using gfortran compiler, but like Intel OpenMP library as threading library,   the advisor will give you 

 -Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -lmkl_gf_lp64 -lmkl_core -lmkl_intel_thread -liomp5 -ldl -lpthread -lm

and compiler option:  -m64 -I${MKLROOT}/include. 

Best Regards,

Ying

[lejeczek]

that is what I was saying, Advisor is gives this:

-Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -ldl -lpthread -lm

for GNU C + MP, instead of mkl_gf_lp64

And I was asking if Advisor might be bugged, giving wrong advises?

[Ying_H_Intel]

I see.  

First in interface layer: regarding the mkl_intel_lp64 and mkl_gf_lp64 ,  because there is no variation for C language in implementations, but For Fortran interface the variation is sometimes big making them fully incompatible between each other.  

For GNU C compiler, we still go via Intel Compiler, to use mkl_intel_lp64. 

For GNU fortran compiler, we use mkl_gf_lp64,  which is only for GNU fortran compiler and Absoft compiler. 

Second in  threading layer: both GNU OMP threading  and Intel OMP threading layer can work with the GNU  compiler interface.  

So if with GNU C compiler +  GNU OMP ,  right,  it is expected to use 

-Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -ldl -lpthread -lm

and -fopenmp -m64 -I${MKLROOT}/include, which help to resolve the undefined references to GOMP.

Best Regards,

Ying 

[lejeczek]

nope it does not, that was why I asked the question.

I had (for static)

 -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_gnu_thread.a -Wl,--end-group -ldl -lpthread -lm

 -fopenmp -m64 -I${MKLROOT}/include

and I kept getting:

../../lib/libR.so: undefined reference to `omp_get_thread_num'
../../lib/libR.so: undefined reference to `omp_get_num_threads'
../../lib/libR.so: undefined reference to `GOMP_parallel'
collect2: error: ld returned 1 exit status

[lejeczek]

same for dynamic linking

[TimP]

If that's all the link advisor said, that is an omission, as you must either set -fopenmp (which implies -lpthread) or explicitly -lgomp so as to link libgomp (or explicitly link libiomp5).  Only .so is available for the OpenMP libraries, which will work for either static or dynamic MKL libraries, so you must have the run-time path (LD_LIBRARY_PATH or equivalent) set.

Maybe your link command order is confused.  Something like

gcc -fopenmp   *.o -Wl,--start-group ......

The start-group...end-group repeats searches only within the group, it doesn't take care of random order specifications outside the group.

If you link libiomp5 rather than libgomp, the non-gnu intel_thread library would be recommended.  I don't know why you have that library inside the group directives; it should work after them.

[Ying_H_Intel]

Hi lejeczek,

What is your OS and the whole build command looks like?

I'm afraid that the problem is not about MKL advisor, gf or intel, gnu gomp vs intel openmp,  but may be in  the build and linker itself. 

As you know,  the error 

../../lib/libR.so: undefined reference to `omp_get_thread_num'
../../lib/libR.so: undefined reference to `omp_get_num_threads'
../../lib/libR.so: undefined reference to `GOMP_parallel'

all of the symbol should be  defined in OpenMP libraries. 

for exmaple, 

[root@snb04 intel64]# nm libiomp5.so | grep 'omp_get_thread_num'
000000000005fba0 t L__routine_start___kmp_api_omp_get_thread_num_22
0000000000060780 t L__routine_start___kmp_api_omp_get_thread_num__22
000000000005fba0 t __kmp_api_omp_get_thread_num
0000000000060780 t __kmp_api_omp_get_thread_num_
000000000005fba0 t __kmp_api_omp_get_thread_num_10_alias
0000000000060780 t __kmp_api_omp_get_thread_num__10_alias
000000000005fba0 T omp_get_thread_num@@VERSION
000000000005fba0 T omp_get_thread_num@OMP_1.0
0000000000060780 T omp_get_thread_num_@@VERSION
0000000000060780 T omp_get_thread_num_@OMP_1.0

So we can solve them by link the library.  But it may be issues when link them together

We saw the problems like below 

1) the order of obj and required library . for example, A depend on B,  then the link order should be A then B.   

2) to add  –no-as-needed explicitly in link line to force the linker to link all specified libraries  in some system like Ubuntu. 

for example https://software.intel.com/en-us/articles/symbol-lookup-error-when-linking-intel-mkl-with-gcc-on-ubuntu

gcc  -fopenmp -m64 -I$MKLROOT/include lapacke_dgelsd_row.c  -L$MKLROOT/lib/intel64 -Wl,--no-as-needed -lmkl_intel_lp64 -lmkl_core -lmkl_gnu_thread -lpthread -lm. 

In your case, it may be 

gcc  -shared -m64 -I$MKLROOT/include a.c  -Wl,--no-as-needed -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_gnu_thread.a -Wl,--end-group   -Wl,--no-as-needed  -lgomp -ldl -lpthread -lm

Let us know if any result. 

Best Regards,

Ying