Error linking LAMMPS with ltbbmalloc

[luiceur]

Hi,

I am facing a very extrange error linking LAMMPS with ltbbmalloc. I am building LAMMPS with USER_INTEL package on a Cray with KNL. It all goes well, but I am stuck at this linking error. Even if I explicitly specify the location of the TBB library:

dump_xyz_mpiio.o pair_polymorphic.o fix_nve_tri.o comm_brick.o remap.o pair_tersoff_mod.o dump_atom_gz.o   -lfftw3  -lz  -L/opt/intel/tbb/lib/intel64/gcc4.1/ -ltbbmalloc -o ../lmp_intelknl
ld: cannot find -ltbbmalloc
Makefile:92: recipe for target '../lmp_intelknl' failed
make[1]: *** [../lmp_intelknl] Error 1
make[1]: Leaving directory '/home1/z01/z01/user/lammps/src/Obj_intelknl'
Makefile:132: recipe for target 'intelknl' failed
make: *** [intelknl] Error 2

It does not make much sense to me.

ls /opt/intel/tbb/lib/intel64/gcc4.1
libtbb.so            libtbb_preview_debug.so    libtbbmalloc_proxy.so
libtbb.so.2          libtbb_preview_debug.so.2  libtbbmalloc_proxy.so.2
libtbb_debug.so      libtbbmalloc.so            libtbbmalloc_proxy_debug.so
libtbb_debug.so.2    libtbbmalloc.so.2          libtbbmalloc_proxy_debug.so.2
libtbb_preview.so    libtbbmalloc_debug.so
libtbb_preview.so.2  libtbbmalloc_debug.so.2

Any explanation for this?

Luis

[Alexei_K_Intel]

Hi Luis,

Do you use Intel Compiler? How do you source the compiler and Intel TBB library? Could you provide the compiler/linker command line, please? How do you "explicitly specify the location of the TBB library"?

Regards, Alex

[luiceur]

Hi Alex,

Yes, I am using the Intel Compiler. On a Cray machine is highly recommended to load the Intel Programming Environment which is responsible for sourcing header and linking flags. So, I can see that TBB is installed 

ls /opt/intel/tbb/
bin  include  index.html  lib  python

and the lib directory is also added to the LD_LIBRARY_PATH. However the compiler is unable to find it at linking time.

The compiler command behind the Cray wrapper looks like

icpc -xmic-avx512 -static -D__CRAYXC -D__CRAY_MIC_KNL 
-D__CRAYXT_COMPUTE_LINUX_TARGET -I/opt/cray/pe/fftw/3.3.4.10/mic_knl/include 
-I/opt/cray/pe/libsci/16.09.1/INTEL/15.0/mic_knl/include 
-I/opt/cray/pe/mpt/7.4.4/gni/mpich-intel/16.0/include 
-I/opt/cray/rca/1.0.0-6.21/include 
-I/opt/cray/pe/pmi/5.0.10-1.0000.11050.0.0.ari/include 
-I/opt/cray/xpmem/0.1-4.5/include 
-I/opt/cray/gni-headers/5.0.7-3.1/include
-I/opt/cray/ugni/6.0.12-2.1/include 
-I/opt/cray/udreg/2.3.2-4.6/include 
-I/opt/cray/alps/6.1.6-20.1/include 
-I/opt/cray/wlm_detect/1.1.0-4.2/include 
-I/opt/cray/krca/1.0.0-3.55/include 
-I/opt/cray-hss-devel/8.0.0/include 
-L/opt/cray/pe/fftw/3.3.4.10/mic_knl/lib 
-L/opt/cray/pe/libsci/16.09.1/INTEL/15.0/mic_knl/lib 
-L/opt/cray/dmapp/default/lib64 
-L/opt/cray/pe/mpt/7.4.4/gni/mpich-intel/16.0/lib 
-L/opt/cray/dmapp/default/lib64 
-L/opt/cray/pe/mpt/7.4.4/gni/mpich-intel/16.0/lib 
-L/opt/cray/rca/1.0.0-6.21/lib64 
-L/opt/cray/pe/pmi/5.0.10-1.0000.11050.0.0.ari/lib64 
-L/opt/cray/xpmem/0.1-4.5/lib64 
-L/opt/cray/ugni/6.0.12-2.1/lib64 
-L/opt/cray/udreg/2.3.2-4.6/lib64 
-L/opt/cray/alps/6.1.6-20.1/lib64 
-L/opt/cray/pe/atp/2.0.3/libApp 
-L/opt/cray/wlm_detect/1.1.0-4.2/lib64 -Wl,--no-as-needed,-lAtpSigHandler,-lAtpSigHCommData -Wl,--undefined=_ATP_Data_Globals -Wl,--undefined=__atpHandlerInstall -lpthread -lfftw3f_mpi -lfftw3f_threads -lfftw3f -lfftw3_mpi -lfftw3_threads -lfftw3 -lsci_intel_mpi -lsci_intel -lm -ldl -lsci_intel -lm -ldl -lhugetlbfs -lmpichcxx_intel -lrt -lugni -lpthread -lpmi -lm -ldl -lmpich_intel -lrt -lugni -lpthread -lpmi -lm -ldl -lpmi -lpthread -lalpslli -lpthread -lwlm_detect -lalpsutil -lpthread -lrca -lugni -lpthread -lxpmem -ludreg -Wl,--as-needed,-lstdc++,--no-as-needed -Wl,--as-needed,-limf,--no-as-needed -Wl,--as-needed,-lm,--no-as-needed -Wl,--as-needed,-lifcore,--no-as-needed -Wl,--as-needed,-lifport,--no-as-needed -Wl,--as-needed,-lpthread,--no-as-needed 

And a compilation command

CC -craype-verbose -O2 -qopenmp -DLAMMPS_MEMALIGN=64 -qno-offload -fno-alias -ansi-alias -restrict -xMIC-AVX512 -O2 -fp-model fast=2 -no-prec-div -qoverride-limits -DLAMMPS_GZIP  -DLMP_USER_INTEL -DLMP_USER_OMP -DLMP_MPIIO  -DMPICH_SKIP_MPICXX -DOMPI_SKIP_MPICXX=1 -DFFT_FFTW3     -M ../pair_lj_cubic.cpp > pair_lj_cubic.d

I explicitly specify the location of the TBB libraries with (for instance) 

-L/opt/intel/tbb/lib/intel64/gcc4.1/ -ltbbmalloc -o ../lmp_intelknl

[Alexei_K_Intel]

Have you tried the simplest example? E.g.

echo "int main() { return 0; }" > /tmp/test.cpp
icpc /tmp/test.cpp -ltbbmalloc

 the lib directory is also added to the LD_LIBRARY_PATH. However the compiler is unable to find it at linking time.

As far as I know, LIBRARY_PATH is used at link-time and LD_LIBRARY_PATH is used at run-time.