- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
I installed Fupermod software framwork with MKL it installed correctally but when I run my application i got this error:
tania@tania-Latitude-E5540:~/openform_fupermod_test$ mpirun -np 6 -machinefile machinefile ~/build_fupermod/mkl/tools/builder -l ~/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so -U2000 -s100
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
-------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[32927,1],2]
Exit code: 2
I posted about this error in this thread. But still didn't get any solution.
https://software.intel.com/en-us/comment/1853675#comment-1853675
Here is my platform description.
tania@tania-Latitude-E5540:~/openform_fupermod_test$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 69
Stepping: 1
CPU MHz: 1399.992
BogoMIPS: 5387.86
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 4096K
NUMA node0 CPU(s): 0-3
I run MKL example cblas it works properly. Here is the output
tania@tania-Latitude-E5540:/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/examples/cblas$ sudo make libintel64 function=cblas_dgemm compiler=gnu
make cblas_dgemmx.res _IA=intel64 EXT=a RES_EXT=lib
make[1]: Entering directory `/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/examples/cblas'
mkdir -p ./_results/gnu_lp64_parallel_intel64_lib
gcc -m64 -w -I../../include source/cblas_dgemmx.c source/common_func.c -Wl,--start-group "../../lib/intel64"/libmkl_intel_lp64.a "../../lib/intel64"/libmkl_intel_thread.a "../../lib/intel64"/libmkl_core.a -Wl,--end-group -L"../../../compiler/lib/intel64" -liomp5 -lpthread -ldl -lm -o _results/gnu_lp64_parallel_intel64_lib/cblas_dgemmx.out
Here you can see with simple c program, I also checked linking it has no issue.
tania@tania-Latitude-E5540:/tmp$ gcc -o test test.c -L/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin -lmkl_avx2 -lmkl_def
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$ ./test
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$ mpicc -o test test.c -L/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_sequential -lmkl_core
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$ ./test
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$ mpicc -fopenmp -o test test.c -L/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_sequential -lmkl_core -lgomp
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$ ./test
tania@tania-Latitude-E5540:/tmp$ mpicc -O0 -fopenmp -o test test.c -L/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_sequential -lmkl_core -lgsl -lgslcblas -ldl -lm -lrt -lgomp -pthread
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$ ./test
tania@tania-Latitude-E5540:/tmp$
tania@tania-Latitude-E5540:/tmp$ mpicc -O0 -fopenmp -o test test.c -L/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_sequential -lmkl_core -lgsl -lgslcblas -ldl -lm -lrt -lgomp -pthread
Now I am sure there is some problem with MKL with my platform. As it ask for the ibmkl_avx2.so or libmkl_def.so. Although the path includes these files. But I don't know why I am getting this error. Plz help me. And plz let me know, If it is a bug in MKL how can I report it?
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Tania,
1) Does your app run fine if executed without mpirun?
2) Could you please print the contents of LD_LIBRARY_PATH before running the program?
3) Which mpi do you use? If Intel's, could you please add -genvall to mpirun's options?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
(1) I got the same error without Mpirun
tania@tania-Latitude-E5540:~/openform_fupermod_test$ ~/build_fupermod/mkl/tools/builder -l ~/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so -U2000 -s100
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
(2) Here is the output of echo $LD_LIBRARY_PATH
tania@tania-Latitude-E5540:~/openform_fupermod_test$ echo $LD_LIBRARY_PATH
/home/tania/.openmpi/lib/:/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin:/opt/ThirdParty-3.0.1/platforms/linux64Gcc/gperftools-svn/lib:/opt/paraviewopenfoam44/lib/paraview-4.4:/opt/openfoam30/platforms/linux64GccDPInt32Opt/lib/openmpi-system:/opt/ThirdParty-3.0.1/platforms/linux64GccDPInt32Opt/lib/openmpi-system:/home/tania/.openmpi/lib:/home/tania/OpenFOAM/tania-3.0.1/platforms/linux64GccDPInt32Opt/lib:/opt/site/3.0.1/platforms/linux64GccDPInt32Opt/lib:/opt/openfoam30/platforms/linux64GccDPInt32Opt/lib:/opt/ThirdParty-3.0.1/platforms/linux64GccDPInt32Opt/lib:/opt/openfoam30/platforms/linux64GccDPInt32Opt/lib/dummy:/home/tania/.openmpi/lib/
tania@tania-Latitude-E5540:~/openform_fupermod_test$ mpirun -np 6 -machinefile machinefile ~/build_fupermod/mkl/tools/builder -l ~/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so -U2000 -s100
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Kernel:/home/tania/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
-------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[56104,1],1]
Exit code: 2
(3) I am using openmpi-1.10.1
Also when I install Fupermod with GSL . My application run perfactlly. Plus on grid5000 platform it works with MKL too, but not working on my laptop.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here is the ldd output
tania@tania-Latitude-E5540:~/openform_fupermod_test$ ldd ~/build_fupermod/mkl/routines/mxm/.libs/libmxm_1d.so
linux-vdso.so.1 => (0x00007ffcf9139000)
libmpi_cxx.so.1 => /home/tania/.openmpi/lib/libmpi_cxx.so.1 (0x00007f2522ba4000)
libmpi.so.12 => /home/tania/.openmpi/lib/libmpi.so.12 (0x00007f25228cf000)
libmkl_intel_lp64.so => /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin/libmkl_intel_lp64.so (0x00007f2521f91000)
libmkl_gnu_thread.so => /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin/libmkl_gnu_thread.so (0x00007f252120c000)
libmkl_core.so => /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64_lin/libmkl_core.so (0x00007f251f927000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f251f621000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f251f412000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f251f1fc000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f251efde000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f251ec19000)
libopen-pal.so.13 => /home/tania/.openmpi/lib/libopen-pal.so.13 (0x00007f251e93b000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f251e637000)
libibverbs.so.1 => /usr/lib/libibverbs.so.1 (0x00007f251e429000)
libopen-rte.so.12 => /home/tania/.openmpi/lib/libopen-rte.so.12 (0x00007f251e1ad000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f251dfa9000)
/lib64/ld-linux-x86-64.so.2 (0x00007f2522fd7000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f251dda1000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f251db9e000)
Seems like libmkl_intel_lp64.so or ibmkl_gnu_thread.so or libmkl_core.so internally invokes libmkl_avx2.so or libmkl_def.so. Because in my application I don't need them directaly.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Tania,
The libmxm_1d.so build looks fine. You are exactly right, libmkl_intel_lp64.so or ibmkl_gnu_thread.so or libmkl_core.so internally invokes libmkl_avx2.so or libmkl_def.so( they are processor-optimized library). So the problem is the calling environment can't recognize the following Link Path. Could you rebuild the libmxm_1d.so with static library? for example, you can link the libmkl_intel_lp64.a libmkl_gnu_thread.a libmkl_core.a in your link line.
Then when ldd libmxm_1d.so, no mkl library needed.
Best Regards,
Ying
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ying
I have to build the libmxm_1d.so with dynamic library. Because I have to use MKL laibrary threading with this builder application. Is their any other option to make it fix please?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Tania,
YOu can build the libmxm_1d.so with threading MKL library, whatever link with dynamic library and static library. For example, libmkl_intel_lp64.a libmkl_gnu_thread.a libmkl_core.a + dynamic gomp library are static library and GNU openmp threading.
If have to use threading MKL library, you may need to add -rpath option and -Wl -as-no-needed option in your build link. as I mentioned last time.
Best Regards,
Ying

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page