- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm trying to use Intel MKL 10.0.4.023 with some third party code (NumPy) on 64-bit Linux.
I am able to successfully compile and link the code, but when I run it and call a function that uses MKL, I get the following error message:
MKL FATAL ERROR: /opt/intel/mkl/10.0.4.023/lib/em64t/libmkl_def.so: undefined symbol: i_free
The library that is called was linked against MKL with:
-shared -lmkl -lguide -lpthread -Wl,-rpath=/opt/intel/mkl/10.0.4.023/lib/em64t
I also tried using the "pure layered model" libraries as described in the User's Guide, but I get the same error.
The User's Guide also says:
Redefining is possible because Intel MKL actually uses pointers to memory functions (i_malloc, i_free, i_calloc, i_realloc) rather than the functions themselves.
If these symbols are not redefined, where are they supposed to come from?
Thanks for any help.
Regards,
Albert
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I tried again with Intel MKL 10.1 beta 2 10.1.0.009. I linked against the following libraries:
mkl_gnu_thread, mkl_lapack, mkl_core, iomp5, pthread, m, gfortran
/opt/intel/mkl/10.1.0.009/lib/em64t was included in the library's rpath.
When I run the program, I get the following error:
MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so
Even if I add /opt/intel/mkl/10.1.0.009/lib/em64t to LD_LIBRARY_PATH, the same thing happens.
The strange thing is this case is that if I strace the process, it looks like it successfully opens these files:
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
Any thoughts?
Regards,
Albert
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I ran into the same problem with 10.0.4.023, but on an ia32 architecture (centrino). The error message is the same as in your case, just with 'em64t' replaced by '32' in the paths to the affected library.
As I rely on python and numpy quite a lot, I wasn't brave enough to test out the recent beta version; instead, I went backwards...
With 10.0.3.020, I get the following error:
MKL FATAL ERROR: /opt/intel/mkl/10.0.3.020/lib/32/: cannot read file data: Is a directory
and I do remember that the same error message came up when I tried this version (10.0.3.020) when it came out, but with numpy-1.0.4. The directory in question is readable by everyone on the system, so there's definitely no permission problem. I believe I looked with strace as well and found out that MKL apparently tried to open a directory as a plain file, but I'm not a 100% sure; it's looo long ago...:-((
I went further back in time / versions to 10.0.2.018, and this version does seem to work; at least, numpy.test() runs without failures.
Hope this helps,
Christian.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm trying 10.0.4.023 again. After more perusal of the User's Guide, I added mkl_intel_lp64 to my list of libraries. I now have:
mkl_intel_lp64, mkl_gnu_thread, mkl_lapack, mkl_core, iomp5, pthread, m, gfortran
As far as I can tell, this should do the trick. An interesting issue was that using mkl_intel_ilp64 instead of mkl_intel_lp64 caused the linker to segfault.
However, I still get this error:
MKL FATAL ERROR: /opt/intel/mkl/10.0.4.023/lib/em64t/libmkl_def.so: undefined symbol: i_free
A MATLAB MEX file that I have compiled using these libraries seems to work fine. Maybe the problems are being caused by some weird interaction when Python loads the module which is linked against MKL.
Cheers,
Albert
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
MKL FATAL ERROR: /opt/intel/mkl/10.0.4.023/lib/em64t/libmkl_def.so: undefined symbol: i_free
I've tested with 10.1.0.018, the same issue occurs with Numpy. I have to say that it does also occur with Matlab : http://www.mathworks.com/matlabcentral/newsreader/view_thread/237079
I tried to load libmkl_def.so directly, the function is missing as well (tried libmkl_lacpack with the same result, missing __mkl_cfg_file_readed_extern function). So there is something that a lot of people are missing, and it would be great if Intel gave the answer.
This issue occured as well on some other programs. I have a shared library that uses MKL (LAPACK and FFT). If this library is linked to a program, it works. If I link it to a library that is dynamically loaded (f2py), the crash occurs.
I heard of libimalloc, but I didn't have the time to check now, but I'll try tomorrow to link against it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I've tested with 10.1.0.018, the same issue occurs with Numpy. I have to say that it does also occur with Matlab : http://www.mathworks.com/matlabcentral/newsreader/view_thread/237079
I tried to load libmkl_def.so directly, the function is missing as well (tried libmkl_lacpack with the same result, missing __mkl_cfg_file_readed_extern function). So there is something that a lot of people are missing, and it would be great if Intel gave the answer.
This issue occured as well on some other programs. I have a shared library that uses MKL (LAPACK and FFT). If this library is linked to a program, it works. If I link it to a library that is dynamically loaded (f2py), the crash occurs.
I heard of libimalloc, but I didn't have the time to check now, but I'll try tomorrow to link against it.
libimalloc is for Windows, so no luck for me :(
Only the FFT has an issue, I tracked down the problem to a call to DftiCommitDescriptor in my own code (don't know where it crashes exactly for Numpy or Matlab).
I tested 10.0.5 and 10.0.4 without any luck, but 10.0.011 seems OK, I didn't encounter the crash this time. I don't know what changed between 10.0.011 and 10.0.4 regarding i_free, so Intel should help us solve this issue ;)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I tested 10.0.5 and 10.0.4 without any luck, but 10.0.011 seems OK, I didn't encounter the crash this time. I don't know what changed between 10.0.011 and 10.0.4 regarding i_free, so Intel should help us solve this issue ;)
I've just tested 10.0.2 and 10.0.3. 10.0.2 is working, 10.0.3 is not, so there is a difference between them that is causing the troubles we are seeing.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think the key point is here:
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
It is not appropriate way of using for MKL 10.
As workaroundyou can tryopen("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_core.so", O_RDONLY) prior to open libmkl_[mc|def].so
"Note, we don't support the loading of processor-specific libraries and the direct usage of symbols from these libraries.
The symbols i_malloc, i_free,... are defined in libmkl_core.so. Starting with MKL 10.0 update 3, these symbols are *Undefined* in the libraries libmkl_def.so, libmkl_p4p.so and so on. We have removed some processor-independent functionality from processor-specific libraries.
More likely the reason of undefined symbol i_free is - the libmkl_def.so was "loaded on the fly"**before**libmkl_core.so has been loaded.
"
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think the key point is here:
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
It is not appropriate way of using for MKL 10.
As workaroundyou can tryopen("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_core.so", O_RDONLY) prior to open libmkl_[mc|def].so
"Note, we don't support the loading of processor-specific libraries and the direct usage of symbols from these libraries.
The symbols i_malloc, i_free,... are defined in libmkl_core.so. Starting with MKL 10.0 update 3, these symbols are *Undefined* in the libraries libmkl_def.so, libmkl_p4p.so and so on. We have removed some processor-independent functionality from processor-specific libraries.
More likely the reason of undefined symbol i_free is - the libmkl_def.so was "loaded on the fly"**before**libmkl_core.so has been loaded.
"
INTEL_LIB= /mnt/gpfs/opt/compiler/intel/ict/3.2/mkl/10.1.0.015/lib/em64t
LIBS= -L$(INTEL_LIB) -lmkl_scalapack -lmkl_blacs_intelmpi_lp64 -lmkl_core -lmkl_lapack -lmkl_em64t -lguide -lmkl_mc -lmkl_def -lmkl_gnu_thread -liomp5
Highlighted libs added will now load dynamically; a weird fix that works.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I tried again with Intel MKL 10.1 beta 2 10.1.0.009. I linked against the following libraries:
mkl_gnu_thread, mkl_lapack, mkl_core, iomp5, pthread, m, gfortran
/opt/intel/mkl/10.1.0.009/lib/em64t was included in the library's rpath.
When I run the program, I get the following error:
MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so
Even if I add /opt/intel/mkl/10.1.0.009/lib/em64t to LD_LIBRARY_PATH, the same thing happens.
The strange thing is this case is that if I strace the process, it looks like it successfully opens these files:
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
Any thoughts?
Regards,
Albert
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I tried again with Intel MKL 10.1 beta 2 10.1.0.009. I linked against the following libraries:
mkl_gnu_thread, mkl_lapack, mkl_core, iomp5, pthread, m, gfortran
/opt/intel/mkl/10.1.0.009/lib/em64t was included in the library's rpath.
When I run the program, I get the following error:
MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so
Even if I add /opt/intel/mkl/10.1.0.009/lib/em64t to LD_LIBRARY_PATH, the same thing happens.
The strange thing is this case is that if I strace the process, it looks like it successfully opens these files:
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
Any thoughts?
Regards,
Albert
export LD_PRELOAD=/share/apps/ifc/mkl/lib/em64t/libmkl_core.so:/share/apps/ifc/mkl/lib/em64t/libmkl_sequential.so
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just to let you know, I've upgrade my MKL to the latest version (11.3.2), and this bug in mkl_rt is still happening. Also, the LD_PRELOAD hack is still working.
I had this "Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so" problem while running tests from petsc4py. Everything is standard, except numpy and PETSc are configured to use mkl (in the most standard way, through mkl_rt only).
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page