Intel® oneAPI Math Kernel Library
Ask questions and share information with other developers who use Intel® Math Kernel Library.

MKL FATAL ERROR: undefined symbol: i_free

fullung
Beginner
2,122 Views
Hello,

I'm trying to use Intel MKL 10.0.4.023 with some third party code (NumPy) on 64-bit Linux.

I am able to successfully compile and link the code, but when I run it and call a function that uses MKL, I get the following error message:

MKL FATAL ERROR: /opt/intel/mkl/10.0.4.023/lib/em64t/libmkl_def.so: undefined symbol: i_free

The library that is called was linked against MKL with:

-shared -lmkl -lguide -lpthread -Wl,-rpath=/opt/intel/mkl/10.0.4.023/lib/em64t

I also tried using the "pure layered model" libraries as described in the User's Guide, but I get the same error.

The User's Guide also says:

Redefining is possible because Intel MKL actually uses pointers to memory functions (i_malloc, i_free, i_calloc, i_realloc) rather than the functions themselves.

If these symbols are not redefined, where are they supposed to come from?

Thanks for any help.

Regards,

Albert
0 Kudos
13 Replies
fullung
Beginner
2,122 Views
Hello,

I tried again with Intel MKL 10.1 beta 2 10.1.0.009. I linked against the following libraries:

mkl_gnu_thread, mkl_lapack, mkl_core, iomp5, pthread, m, gfortran

/opt/intel/mkl/10.1.0.009/lib/em64t was included in the library's rpath.

When I run the program, I get the following error:

MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so

Even if I add /opt/intel/mkl/10.1.0.009/lib/em64t to LD_LIBRARY_PATH, the same thing happens.

The strange thing is this case is that if I strace the process, it looks like it successfully opens these files:

open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3

Any thoughts?

Regards,

Albert
0 Kudos
Christian_M_
Beginner
2,122 Views
Hi,

I ran into the same problem with 10.0.4.023, but on an ia32 architecture (centrino). The error message is the same as in your case, just with 'em64t' replaced by '32' in the paths to the affected library.

As I rely on python and numpy quite a lot, I wasn't brave enough to test out the recent beta version; instead, I went backwards...

With 10.0.3.020, I get the following error:

MKL FATAL ERROR: /opt/intel/mkl/10.0.3.020/lib/32/: cannot read file data: Is a directory

and I do remember that the same error message came up when I tried this version (10.0.3.020) when it came out, but with numpy-1.0.4. The directory in question is readable by everyone on the system, so there's definitely no permission problem. I believe I looked with strace as well and found out that MKL apparently tried to open a directory as a plain file, but I'm not a 100% sure; it's looo long ago...:-((

I went further back in time / versions to 10.0.2.018, and this version does seem to work; at least, numpy.test() runs without failures.

Hope this helps,

Christian.
0 Kudos
fullung
Beginner
2,122 Views
Hello all,

I'm trying 10.0.4.023 again. After more perusal of the User's Guide, I added mkl_intel_lp64 to my list of libraries. I now have:

mkl_intel_lp64, mkl_gnu_thread, mkl_lapack, mkl_core, iomp5, pthread, m, gfortran

As far as I can tell, this should do the trick. An interesting issue was that using mkl_intel_ilp64 instead of mkl_intel_lp64 caused the linker to segfault.

However, I still get this error:

MKL FATAL ERROR: /opt/intel/mkl/10.0.4.023/lib/em64t/libmkl_def.so: undefined symbol: i_free

A MATLAB MEX file that I have compiled using these libraries seems to work fine. Maybe the problems are being caused by some weird interaction when Python loads the module which is linked against MKL.

Cheers,

Albert
0 Kudos
TimP
Honored Contributor III
2,122 Views
In this list of libraries, you should not require the mkl_lapack script. If you use libmkl_intel_thread (specified in the mkl_lapack), you should not also attempt to use the gnu_thread library. Likewise, as you saw, if you specify both ilp64 and lp64, you will have trouble. If you are using .so libraries, you should not need to specify libraries more than once.
0 Kudos
Matthieu_Brucher
Beginner
2,122 Views
However, I still get this error:

MKL FATAL ERROR: /opt/intel/mkl/10.0.4.023/lib/em64t/libmkl_def.so: undefined symbol: i_free

I've tested with 10.1.0.018, the same issue occurs with Numpy. I have to say that it does also occur with Matlab : http://www.mathworks.com/matlabcentral/newsreader/view_thread/237079

I tried to load libmkl_def.so directly, the function is missing as well (tried libmkl_lacpack with the same result, missing __mkl_cfg_file_readed_extern function). So there is something that a lot of people are missing, and it would be great if Intel gave the answer.

This issue occured as well on some other programs. I have a shared library that uses MKL (LAPACK and FFT). If this library is linked to a program, it works. If I link it to a library that is dynamically loaded (f2py), the crash occurs.

I heard of libimalloc, but I didn't have the time to check now, but I'll try tomorrow to link against it.

0 Kudos
Matthieu_Brucher
Beginner
2,122 Views
Quoting - matthieubrucher

I've tested with 10.1.0.018, the same issue occurs with Numpy. I have to say that it does also occur with Matlab : http://www.mathworks.com/matlabcentral/newsreader/view_thread/237079

I tried to load libmkl_def.so directly, the function is missing as well (tried libmkl_lacpack with the same result, missing __mkl_cfg_file_readed_extern function). So there is something that a lot of people are missing, and it would be great if Intel gave the answer.

This issue occured as well on some other programs. I have a shared library that uses MKL (LAPACK and FFT). If this library is linked to a program, it works. If I link it to a library that is dynamically loaded (f2py), the crash occurs.

I heard of libimalloc, but I didn't have the time to check now, but I'll try tomorrow to link against it.

libimalloc is for Windows, so no luck for me :(

Only the FFT has an issue, I tracked down the problem to a call to DftiCommitDescriptor in my own code (don't know where it crashes exactly for Numpy or Matlab).

I tested 10.0.5 and 10.0.4 without any luck, but 10.0.011 seems OK, I didn't encounter the crash this time. I don't know what changed between 10.0.011 and 10.0.4 regarding i_free, so Intel should help us solve this issue ;)

0 Kudos
Matthieu_Brucher
Beginner
2,122 Views
Quoting - matthieubrucher

I tested 10.0.5 and 10.0.4 without any luck, but 10.0.011 seems OK, I didn't encounter the crash this time. I don't know what changed between 10.0.011 and 10.0.4 regarding i_free, so Intel should help us solve this issue ;)

I've just tested 10.0.2 and 10.0.3. 10.0.2 is working, 10.0.3 is not, so there is a difference between them that is causing the troubles we are seeing.

0 Kudos
Andrey_Bespalov
New Contributor I
2,122 Views

I think the key point is here:

open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3

It is not appropriate way of using for MKL 10.

As workaroundyou can tryopen("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_core.so", O_RDONLY) prior to open libmkl_[mc|def].so

"Note, we don't support the loading of processor-specific libraries and the direct usage of symbols from these libraries.

The symbols i_malloc, i_free,... are defined in libmkl_core.so. Starting with MKL 10.0 update 3, these symbols are *Undefined* in the libraries libmkl_def.so, libmkl_p4p.so and so on. We have removed some processor-independent functionality from processor-specific libraries.

More likely the reason of undefined symbol i_free is - the libmkl_def.so was "loaded on the fly"**before**libmkl_core.so has been loaded.
"

0 Kudos
thomasmd
Beginner
2,122 Views

I think the key point is here:

open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3

It is not appropriate way of using for MKL 10.

As workaroundyou can tryopen("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_core.so", O_RDONLY) prior to open libmkl_[mc|def].so

"Note, we don't support the loading of processor-specific libraries and the direct usage of symbols from these libraries.

The symbols i_malloc, i_free,... are defined in libmkl_core.so. Starting with MKL 10.0 update 3, these symbols are *Undefined* in the libraries libmkl_def.so, libmkl_p4p.so and so on. We have removed some processor-independent functionality from processor-specific libraries.

More likely the reason of undefined symbol i_free is - the libmkl_def.so was "loaded on the fly"**before**libmkl_core.so has been loaded.
"

Add the libs to the make file e.g.
INTEL_LIB= /mnt/gpfs/opt/compiler/intel/ict/3.2/mkl/10.1.0.015/lib/em64t
LIBS= -L$(INTEL_LIB) -lmkl_scalapack -lmkl_blacs_intelmpi_lp64 -lmkl_core -lmkl_lapack -lmkl_em64t -lguide -lmkl_mc -lmkl_def -lmkl_gnu_thread -liomp5

Highlighted libs added will now load dynamically; a weird fix that works.

0 Kudos
Ron_Cohen
Beginner
2,122 Views
Hello,

I tried again with Intel MKL 10.1 beta 2 10.1.0.009. I linked against the following libraries:

mkl_gnu_thread, mkl_lapack, mkl_core, iomp5, pthread, m, gfortran

/opt/intel/mkl/10.1.0.009/lib/em64t was included in the library's rpath.

When I run the program, I get the following error:

MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so

Even if I add /opt/intel/mkl/10.1.0.009/lib/em64t to LD_LIBRARY_PATH, the same thing happens.

The strange thing is this case is that if I strace the process, it looks like it successfully opens these files:

open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3

Any thoughts?

Regards,

Albert

0 Kudos
Ron_Cohen
Beginner
2,122 Views
Hello,

I tried again with Intel MKL 10.1 beta 2 10.1.0.009. I linked against the following libraries:

mkl_gnu_thread, mkl_lapack, mkl_core, iomp5, pthread, m, gfortran

/opt/intel/mkl/10.1.0.009/lib/em64t was included in the library's rpath.

When I run the program, I get the following error:

MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so

Even if I add /opt/intel/mkl/10.1.0.009/lib/em64t to LD_LIBRARY_PATH, the same thing happens.

The strange thing is this case is that if I strace the process, it looks like it successfully opens these files:

open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_mc.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3
open("/opt/intel/mkl/10.1.0.009/lib/em64t/libmkl_def.so", O_RDONLY) = 3

Any thoughts?

Regards,

Albert
I found this to work:
export LD_PRELOAD=/share/apps/ifc/mkl/lib/em64t/libmkl_core.so:/share/apps/ifc/mkl/lib/em64t/libmkl_sequential.so
0 Kudos
John_P_
Beginner
2,122 Views
The "export LD_PRELOAD" trick worked for me too. I couldn't run the simple python numpy example because I was getting the "MKL FATAL ERROR: Cannot load neither libmkl_mc.so nor libmkl_def.so"
0 Kudos
Edson_Tadeu_M_
Beginner
2,122 Views

Just to let you know, I've upgrade my MKL to the latest version (11.3.2), and this bug in mkl_rt is still happening. Also, the LD_PRELOAD hack is still working.

I had this "Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so" problem while running tests from petsc4py. Everything is standard, except numpy and PETSc are configured to use mkl (in the most standard way, through mkl_rt only).

0 Kudos
Reply