- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I am struggling with the recompilation of R using the Intel MKL. I have identified the problem to the following:
gfortran -c testf.f
gcc -c test.c
gcc -L${MKL_LIB_PATH} -o test test.o testf.o -lmkl_intel_ilp64 -lmkl_core -lmkl_sequential
When running the program, I get a sigfault 11 error message.
I tried debugging with gdb but couldn't really pinpoint the problem. It seems all the good libraries are being used (otool -L) points to the MKL libraries...
I am running out of options...
thanks,
Bernd
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The problems are caused by the fact that your chosen name for the executable, i.e., "test", is the same as the name of a standard Linux/Unix utility, see http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html .
Use a different name for your executable file, or set PATH so that your "test" occurs before the system "test".
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for the reply but I am calling the program like this:
./test
and the unix test doesn't give a sig-fault...
Any other suggestions?
thanks again
Bernd
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello
Could you please see if this command line
>source /opt/intel/compilers_and_libraries_2018/mac/mkl/bin/mklvars.sh intel64 (important as environment variable settting)
>gcc -L${MKL_LIB_PATH} -o test test.o testf.o -lmkl_rt
work?
and here is some articles for your reference :
https://software.intel.com/en-us/articles/quick-linking-intel-mkl-blas-lapack-to-r ; (quick)
and others:
Using Intel® MKL with R | Intel® Software
Best Regards,
Ying
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I looked at the two files that you attached. Unless you did something behind the scenes (with compiler options, environment variables, etc.), the integer arguments being passed to the MKL routine are default integers, i.e, 4-byte integers. If this guess is correct, the MKL library you used is not a proper match. You should probably try -lmkl_intel_lp64 instead of -lmkl_intel_ilp64. The "ilp64" library is for 64-bit Integer, Long, Pointer.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ying,
Thanks for your help. Here is what I did:
MD18-1021:R-patched2 bernd$ source /opt/intel/compilers_and_libraries_2018/mac/mkl/bin/mklvars.sh intel64 verbose DYLD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib NLSPATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/locale/%l_%t/%N:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/locale/%l_%t/%N CPATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/include:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/include PKG_CONFIG_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/bin/pkgconfig:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/bin/pkgconfig MD18-1021:R-patched2 bernd$ echo $MKL_LIB_PATH /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/ MD18-1021:R-patched2 bernd$ gcc -L${MKL_LIB_PATH} -o test test.o testf.o -lmkl_rt MD18-1021:R-patched2 bernd$ otool -L test test: @rpath/libmkl_rt.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4) MD18-1021:R-patched2 bernd$ ./test Segmentation fault: 11 MD18-1021:R-patched2 bernd$
I have started from a clean (new High Sierra) installed system with the following additional installations:
MD18-1021:R-patched2 bernd$ brew list boost cmake eigen freetype gdb glib gsl jpeg libgit2 libmpc libssh2 libunistring openssl pixman texinfo xz cairo curl fontconfig gcc gettext gmp isl libffi libidn2 libpng libtiff mpfr pcre pkg-config wget
and java jdk
and the other variables (from set):
BASH=/bin/bash CPATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/include:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/include CPRO_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac DISPLAY=/private/tmp/com.apple.launchd.pWYIDgzn7y/org.macosforge.xquartz:0 DYLD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib HOSTTYPE=x86_64 LC_CTYPE=UTF-8 LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/ LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/tbb/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/compiler/lib:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib MACHTYPE=x86_64-apple-darwin17 MKLROOT=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl MKL_LIB_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/ NLSPATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/locale/%l_%t/%N:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/locale/%l_%t/%N OMP_NUM_THREADS=2 OPTERR=1 OPTIND=1 OSTYPE=darwin17 PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin PKG_CONFIG_PATH=/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/bin/pkgconfig:/opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/bin/pkgconfig SHELL=/bin/bash
Please let me know if you need additional information.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
another sanity check: I removed all the env variables that gcc used such that compilation and then execution failed and added the info specifically later... same result....
# with nothing compilation fails MD18-1021:R-patched2 bernd$ gcc -o test test.o testf.o -lmkl_rt ld: library not found for -lmkl_rt clang: error: linker command failed with exit code 1 (use -v to see invocation) MD18-1021:R-patched2 bernd$ gcc -L$MKL_LIB_PATH -o test test.o testf.o -lmkl_rt # execution fails because LDLIBRARY_PATH is not set MD18-1021:R-patched2 bernd$ ./test dyld: Library not loaded: @rpath/libmkl_rt.dylib Referenced from: /Users/bernd/Downloads/R-patched2/./test Reason: image not found Abort trap: 6 MD18-1021:R-patched2 bernd$ echo $MKLROOT /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl # specifially write library location in executable MD18-1021:R-patched2 bernd$ gcc -L$MKL_LIB_PATH -Wl,-rpath,${MKLROOT}/lib -o test test.o testf.o -lmkl_rt # execution problem... MD18-1021:R-patched2 bernd$ ./test Segmentation fault: 11
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
and with different library options:
MD18-1021:R-patched2 bernd$ gcc -L$MKL_LIB_PATH -Wl,-rpath,${MKLROOT}/lib -o test test.o testf.o -lmkl_rt MD18-1021:R-patched2 bernd$ ./test Segmentation fault: 11 MD18-1021:R-patched2 bernd$ gcc -L$MKL_LIB_PATH -Wl,-rpath,${MKLROOT}/lib -o test test.o testf.o -lmkl_intel_ilp64 -lmkl_core -lmkl_sequential MD18-1021:R-patched2 bernd$ ./test Segmentation fault: 11 MD18-1021:R-patched2 bernd$ gcc -L$MKL_LIB_PATH -Wl,-rpath,${MKLROOT}/lib -o test test.o testf.o -lmkl_intel_lp64 -lmkl_core -lmkl_sequential MD18-1021:R-patched2 bernd$ ./test Segmentation fault: 11
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The test program runs fine on Linux, and I cannot test on a Mac. You could try compiling with traceback enabled and localize the place where the segfault occurs.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
thanks for the effort and time. Here is the gdb output. Please let me know what you want me to do as explicit as possible to avoid misunderstandings ;)
MD18-1021:R-patched2 bernd$ gcc -c test.c -g MD18-1021:R-patched2 bernd$ gfortran -c testf.f -g MD18-1021:R-patched2 bernd$ gcc -L$MKL_LIB_PATH -Wl,-rpath,${MKLROOT}/lib -o test test.o testf.o -lmkl_rt -g MD18-1021:R-patched2 bernd$ gdb ./test GNU gdb (GDB) 8.0.1 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-apple-darwin17.0.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./test...done. (gdb) r Starting program: /Users/bernd/Downloads/R-patched2/test [New Thread 0x2703 of process 80741] warning: unhandled dyld version (15) Thread 2 received signal SIGSEGV, Segmentation fault. 0x00000001083e14dc in ?? () (gdb) bt #0 0x00000001083e14dc in ?? () #1 0x0000000000000000 in ?? () (gdb)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can it be that the order of architectures is important? (since there is this unhandled dyld version problem)...
I get for the two libraries that are used:
MD18-1021:R-patched2 bernd$ file /usr/lib/libSystem.B.dylib /usr/lib/libSystem.B.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [i386:Mach-O dynamically linked shared library i386] /usr/lib/libSystem.B.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64 /usr/lib/libSystem.B.dylib (for architecture i386): Mach-O dynamically linked shared library i386 MD18-1021:R-patched2 bernd$ file /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/libmkl_rt.dylib /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/libmkl_rt.dylib: Mach-O universal binary with 2 architectures: [i386:Mach-O dynamically linked shared library i386] [x86_64] /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/libmkl_rt.dylib (for architecture i386): Mach-O dynamically linked shared library i386 /opt/intel/compilers_and_libraries_2018.3.185/mac/mkl/lib/libmkl_rt.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
For reference the system variables are set like this:
HOSTTYPE=x86_64 MACHTYPE=x86_64-apple-darwin17
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
the warning is not important and some gdb/high sierra problem. Hello world works:
Starting program: /Users/bernd/Downloads/R-patched2/test2 [New Thread 0x2603 of process 1635] warning: unhandled dyld version (15) Hello, World![Inferior 1 (process 1635) exited normally] (gdb)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ok, I understood gdb a bit better... zdotu is causing the problem...
Thread 2 hit Breakpoint 1, main () at test.c:12 12 F77_SYMBOL(test1)(&iflag); (gdb) s test1 (iflag=16850) at testf.f:10 10 zx(1) = (3.1d0,1.7d0) (gdb) s 11 zx(2) = (1.6d0,-0.6d0) (gdb) s 12 zres = zdotu(2, zx, 1, zx, 1) (gdb) s Thread 2 received signal SIGSEGV, Segmentation fault. 0x00000001082534dc in ?? ()
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Using a Fortran main instead of the C main, and setting the compiler options -g and -fbacktrace, would provide useful information regarding the location of the fault.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Currently, I am trying to understand gdb better to see if I can locate the zdotu function, which I cannot at the moment. My Fortran experience is already a few decades old and I it is even less evident to me how to transform the function into a real Fortran program.
Do you know how to list the functions from linked libraries in gdb? I get the following, which leads me to try to verify that the libraries are correctly bound:
MD18-1021:R-patched2 bernd$ nm -add-dyldinfo -no-sort -print-file-name test test: 0000000100000000 T __mh_execute_header test: 0000000100000ca0 T _main test: 0000000100000cd3 T _test1_ test: 0000000100000cc0 T _xerbla_ test: U _cabs test: U _exit test: U _zdotu_ test: U dyld_stub_binder
But maybe it is just because the libraries are not compiled with -g...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have now statically linked the mkl libraries and get the same error at the same line (zdotu). Since there is no debugging information available I don't think that compiling a new Fortran function would work.
On the internet, I saw comments about the stack not being sufficient, but when I looked at the offending assembler it was a mov command that caused the problem...
Is there a way to get hold of a mkl library that allows further debugging?
Maybe this has something to do with this problem: when compiling R I can link to the Acceleration library from Apple (MKL is causing the problem above), but I don't think BLAS is working in the performance tests: I can only see one CPU active in the task manager and I don't see any effect greater than a factor of 1.3 when using the Accelerate packages or not. (using disable-blas) If you have any idea on how to verify this side of my problem of getting more performance I would appreciate it a lot as well.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If Fortran is not your forte, you could consider using C-BLAS and LAPACKE, both of which are supported by MKL, provided the number of BLAS and Lapack routines that you use is small. You would then need no Fortran code at all. See, for example, https://software.intel.com/en-us/mkl-developer-reference-c-cblas-dotu#E81FF20A-C401-4A15-A64F-CC0DAA2BD65D .
I have not used Macintoshes since they switched to OSX and Intel CPUs (~ 20 years ago).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just to clarify, I ran into this problem on a very fresh Mac high sierra when trying to recompile R with the MKL support as it was highly recommended to me by several sources. Specifically, the configure process didn't work and I was able to narrow down the problem to the compilation of this example, which I could reproduce.
I don't see which additional information one could gain by only using Fortran, as I have shown that the zdotu function is causing the problem. If you could further explain this, it would clearly motivate me to go further in this direction.
Thanks a lot for your support,
Kind regards,
Bernd
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I just saw this in the R documentation:
On some systems it has been necessary that an external BLAS/LAPACK was built with the same FORTRAN compiler used to build R.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
jagla, bernd wrote:
I don't see which additional information one could gain by only using Fortran, as I have shown that the zdotu function is causing the problem. If you could further explain this, it would clearly motivate me to go further in this direction.
My suggestion was the exact opposite, i.e., to use no Fortran code at all. MKL gives you complete access to BLAS and Lapack from C, with function prototypes provided to check function arguments. As long as you (well, the parts of the source code of R that use BLAS/Lapack) adhere to these prototypes, you do not need to use a Fortran compiler at all, unless there are pieces of R itself that are written in Fortran (I don't know this).
As a "proof-of-concept" exercise, you could attempt to build R using the system-provided BLAS and Lapack. If that works, you could replace those libraries with MKL. You may be able to just relink, without having to recompile anything.

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page