Using MKL on Mac with R-3.0.2

[Simon_Z_]

I am building R-3.0.2 on my MacBook Pro (OS X 10.9, Intel i7 DCore). When I look into my configure log I see:

configure:29151: checking for dgemm_ in -L/opt/intel/composer_xe_2013_sp1.1.103/mkl/lib -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm
configure:29172: /opt/intel/composer_xe_2013_sp1.1.103/bin/intel64/icc -std=gnu99 -o conftest -O3 -openmp -xHost -fPIC -I/usr/local/include -openmp conftest.c -L/opt/intel/composer_xe_2013_sp1.1.103/mkl/lib -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libifport.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libifcoremt.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libimf.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libsvml.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libipgo.a -liomp5 /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libirc.a -lpthread /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.0/lib/darwin/libclang_rt.osx.a -lm >&5
conftest.c(203): warning #266: function "dgemm_" declared implicitly
dgemm_()
^

The DYLD_LIBRARY_PATH seems to be correct. I cchecked also the libraries on the Intel linker line tool - seem to be the correct versions. Similar outputs can be seen for other blas functions. Always: declared implicitly.

Furthermore I get

configure:30282: /opt/intel/composer_xe_2013_sp1.1.103/bin/intel64/icc -std=gnu99 -o conftest -O3 -openmp -xHost -fPIC -I/usr/local/include -openmp conftest.c -lm >&5
Undefined symbols for architecture x86_64:
"_iconv", referenced from:
_main in icc2Cz8oX.o
"_iconv_close", referenced from:
_main in icc2Cz8oX.o
"_iconv_open", referenced from:
_main in icc2Cz8oX.o
ld: symbol(s) not found for architecture x86_64

What could be the reason for this behavior?

[TimP]

A command such as

source /opt/intel/composer_xe_2013_sp1.1.103/bin/compilervars.sh intel64

would set up the environment so that both mkl ifort and icc compiler libraries can be found without your adding specific -L paths or listing libirc, libifcore, libifport, or libsvml in your link command.  Then you could use "icc" without specifying its full path.

libifcore and libifport would be provided with ifort, which (if the same version as icc) would also be set up by compilervars

The implicit function warning could be avoided by including whichever .h file from mkl/include has that prototype.  There you would probably need to add the -I path for the mkl/include directory.  The implicit warning isn't necessarily a problem but it warns you that your dgemm function call can't be checked.

The -openmp option in your command will take care of -liomp5 -lpthread -lm, so those should be redundant.

Does your build need -liconv  ?  I don't know if libiconv (a popular open source package) would come with your system.

[Simon_Z_]

So, I followed your suggestions above and sourced compilervars.sh. Actually what I did was to create a modulefile for my environment modules and created therein the same variables. I checked several times via env | sort > envbefore; source compilervars.sh; env | sort > envafter; diff envbefore envafter;  Then, icc and ifort as well as mkl are found with no problem. 

I tried to get rid of the implicit function warning by including the headers via -I$MKLROOT/include, but there was no difference: 

configure:29151: checking for dgemm_ in -mkl -I/opt/intel/composer_xe_2013_sp1.1.103/mkl/include
configure:29172: icc -std=gnu99 -o conftest -O3 -openmp -xHost -fPIC -I/usr/local/include -openmp conftest.c -mkl -I/opt/intel/composer_xe_2013_sp1.1.103/mkl/include /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libifport.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libifcoremt.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libimf.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libsvml.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libipgo.a -liomp5 /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libirc.a -lpthread /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.0/lib/darwin/libclang_rt.osx.a -lm >&5
conftest.c(203): warning #266: function "dgemm_" declared implicitly
dgemm_()

The liconv seems not to be needed for the build of R. I still get the errors: 

configure:30840: checking for wctrans_t
configure:30840: icc -std=gnu99 -c -O3 -openmp -xHost -fPIC -I/usr/local/include conftest.c >&5
configure:30840: $? = 0
configure:30840: icc -std=gnu99 -c -O3 -openmp -xHost -fPIC -I/usr/local/include conftest.c >&5
conftest.c(215): error: expected an expression
if (sizeof ((wctrans_t)))
^

compilation aborted for conftest.c (code 2)

but it builds. Though at the end, when calling sudo make install I get the error: 

dyld: Library not loaded: libiomp5.dylib
Referenced from: /Users/simonzehnder/Downloads/build/bin/exec/R
Reason: image not found
/bin/sh: line 1: 1845 Done echo "utils:::make.packages.html(.Library, verbose=FALSE, docdir=\"/usr/local/R/R-intel/R.framework/Versions/3.0/Resources/doc\")"
1846 Trace/BPT trap: 5 | R_DEFAULT_PACKAGES=NULL LC_ALL=C ../../bin/R --vanilla --slave > /dev/null
make[2]: *** [install] Error 133

Looking via otool -L into the libR.dylib I see, that the intel libraries are not included with an absolute path:

libR.dylib (compatibility version 3.0.0, current version 3.0.2)
libmkl_intel_lp64.dylib (compatibility version 0.0.0, current version 0.0.0)
libmkl_intel_thread.dylib (compatibility version 0.0.0, current version 0.0.0)
libmkl_core.dylib (compatibility version 0.0.0, current version 0.0.0)
libiomp5.dylib (compatibility version 5.0.0, current version 5.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 855.11.0)
/usr/lib/libedit.3.dylib (compatibility version 2.0.0, current version 3.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
/usr/lib/libicucore.A.dylib (compatibility version 1.0.0, current version 51.1.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 60.0.0)

One of the R core members told me hereto:

"^^ your Intel libs (or at least libiomp5) are missing paths in the id string - probably not installed properly? You can hack around that using DYLD_FALLBACK_LIBRARY_PATH but it's better to fix the issue in libiomp5 directly." 

Looking into the config.log I find: shlibpath_var='DYLD_LIBRARY_PATH'; so it seems that building makes use of the DYLD_LIBRARY_PATH. In the Makeconf file I though get only the following variables: 

DYLIB_EXT = .dylib
DYLIB_LD = $(CC)
DYLIB_LDFLAGS = -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress # $(DYLIBS_LTO)
DYLIB_LINK = $(DYLIB_LD) $(DYLIB_LDFLAGS) $(LDFLAGS)DYLIBS_LTO = $(CFLAGS) $(CPICFLAGS)

I then asked the core member: 

Where does R in make install get its libraries/library paths from to be installed? I set DYLD_LIBRARY_PATH and libiomp5.dylib is on that path … Is there another variable to be set (DYLD_FALLBACK_LIBRARY_PATH I want to use only, if no other possibility remains).

and the answer was: Well, you really don't want to set DYLD_LIBRARY_PATH since it breaks dependencies, but that's another topic. However, the error tells you that you don't have it on the path, otherwise you wouldn't get "image not found". Either way, as I said, what you really want is to fix the id in the iomp library so you don't get into the path troubles in the first place. Another way to go about it is to put the Intel dependencies into $R_HOME/lib - I wouldn't do that for development but that's a good option if you want to distribute the build to other machines. 

I do not have any clue what he meant with id, probably to use install_names_tool @exectuable/path/to/intel/library/? I wonder if I could simply tell it to use xild instead of ld. xild uses the DYLD_LIBRARY_PATH I guess and the variable $LD is set to xild? 

I attached the config.log, the Makefile and the Makeconf to this message. 

[VipinKumar_E_Intel]

Simon,

   From the config.log I see the following,

configure:29151: checking for dgemm_ in -mkl -I/opt/intel/composer_xe_2013_sp1.1.103/mkl/include
configure:29172: icc -std=gnu99 -o conftest -O3 -openmp -xHost -fPIC  -I/usr/local/include  -openmp conftest.c -mkl -I/opt/intel/composer_xe_2013_sp1.1.103/mkl/include  /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libifport.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libifcoremt.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libimf.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libsvml.a /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libipgo.a -liomp5 /opt/intel/composer_xe_2013_sp1.1.103/compiler/lib/libirc.a -lpthread /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.0/lib/darwin/libclang_rt.osx.a -lm  >&5
conftest.c(203): warning #266: function "dgemm_" declared implicitly
  dgemm_()
  ^

configure:29172: $? = 0
configure:29179: result: yes
configure:29696: checking whether double complex BLAS can be used
configure:29767: result: yes
configure:29787: checking whether the BLAS is complete

The exit status and the results for BLAS has been reported as 0 and yes, if it failed it would have been 1 or no, and you can ignore the implicit warnings.

As it mentioned in your previous communication, can you set the LD="xild" and AR="xiar"  the intel compiler linker and lib archive tool and try?

We are also investigating this at our end will update the thread soon.

--Vipin

[Simon_Z_]

Hi Vipin,

I tried your suggestions immediately. Results remain unchanged:

make[1]: Nothing to be done for `install'.
make[1]: Nothing to be done for `install'.
installing doc ...
installing doc/html ...
installing doc/manual ...
installing etc ...
installing share ...
make[3]: Nothing to be done for `install'.
make[3]: Nothing to be done for `install'.
make[3]: Nothing to be done for `install'.
make[3]: Nothing to be done for `install'.
make[3]: Nothing to be done for `install'.
make[3]: Nothing to be done for `install'.
make[3]: Nothing to be done for `install'.
make[3]: Nothing to be done for `install'.
make[2]: Nothing to be done for `install'.
make[2]: Nothing to be done for `install'.
icc -std=gnu99 -I. -I../../src/include -I../../../R-3.0.2/src/include -I/usr/X11/include/ -I/usr/local/include -DHAVE_CONFIG_H    -fPIC  -O3 -openmp -xHost  -DR_HOME='"/usr/local/R/R-intel/R.framework/Versions/3.0/Resources"' -o Rscript \
      ../../../R-3.0.2/src/unix/Rscript.c
/usr/local/R/R-intel/R.framework/Versions/3.0/Resources/bin/exec/R is unchanged
installing packages ...
  building HTML index ...
dyld: Library not loaded: libiomp5.dylib
  Referenced from: /Users/simonzehnder/Downloads/build/bin/exec/R
  Reason: image not found
/bin/sh: line 1: 88676 Done                    echo "utils:::make.packages.html(.Library, verbose=FALSE, docdir=\"/usr/local/R/R-intel/R.framework/Versions/3.0/Resources/doc\")"
     88677 Trace/BPT trap: 5       | R_DEFAULT_PACKAGES=NULL LC_ALL=C ../../bin/R --vanilla --slave > /dev/null
make[2]: *** [install] Error 133
make[1]: *** [install] Error 1
make: *** [install] Error 1

My configure command was: ../R-3.0.2/configure OBJC=clang --enable-R-framework=/usr/local/R/R-intel --x-includes=/usr/X11/include/ --x-libraries=/usr/X11/lib/ --enable-R-profiling --with-system-zlib --with-openmp --with-blas=-mkl --with-lapack=-mkl AR="xiar" LD="xild"

Best

Simon