Linking BLA95 and LAPACK 95 to a shared library in Linux

[eos_pengwern]

I'm in the process of porting an application from Windows to Linux, and one component of my application is a shared library (a DLL on Windows) compiled with Intel Fortran and with heavy use of the BLAS and LAPACK libraries. 

When trying to compile the library in Linux, my first preference was to link the MKL libraries to it statically. According to the link line advisor, the required definition for LINKLIBS in my makefile was to be:

 $(MKLROOT)/lib/intel64/libmkl_blas95_lp64.a $(MKLROOT)/lib/intel64/libmkl_lapack95_lp64.a -Wl,--start-group $(MKLROOT)/lib/intel64/libmkl_intel_lp64.a $(MKLROOT)/lib/intel64/libmkl_core.a $(MKLROOT)/lib/intel64/libmkl_sequential.a -Wl,--end-group -lpthread -lm

Using this, I obtained the following error message:

ld: /opt/intel/composer_xe_2013_sp1.3.174/mkl/lib/intel64/libmkl_lapack95_lp64.a(dgesdd.o): relocation R_X86_64_32 against `__STRLITPACK_1' can not be used when making a shared object; recompile with -fPIC
/opt/intel/composer_xe_2013_sp1.3.174/mkl/lib/intel64/libmkl_lapack95_lp64.a: could not read symbols: Bad value

What this implies is that libkml_lapack95_lp64.a is only intended for being linked to main applications or other static libraries, not to dynamic libraries. I verified this by typing:

readelf --relocs libmkl_lapack95_lp64.a | egrep '(GOT|PLT|JU?MP|SLOT)'

as suggested here and, sure enough, there is no offset table (even though most of the other MKL static libraries do in fact have them).

This is frustrating (I have another C shared library which I link with the static versions of the IPP libraries without any issues) but I can live with it. So, I went back to the MKL link-line advisor to find how to link the MKL dynamically, and the advice is:

"Use this link line:

 $(MKLROOT)/lib/intel64/libmkl_blas95_lp64 $(MKLROOT)/lib/intel64/libmkl_lapack95_lp64 -L$(MKLROOT)/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lm"

I obeyed this to the letter, and the result is:

ifort: error #10236: File not found:  '/opt/intel/composer_xe_2013_sp1.3.174/mkl/lib/intel64/libmkl_blas95_lp64'
ifort: error #10236: File not found:  '/opt/intel/composer_xe_2013_sp1.3.174/mkl/lib/intel64/libmkl_lapack95_lp64'

Not altogether surprising, because indeed there aren't any files with these names that have either no suffix or a .so suffix. There are just the ".a" ones, and if I add the ".a" at the ends of the filenames in my link line then of course I'm using the same static libraries as before and I get the same relocation error as before. By the way, the same happens if I use the "-lmkl_blas95_lp64" syntax that I've seen documented elsewhere.

So, should I compile my own shared library? It seems that not even this is supported, since if I dig down into /opt/intel/composer_xe_2013_sp1.3.174/mkl/interfaces/blas95 and examine the makefile that is provided for doing a custom build, even that only supports static builds.

So, finally, my question: what should I do? I see three alternatives:

(1) Manually edit the Intel makefiles to recompile libmkl_blas95_lp64.a and libmkl_lapack95_lp64.a with the ifort equivalent to '-fPIC' (if there is one).

(2) Manually edit the Intel makefiles to create ibmkl_blas95_lp64.so and libmkl_lapack95_lp64.so.

(3) Conclude that I'm doing something which, for some reason I can't discern, is unsupported and inadvisable.

Hopefully, there is a fourth alternative which is simpler than any of the above, but I don't know what that may be. I'd very much appreciate some advice. 

[Ying_H_Intel]

Dear eos pengwern, 

Thank you for the nicety reports.  It seems a rare case to build a shared library based on the blas95 library, relative to using them in main program.   As i understand,  the No. 1 should be an easiest way.

(just add -fPIC in Makefile)

opts0    = -fPIC
optsc    = -c $(opts0)

and rebuild them with the command : make libintel64 INSTALL_DIR=lib95. Then use the libraries instead of the prebuild. 

Regarding the shared library of blas95/lapack95, you are right, at present we only provide static library, there is not *.so. So the *.a shoud be  add libmkl_blas95_lp64.a and libmkl_lapack95_lp64.a. I will ask our developer to fix the problem in MKL link adviser. 

One tiny question, you mentioned your application with heavy use of the BLAS and LAPACK libraries. are you using Fortran 95 interface or just common fortran interface? if just fortran interface, then the lapack95, blas95 linterface library are not needed. 

Best Regards,

Ying 

[eos_pengwern]

Thank you Ying; that has worked!

I’m surprised that my use case is a rare one: what I’m doing is to build a library of high-performance math functions that are called from a main application written in Java. The library was written in Fortran 2003 from the beginning, so uses Fortran 95 interfaces for BLAS and LAPACK throughout.

I have one supplementary question… At the moment I’m not using a Xeon Phi coprocessor, but I plan to get one within the next three months or so. When I do, then presumably I’ll need to link with the versions of libmkl_blas95_lp64.a and libmkl_lapack95_lp64.a given in the /opt/intel/mkl/lib/mic directory. A quick check with ‘readelf’ shows that these don’t have offset tables either, even though the other .a files in the same directory do. I expect I’ll have to build my own versions of these as well, but again I can’t see an explicit option for this in the provided makefile. When the time comes, what should I do?

Many thanks,

Eos.

[Ying_H_Intel]

Hi Eos Pengwern, 

Thanks for sharing. Yes, it seems a problem when using a Xeon Phi coprocessor.  You need to rebuild your own version.  You may need edit the makefile to add option mic 

and the key for mic is to add -mmic option.  Please try it and let us know if any problem. 

opts0    = -fPIC -mmic
optsc    = -c $(opts0)

make libintel64 IA=mic INSTALL_DIR=mic

and the *.mod and *.a  are in mic folder. 

Best Regards,

Ying 

[eos_pengwern]

Thank you Ying. I think that worked. When I did the build, each file gave me an error message saying:

ifort: warning #10362: Environment configuration problem encountered.  Please check for proper MPSS installation and environment setup.

...which I have an inkling may be just because I was compiling on a computer without an MIC. In any case, I got the .a files at the end of the process, but I won't be able to test them until I do get an MIC system in a few months' time.

[TimP]

You should be able to install the MIC library components (included in MPSS) even if you don't have a MIC.  As it's entirely cross compilation and linking, the coprocessor isn't used until run time.  I haven't heard of any changes which would allow the MIC linker to use ordinary .a files; you would need the special tools.

[eos_pengwern]

Thank you Tim. 

I looked into this; you're absolutely right, MPSS can be installed on a system without a Xeon Phi, but it looks like there's another snag: the downloadable versions are only supported on Red Hat or Suse, whilst I'm running Debian. It's clearly possible to install it on Debian (see here for example) but it still looks like a bit of hackery is involved. As this won't be on my critical path until 3-6 months from now, I think I'll leave off getting to grips with this until then.

[schulzey]

Hi Stephen,

I don't mean to hi-jack this thread, but I noticed your issues with trying to find out which MKL DLLs to distribute with your application in a 2011 thread => https://software.intel.com/en-us/forums/topic/282514. Did you find a resolution to the problem?

I have a similar issue. I'm only using two MKL routines (Pardiso and DFeast_scsrgv) and I can't find out which DLLs to distribute. I want to make sure that my application will work with all the different client's hardware.

Regards,
Peter

[eos_pengwern]

Having successfully got static linking to work on Linux, I actually then went back to my Windows build and got static linking to work there as well. Two things were necessary: first of all I put the static linking versions of the libraries onto the "Additional dependencies" line of my Linker/Input dialog within Visual Studio's project properties:

mkl_blas95_lp64.lib mkl_lapack95_lp64.lib mkl_intel_lp64.lib mkl_core.lib mkl_sequential.lib (in my case, but the Link Line Advisor may tell you something different for the libraries you're using).

Secondly (and somewhat counter-intuitively), in the Fortran/Libraries dialog I set the 'Runtime Library' to 'Multithread DLL' (or 'Debug Multithread DLL for the debug build) and set 'Use Math Kernel Library' to 'No'. This is because, even with static linking of the MKL, it is desirable to use dynamic linking for the other Intel Fortran runtime libraries, but by default if you set 'Use Math Kernel Library' to 'Yes' in this dialog then it ignores whatever you put in your Linker dialog and links the dynamic versions of the MKL anyway! Hence in effect what you need to do is to override the automatic linking of the MKL and specify it manually instead, which is what the above steps accomplish.

When I look at my compiled DLL, the size of it (~40MB) indicates that pretty much the whole of the MKL has been linked into it anyway, since there are so many cross-dependencies that it is difficult for the linker to separate things out. Nonetheless, it is still simpler to distribute this one very large file than many small ones which added up to the same total size.

[schulzey]

Thanks for that comprehensive reply. I'll give it a go and see what happens. I was actually trying to minimize the overall size of the MKL DLLs I have to distribute but it sounds like there won't be much saving.

Peter