SDL automatically load three or four library???

yan_c_ · ‎04-25-2017

I came to the BBS for the first time, if the post does not conform to the requirements, please let us know, thanks! About this plate, I have read introduction.

PC-Environments: hp computer, CPU——Intel Core i5 4 core，Memory——8G, OS——CentOS 7, use miniconda3(Python3.6) as virtualenv environment, numpy I compiled from source code.

I have installed Intel® System Studio Ultimate Edition 2017, so I have gotten MKL library. In order to make environments clean and tidy, I copy it to /tmp/mkl-nfs/lib directory.
According to Intel-DOC[https://software.intel.com/en-us/node/528514], mkl_rt(SDL) automatically links interface, threading, and computational libraries, but I use sample code to look, something else happened.

$ tail numpy/site.cfg
#[fftw]
#libraries = fftw3
#
# For djbfft, numpy.distutils will look for either djbfft.a or libdjbfft.a . 
#[djbfft]
#include_dirs = /usr/local/djbfft/include
#library_dirs = /usr/local/djbfft/lib
[mkl]
library_dirs = /tmp/mkl-nfs/lib
mkl_libs = mkl_rt

$ python
>>> import numpy as np
>>> np.show_config()
blas_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/tmp/mkl-nfs/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/local/include', '/usr/include', '/usr/local/miniconda3/envs/perinfoMKL1/include']
blas_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/tmp/mkl-nfs/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/local/include', '/usr/include', '/usr/local/miniconda3/envs/perinfoMKL1/include']
lapack_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/tmp/mkl-nfs/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/local/include', '/usr/include', '/usr/local/miniconda3/envs/perinfoMKL1/include']
lapack_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/tmp/mkl-nfs/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/usr/local/include', '/usr/include', '/usr/local/miniconda3/envs/perinfoMKL1/include']
>>> np.__version__
'1.13.0.dev0+4408f74'
>>>

Below followings is test.py code：

#!/usr/local/miniconda3/envs/perinfoMKL0/bin/python
# -*- coding: utf-8 -*-

import numpy as np
import time

start_time = time.time()
a = 10 ** 4
A = np.random.random((a, a))
B = np.random.random((a, a))
C = A.dot(B)
print("Time: ", time.time() - start_time)

And then, I do execute it code use command:

$ LD_DEBUG=files ./test.py 2> files.log
$ grep "libmkl_rt.so" files.log 
     24311:	file=libmkl_rt.so [0];  needed by /home/yancy/.conda/envs/perinfoMKL/lib/python3.6/site-packages/numpy-1.13.0.dev0+1a9f43f-py3.6-linux-x86_64.egg/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so [0]                         
     24311:	file=libmkl_rt.so [0];  generating link map
     24311:	calling init: /tmp/mkl-nfs/lib/libmkl_rt.so
     24311:	file=/tmp/mkl-nfs/lib/libmkl_core.so [0];  dynamically loaded by /tmp/mkl-nfs/lib/libmkl_rt.so [0] <==========
     24311:	file=/tmp/mkl-nfs/lib/libiomp5.so [0];  dynamically loaded by /tmp/mkl-nfs/lib/libmkl_rt.so [0] <==========
     24311:	file=/tmp/mkl-nfs/lib/libmkl_intel_thread.so [0];  dynamically loaded by /tmp/mkl-nfs/lib/libmkl_rt.so [0] <==========
     24311:	file=/tmp/mkl-nfs/lib/libmkl_intel_lp64.so [0];  dynamically loaded by /tmp/mkl-nfs/lib/libmkl_rt.so [0] <==========
     24311:	calling fini: /tmp/mkl-nfs/lib/libmkl_rt.so [0]
$ grep "libmkl_avx2.so" files.log 
     24311:	file=/tmp/mkl-nfs/lib/libmkl_avx2.so [0];  dynamically loaded by /tmp/mkl-nfs/lib/libmkl_core.so [0]
     24311:	file=/tmp/mkl-nfs/lib/libmkl_avx2.so [0];  generating link map
     24311:	file=/tmp/mkl-nfs/lib/libmkl_intel_thread.so [0];  needed by /tmp/mkl-nfs/lib/libmkl_avx2.so [0] (relocation dependency)
     24311:	file=/tmp/mkl-nfs/lib/libmkl_core.so [0];  needed by /tmp/mkl-nfs/lib/libmkl_avx2.so [0] (relocation dependency)
     24311:	calling init: /tmp/mkl-nfs/lib/libmkl_avx2.so
     24311:	opening file=/tmp/mkl-nfs/lib/libmkl_avx2.so [0]; direct_opencount=1
     24311:	calling fini: /tmp/mkl-nfs/lib/libmkl_avx2.so [0]

Now, we can see interface, threading, and computational run-time libraries (RTL) libraries all loaded by mkl_rt. So, I am confused it, this output is different from Intel-DOC. I see relocation dependency output, but I am hard to undertstand. Someone can explain it, I am grateful to you. Again thanks.

Ying_H_Intel · ‎05-07-2017

Hi Yan,

Thank you a lot for the sharing in https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/733207.

Here the step is expected. :

The Single Dynamic Library (SDL) automatically links interface, threading, and computational libraries and thus simplifies linking

file=/tmp/mkl-nfs/lib/libmkl_core.so [0]; dynamically loaded by /tmp/mkl-nfs/lib/libmkl_rt.so [0] <==========

`07`	`24311:` `file=/tmp/mkl-nfs/lib/libiomp5.so [0]; dynamically loaded by /tmp/mkl-nfs/lib/libmkl_rt.so [0] <==========`

`08`	`24311:` `file=/tmp/mkl-nfs/lib/libmkl_intel_thread.so [0]; dynamically loaded by /tmp/mkl-nfs/lib/libmkl_rt.so [0] <==========`

`09`	`24311:` `file=/tmp/mkl-nfs/lib/libmkl_intel_lp64.so [0]; dynamically loaded by /tmp/mkl-nfs/lib/libmkl_rt.so`

and second step: libmkl_avx2.so (processor-optimized) libraries were loaded by libmkl_core.so . which is expected.

The last message seems caused the confusion. what is exactly of means "libmkl_intel_thread.so needed by libmkl_avx2.so" relocation dependency ? I will check with internal team and see if any more ideas.

Thanks

Ying

file=/tmp/mkl-nfs/lib/libmkl_intel_thread.so [0]; needed by /tmp/mkl-nfs/lib/libmkl_avx2.so [0] (relocation dependency)

`15`	`24311:` `file=/tmp/mkl-nfs/lib/libmkl_core.so [0]; needed by /tmp/mkl-nfs/lib/libmkl_avx2.so [0] (relocation`

yan_c_ · ‎05-08-2017

Hi, Ying:

Thanks your sincere helping. Ok, I will be look forward to your reply. If you want to some data and test-code and even some things, please let me know.

Best, yancy.