Intel® oneAPI Math Kernel Library
Ask questions and share information with other developers who use Intel® Math Kernel Library.
7185 Discussions

Scipy 15.1 not linked properly when compiled against MKL

Jonah_M_
Beginner
‎09-08-2015 09:11 PM
1,783 Views

Hello,

I have been trying to compile numpy and scipy against intel MKL. I have been following the following guide to the letter:

https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl

I am using the versions of numpy and scipy tested with the guide, numpy 1.9.2 and scpy 0.15.1. Numpy installs exactly as I would expect it to and performs well. However, when I run the standard scipy tests, i.e.,

import scipy
scipy.test()

I encounter a large number of errors related to missing symbols, such as 

ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset

Has anyone else experienced this or does anyone have any insight? Thanks in advance for your help.

My LD_LIBRARY_PATH is:

/opt/intel/composer_xe_2015.3.187/compiler/lib/intel64:/opt/intel/composer_xe_2015.3.187/compiler/lib/intel64/:/lib:/usr/local/plc-2.0/lib:/lib64:/opt/intel/composer_xe_2015.3.187/ipp/../compiler/lib/intel64/:/opt/intel/composer_xe_2015.3.187/ipp/../compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/ipp/../compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/ipp/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/tbb/lib/intel64/gcc4.4:/opt/intel/debugger_2016/libipt/intel64/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/../tbb/lib/intel64_lin/gcc4.4:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/../compiler/lib/intel64_lin::/usr/local/cuda/lib64:/usr/local/cuda/lib

The full results of the scipy tests are:

======================================================================
ERROR: Failure: ImportError (scipy/fftpack/_fftpack.so: undefined symbol: __intel_avx_rep_memcpy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/fftpack/__init__.py", line 94, in <module>
    from .basic import *
  File "scipy/fftpack/basic.py", line 12, in <module>
    from . import _fftpack
ImportError: scipy/fftpack/_fftpack.so: undefined symbol: __intel_avx_rep_memcpy

======================================================================
ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/integrate/__init__.py", line 51, in <module>
    from .quadrature import *
  File "scipy/integrate/quadrature.py", line 6, in <module>
    from scipy.special.orthogonal import p_roots
  File "scipy/special/__init__.py", line 586, in <module>
    from ._ufuncs import *
ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/interpolate/__init__.py", line 160, in <module>
    from .interpolate import *
  File "scipy/interpolate/interpolate.py", line 16, in <module>
    import scipy.special as spec
  File "scipy/special/__init__.py", line 586, in <module>
    from ._ufuncs import *
ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/io/__init__.py", line 85, in <module>
    from .matlab import loadmat, savemat, whosmat, byteordercodes
  File "scipy/io/matlab/__init__.py", line 13, in <module>
    from .mio import loadmat, savemat, whosmat
  File "scipy/io/matlab/mio.py", line 12, in <module>
    from .miobase import get_matfile_version, docfiller
  File "scipy/io/matlab/miobase.py", line 22, in <module>
    from scipy.misc import doccer
  File "scipy/misc/__init__.py", line 47, in <module>
    from scipy.special import comb, factorial, factorial2, factorialk
  File "scipy/special/__init__.py", line 586, in <module>
    from ._ufuncs import *
ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/linalg/_interpolative.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/linalg/tests/test_interpolative.py", line 29, in <module>
    import scipy.linalg.interpolative as pymatrixid
  File "scipy/linalg/interpolative.py", line 386, in <module>
    import scipy.linalg._interpolative_backend as backend
  File "scipy/linalg/_interpolative_backend.py", line 34, in <module>
    import scipy.linalg._interpolative as _id
ImportError: scipy/linalg/_interpolative.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/linalg/tests/test_matfuncs.py", line 27, in <module>
    from scipy.linalg import _matfuncs_inv_ssq
  File "scipy/linalg/_matfuncs_inv_ssq.py", line 15, in <module>
    from scipy.sparse.linalg.interface import LinearOperator
  File "scipy/sparse/linalg/__init__.py", line 109, in <module>
    from .isolve import *
  File "scipy/sparse/linalg/isolve/__init__.py", line 6, in <module>
    from .iterative import *
  File "scipy/sparse/linalg/isolve/iterative.py", line 7, in <module>
    from . import _iterative
ImportError: scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (cannot import name doccer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/linalg/tests/test_special_matrices.py", line 12, in <module>
    from scipy.misc import comb
  File "scipy/misc/__init__.py", line 44, in <module>
    from . import doccer
ImportError: cannot import name doccer

======================================================================
ERROR: Failure: ImportError (cannot import name doccer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/misc/__init__.py", line 44, in <module>
    from . import doccer
ImportError: cannot import name doccer

======================================================================
ERROR: Failure: ImportError (cannot import name doccer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/ndimage/__init__.py", line 172, in <module>
    from .filters import *
  File "scipy/ndimage/filters.py", line 37, in <module>
    from scipy.misc import doccer
  File "scipy/misc/__init__.py", line 44, in <module>
    from . import doccer
ImportError: cannot import name doccer

======================================================================
ERROR: Failure: ImportError (scipy/odr/__odrpack.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/odr/__init__.py", line 114, in <module>
    from .odrpack import *
  File "scipy/odr/odrpack.py", line 42, in <module>
    from scipy.odr import __odrpack
ImportError: scipy/odr/__odrpack.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/optimize/_lbfgsb.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/optimize/__init__.py", line 176, in <module>
    from ._minimize import *
  File "scipy/optimize/_minimize.py", line 31, in <module>
    from .lbfgsb import _minimize_lbfgsb
  File "scipy/optimize/lbfgsb.py", line 40, in <module>
    from . import _lbfgsb
ImportError: scipy/optimize/_lbfgsb.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/signal/__init__.py", line 264, in <module>
    from .bsplines import *
  File "scipy/signal/bsplines.py", line 4, in <module>
    import scipy.special
  File "scipy/special/__init__.py", line 586, in <module>
    from ._ufuncs import *
ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/sparse/linalg/__init__.py", line 109, in <module>
    from .isolve import *
  File "scipy/sparse/linalg/isolve/__init__.py", line 6, in <module>
    from .iterative import *
  File "scipy/sparse/linalg/isolve/iterative.py", line 7, in <module>
    from . import _iterative
ImportError: scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/sparse/tests/test_base.py", line 44, in <module>
    from scipy.sparse.linalg import splu, expm, inv
  File "scipy/sparse/linalg/__init__.py", line 109, in <module>
    from .isolve import *
  File "scipy/sparse/linalg/isolve/__init__.py", line 6, in <module>
    from .iterative import *
  File "scipy/sparse/linalg/isolve/iterative.py", line 7, in <module>
    from . import _iterative
ImportError: scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/special/__init__.py", line 586, in <module>
    from ._ufuncs import *
ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset

======================================================================
ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "scipy/stats/__init__.py", line 338, in <module>
    from .stats import *
  File "scipy/stats/stats.py", line 184, in <module>
    import scipy.special as special
  File "scipy/special/__init__.py", line 586, in <module>
    from ._ufuncs import *
ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset

======================================================================
FAIL: test_qhull.TestDelaunay.test_incremental('pathological-1-chunk-1',)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/spatial/tests/test_qhull.py", line 507, in check
    np.arange(points.shape[0]))
  File "/usr/local/numpy-1.9.2/lib/python2.7/site-packages/numpy/testing/utils.py", line 739, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/usr/local/numpy-1.9.2/lib/python2.7/site-packages/numpy/testing/utils.py", line 620, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

(shapes (79,), (81,) mismatch)
 x: array([ 0,  2,  3,  4,  5,  6,  7,  8,  9, 11, 12, 13, 14, 15, 16, 17, 18,
       19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
       36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,...
 y: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,...

======================================================================
FAIL: test_qhull.TestUtilities.test_degenerate_barycentric_transforms
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/local/numpy-1.9.2/lib/python2.7/site-packages/numpy/testing/decorators.py", line 146, in skipper_func
    return f(*args, **kwargs)
  File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/spatial/tests/test_qhull.py", line 296, in test_degenerate_barycentric_transforms
    assert_(bad_count < 20, bad_count)
  File "/usr/local/numpy-1.9.2/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_
    raise AssertionError(smsg)
AssertionError: 26

----------------------------------------------------------------------
Ran 1760 tests in 13.580s

 

0 Kudos

Link Copied

8 Replies
Ying_H_Intel
Moderator
‎09-10-2015 12:10 AM
1,783 Views

Hi Jonah,

What are your c and fortran compiler you are using?  The symbol __intel_avx_rep_memset look like from  one of Intel library, but i haven't found it so far.    Anyway, could you please try 

in the site.cfg file, you must link with mkl_rt only and any other linking method will not work.

Export the compiler flags as:

view sourceprint?
1 $export CFLAGS="-fopenmp -m64 -mtune=native -O3 -Wl,--no-as-needed"
2 $export CXXFLAGS="-fopenmp -m64 -mtune=native -O3 -Wl,--no-as-needed"
3 $export LDFLAGS="-ldl -lm"
4 $export FFLAGS="-fopenmp -m64 -mtune=native -O3 -Wl, --no-as-needed"

Best Regards,

Ying

0 Kudos
Copy link
Yuan_C_Intel
Employee
‎09-10-2015 01:45 AM
1,783 Views

Hi, Jonah

Did try to build Numpy/Scipy with 16.0 Intel compiler, but run with 15.0 libraries?

I noticed both 16.0 and 15.0 update 3 are installed on your system.

The "__intel_avx_rep_memset" is available in 16.0 libirc.so, but not in 15.0.

Thanks

Yolanda

0 Kudos
Copy link
Jonah_M_
Beginner
‎09-10-2015 12:25 PM
1,783 Views

Hi Ying,

Thanks for the reply. I have tried exporting the values you suggest. Now scipy fails in teh build stage with the following error:

creating build/lib.linux-x86_64-2.7/scipy/fftpack
/opt/intel/compilers_and_libraries_2016.0.109/linux/bin/intel64/ifort -shared -ldl -lm build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/_fftpackmodule.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/drfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zrfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zfftnd.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/src/dct.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/src/dst.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/fortranobject.o -Lbuild/temp.linux-x86_64-2.7 -ldfftpack -lfftpack -o build/lib.linux-x86_64-2.7/scipy/fftpack/_fftpack.so
ifort: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
ld: build/temp.linux-x86_64-2.7/libdfftpack.a(zffti1.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
build/temp.linux-x86_64-2.7/libdfftpack.a: error adding symbols: Bad value
error: Command "/opt/intel/compilers_and_libraries_2016.0.109/linux/bin/intel64/ifort -shared -ldl -lm build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/_fftpackmodule.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/drfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zrfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zfftnd.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/src/dct.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/src/dst.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/fortranobject.o -Lbuild/temp.linux-x86_64-2.7 -ldfftpack -lfftpack -o build/lib.linux-x86_64-2.7/scipy/fftpack/_fftpack.so" failed with exit status 1ifort: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance
ld: build/temp.linux-x86_64-2.7/libdfftpack.a(zffti1.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
build/temp.linux-x86_64-2.7/libdfftpack.a: error adding symbols: Bad value

Thank you for your help!

Best,

Jonah

0 Kudos
Copy link
Jonah_M_
Beginner
‎09-10-2015 12:27 PM
1,783 Views

Hi Yolanda,

Thank you for your reply! Yes, this is possible. To test this, I reset my LD_LIBRARY_PATH to exclude the 2015 compilers before and after building. However, when I do this, the code segfaults when loading fftw, all else remaining the same. I'm not sure how to interpret that.

Best,

Jonah

0 Kudos
Copy link
Ying_H_Intel
Moderator
‎09-10-2015 08:26 PM
1,783 Views

Hi Jonah, 

Let us clarify, 

1) when you try 2015 compiler and MKL + the original setting, what is the error?

 

2)  when you try 2015 compiler and MKL + the  setting in post #2, what is the error?

 

Regarding the last error itself,  relocation R_X86_64_32S against `.rodata' can not be used itself, it seems ask recompile -fPIC

so may you add 

$export FFLAGS="-fopenmp -m64 -mtune=native -O3 -Wl, --no-as-needed -fPIC"

and let us know the result. 

Regards,

Ying 

0 Kudos
Copy link
Jonah_M_
Beginner
‎09-11-2015 08:57 PM
1,783 Views

Hi Ying, Yolanda,

Thank you for your help. It seems the error may have only been the library path. I have did the following to reset my path:

export LD_LIBRARY_PATH=
source /opt/intel/bin/compilervars.sh intel64

resulting in the following library path:

/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/ipp/../compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/ipp/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/tbb/lib/intel64/gcc4.4:/opt/intel/debugger_2016/libipt/intel64/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/../compiler/lib/intel64_lin:

Then I compiled as before and ran the tests. Numpy (of course) passed all tests. Scipy had 13 errors, which seemed to be genuine errors:

======================================================================
FAIL: test_definition (test_basic.TestIfftnSingle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/fftpack/tests/test_basic.py", line 657, in test_definition
    assert_array_almost_equal_nulp(y,direct_idftn(x),self.maxnlp)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 1358, in assert_array_almost_equal_nulp
    raise AssertionError(msg)
AssertionError: X and Y are not equal to 3500 ULP

======================================================================
FAIL: test_lorentz (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 293, in test_lorentz
    3.7798193600109009e+00]),
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal
    precision=decimal)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([  1.000000e+03,   1.000000e-01,   3.800000e+00])
 y: array([  1.430678e+03,   1.339051e-01,   3.779819e+00])

======================================================================
FAIL: test_multi (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 190, in test_multi
    0.5101147161764654, 0.5173902330489161]),
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal
    precision=decimal)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([ 4. ,  2. ,  7. ,  0.4,  0.5])
 y: array([ 4.379988,  2.433306,  8.002885,  0.510115,  0.51739 ])

======================================================================
FAIL: test_pearson (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 236, in test_pearson
    np.array([5.4767400299231674, -0.4796082367610305]),
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal
    precision=decimal)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([ 1.,  1.])
 y: array([ 5.47674 , -0.479608])

======================================================================
FAIL: test_qhull.TestDelaunay.test_incremental('pathological-1-chunk-1',)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/spatial/tests/test_qhull.py", line 507, in check
    np.arange(points.shape[0]))
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 739, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 620, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

(shapes (79,), (81,) mismatch)
 x: array([ 0,  2,  3,  4,  5,  6,  7,  8,  9, 11, 12, 13, 14, 15, 16, 17, 18,
       19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
       36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,...
 y: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,...

======================================================================
FAIL: test_qhull.TestUtilities.test_degenerate_barycentric_transforms
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/decorators.py", line 146, in skipper_func
    return f(*args, **kwargs)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/spatial/tests/test_qhull.py", line 296, in test_degenerate_barycentric_transforms
    assert_(bad_count < 20, bad_count)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_
    raise AssertionError(smsg)
AssertionError: 26

======================================================================
FAIL: test_bdtrc (test_basic.TestCephes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_basic.py", line 118, in test_bdtrc
    assert_equal(cephes.bdtrc(1,3,0.5),0.5)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 334, in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal:
 ACTUAL: 0.50000000000000011
 DESIRED: 0.5

======================================================================
FAIL: test_bdtri (test_basic.TestCephes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_basic.py", line 115, in test_bdtri
    assert_equal(cephes.bdtri(1,3,0.5),0.5)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 334, in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal:
 ACTUAL: 0.49999999999999994
 DESIRED: 0.5

======================================================================
FAIL: test_cosm1 (test_basic.TestCephes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_basic.py", line 214, in test_cosm1
    assert_equal(cephes.cosm1(0),0.0)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 326, in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal:
 ACTUAL: -0.0
 DESIRED: 0.0

======================================================================
FAIL: test_data.test_boost(<Data for eval_legendre_ld: legendre_p_ipp-legendre_p>,)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_data.py", line 481, in _test_factory
    test.check(dtype=dtype)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/_testutils.py", line 292, in check
    assert_(False, "\n".join(msg))
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_
    raise AssertionError(smsg)
AssertionError: 
Max |adiff|: 7.49401e-16
Max |rdiff|: 7.92476e-14
Bad results (1 out of 140) for the following points (in output 0):
                          18.0             -0.8049191832542419 =>          -0.009106206044325216 !=          -0.009106206044324494  (rdiff          7.924759910919454e-14)

======================================================================
FAIL: test_data.test_boost(<Data for gammaincc: igamma_int_data_ipp-igamma_int_data>,)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_data.py", line 481, in _test_factory
    test.check(dtype=dtype)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/_testutils.py", line 292, in check
    assert_(False, "\n".join(msg))
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_
    raise AssertionError(smsg)
AssertionError: 
Max |adiff|: 1.21569e-14
Max |rdiff|: 4.27465e-14
Bad results (1 out of 140) for the following points (in output 0):
                          36.5                            73.0 =>          8.570090512692315e-07 !=          8.570090512692681e-07  (rdiff          4.274654382528594e-14)

======================================================================
FAIL: test_data.test_boost(<Data for gdtrc_: igamma_int_data_ipp-igamma_int_data>,)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_data.py", line 481, in _test_factory
    test.check(dtype=dtype)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/_testutils.py", line 292, in check
    assert_(False, "\n".join(msg))
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_
    raise AssertionError(smsg)
AssertionError: 
Max |adiff|: 1.21569e-14
Max |rdiff|: 4.27465e-14
Bad results (1 out of 140) for the following points (in output 0):
                          36.5                            73.0 =>          8.570090512692315e-07 !=          8.570090512692681e-07  (rdiff          4.274654382528594e-14)

======================================================================
FAIL: test_orthogonal.test_js_roots
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_orthogonal.py", line 371, in test_js_roots
    verify_gauss_quad(roots(68.9, 2.25), evalf(68.9, 2.25), 5, atol=2e-14)
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_orthogonal.py", line 283, in verify_gauss_quad
    assert_allclose(vv, np.eye(N), rtol, atol)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 1297, in assert_allclose
    verbose=verbose, header=header)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-15, atol=2e-14

(mismatch 100.0%)
 x: array([[  1.000000e+00,  -2.932083e-14,   1.628550e-14,  -5.973276e-15,
         -1.360751e-15],
       [ -2.932083e-14,   1.000000e+00,  -1.985946e-14,   1.119072e-14,...
 y: array([[ 1.,  0.,  0.,  0.,  0.],
       [ 0.,  1.,  0.,  0.,  0.],
       [ 0.,  0.,  1.,  0.,  0.],...

----------------------------------------------------------------------
Ran 17005 tests in 117.640s

FAILED (KNOWNFAIL=97, SKIP=1181, failures=13)
<nose.result.TextTestResult run=17005 errors=0 failures=13>

I tried again changed the "-O3" flag to "-O1" in both in both "numpy/distutils/intelcompiler.py" and "numpy/distutils/fcompiler/intel.py". And now both libraries pass all tests.

The error in post 4 was based on using 2015 compiler and MKL + the  setting in post #2.

If I also add the fPIC command and use -O3, the code compiles and scipy.test9) returns only three errors:

======================================================================
FAIL: test_lorentz (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 293, in test_lorentz
    3.7798193600109009e+00]),
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal
    precision=decimal)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([  1.000000e+03,   1.000000e-01,   3.800000e+00])
 y: array([  1.430678e+03,   1.339051e-01,   3.779819e+00])

======================================================================
FAIL: test_multi (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 190, in test_multi
    0.5101147161764654, 0.5173902330489161]),
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal
    precision=decimal)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([ 4. ,  2. ,  7. ,  0.4,  0.5])
 y: array([ 4.379988,  2.433306,  8.002885,  0.510115,  0.51739 ])

======================================================================
FAIL: test_pearson (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 236, in test_pearson
    np.array([5.4767400299231674, -0.4796082367610305]),
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal
    precision=decimal)
  File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([ 1.,  1.])
 y: array([ 5.47674 , -0.479608])

----------------------------------------------------------------------
Ran 17005 tests in 114.097s

FAILED (KNOWNFAIL=97, SKIP=1181, failures=3)
<nose.result.TextTestResult run=17005 errors=0 failures=3>

It looks like I can use a lower optimization setting with this configuration to get everything working. Thank you for your help!

Best,

Jonah

0 Kudos
Copy link
Ying_H_Intel
Moderator
‎09-13-2015 06:20 PM
1,783 Views

Hi Jonah, 

Thanks a lot for the report and verification.   

It seems the O3 optimization will cause the numerical error in Scipy 15.1 as earlier version

Appendix 1:

Known issues:

 When the -O3 or -O2(default) compiler flags (more aggressive compiler optimization flags) used for the ifort, one of the scipy tests may fail and it is a known corner case, to avoid this, as a workaround you can use -O1.

 would you please help to submit the issue to premier.intel.com to intel fortran or C compiler  or their forum: 

https://software.intel.com/en-us/forums/intel-fortran-compiler-for-linux-and-mac-os-x. ;

Thanks

Ying

0 Kudos
Copy link
Jonah_M_
Beginner
‎09-15-2015 12:38 PM
1,783 Views

Hi Ying,

Okay, I've submitted the bug to the fortran compiler forum. The link is here:

https://software.intel.com/en-us/forums/intel-fortran-compiler-for-linux-and-mac-os-x/topic/593518

Thank you for your help.

Best,

Jonah

0 Kudos
Copy link
Reply

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.