- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I have been trying to compile numpy and scipy against intel MKL. I have been following the following guide to the letter:
https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl
I am using the versions of numpy and scipy tested with the guide, numpy 1.9.2 and scpy 0.15.1. Numpy installs exactly as I would expect it to and performs well. However, when I run the standard scipy tests, i.e.,
import scipy scipy.test()
I encounter a large number of errors related to missing symbols, such as
ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset
Has anyone else experienced this or does anyone have any insight? Thanks in advance for your help.
My LD_LIBRARY_PATH is:
/opt/intel/composer_xe_2015.3.187/compiler/lib/intel64:/opt/intel/composer_xe_2015.3.187/compiler/lib/intel64/:/lib:/usr/local/plc-2.0/lib:/lib64:/opt/intel/composer_xe_2015.3.187/ipp/../compiler/lib/intel64/:/opt/intel/composer_xe_2015.3.187/ipp/../compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/ipp/../compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/ipp/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/tbb/lib/intel64/gcc4.4:/opt/intel/debugger_2016/libipt/intel64/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/../tbb/lib/intel64_lin/gcc4.4:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/../compiler/lib/intel64_lin::/usr/local/cuda/lib64:/usr/local/cuda/lib
The full results of the scipy tests are:
====================================================================== ERROR: Failure: ImportError (scipy/fftpack/_fftpack.so: undefined symbol: __intel_avx_rep_memcpy) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/fftpack/__init__.py", line 94, in <module> from .basic import * File "scipy/fftpack/basic.py", line 12, in <module> from . import _fftpack ImportError: scipy/fftpack/_fftpack.so: undefined symbol: __intel_avx_rep_memcpy ====================================================================== ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/integrate/__init__.py", line 51, in <module> from .quadrature import * File "scipy/integrate/quadrature.py", line 6, in <module> from scipy.special.orthogonal import p_roots File "scipy/special/__init__.py", line 586, in <module> from ._ufuncs import * ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/interpolate/__init__.py", line 160, in <module> from .interpolate import * File "scipy/interpolate/interpolate.py", line 16, in <module> import scipy.special as spec File "scipy/special/__init__.py", line 586, in <module> from ._ufuncs import * ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/io/__init__.py", line 85, in <module> from .matlab import loadmat, savemat, whosmat, byteordercodes File "scipy/io/matlab/__init__.py", line 13, in <module> from .mio import loadmat, savemat, whosmat File "scipy/io/matlab/mio.py", line 12, in <module> from .miobase import get_matfile_version, docfiller File "scipy/io/matlab/miobase.py", line 22, in <module> from scipy.misc import doccer File "scipy/misc/__init__.py", line 47, in <module> from scipy.special import comb, factorial, factorial2, factorialk File "scipy/special/__init__.py", line 586, in <module> from ._ufuncs import * ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/linalg/_interpolative.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/linalg/tests/test_interpolative.py", line 29, in <module> import scipy.linalg.interpolative as pymatrixid File "scipy/linalg/interpolative.py", line 386, in <module> import scipy.linalg._interpolative_backend as backend File "scipy/linalg/_interpolative_backend.py", line 34, in <module> import scipy.linalg._interpolative as _id ImportError: scipy/linalg/_interpolative.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/linalg/tests/test_matfuncs.py", line 27, in <module> from scipy.linalg import _matfuncs_inv_ssq File "scipy/linalg/_matfuncs_inv_ssq.py", line 15, in <module> from scipy.sparse.linalg.interface import LinearOperator File "scipy/sparse/linalg/__init__.py", line 109, in <module> from .isolve import * File "scipy/sparse/linalg/isolve/__init__.py", line 6, in <module> from .iterative import * File "scipy/sparse/linalg/isolve/iterative.py", line 7, in <module> from . import _iterative ImportError: scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (cannot import name doccer) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/linalg/tests/test_special_matrices.py", line 12, in <module> from scipy.misc import comb File "scipy/misc/__init__.py", line 44, in <module> from . import doccer ImportError: cannot import name doccer ====================================================================== ERROR: Failure: ImportError (cannot import name doccer) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/misc/__init__.py", line 44, in <module> from . import doccer ImportError: cannot import name doccer ====================================================================== ERROR: Failure: ImportError (cannot import name doccer) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/ndimage/__init__.py", line 172, in <module> from .filters import * File "scipy/ndimage/filters.py", line 37, in <module> from scipy.misc import doccer File "scipy/misc/__init__.py", line 44, in <module> from . import doccer ImportError: cannot import name doccer ====================================================================== ERROR: Failure: ImportError (scipy/odr/__odrpack.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/odr/__init__.py", line 114, in <module> from .odrpack import * File "scipy/odr/odrpack.py", line 42, in <module> from scipy.odr import __odrpack ImportError: scipy/odr/__odrpack.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/optimize/_lbfgsb.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/optimize/__init__.py", line 176, in <module> from ._minimize import * File "scipy/optimize/_minimize.py", line 31, in <module> from .lbfgsb import _minimize_lbfgsb File "scipy/optimize/lbfgsb.py", line 40, in <module> from . import _lbfgsb ImportError: scipy/optimize/_lbfgsb.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/signal/__init__.py", line 264, in <module> from .bsplines import * File "scipy/signal/bsplines.py", line 4, in <module> import scipy.special File "scipy/special/__init__.py", line 586, in <module> from ._ufuncs import * ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/sparse/linalg/__init__.py", line 109, in <module> from .isolve import * File "scipy/sparse/linalg/isolve/__init__.py", line 6, in <module> from .iterative import * File "scipy/sparse/linalg/isolve/iterative.py", line 7, in <module> from . import _iterative ImportError: scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/sparse/tests/test_base.py", line 44, in <module> from scipy.sparse.linalg import splu, expm, inv File "scipy/sparse/linalg/__init__.py", line 109, in <module> from .isolve import * File "scipy/sparse/linalg/isolve/__init__.py", line 6, in <module> from .iterative import * File "scipy/sparse/linalg/isolve/iterative.py", line 7, in <module> from . import _iterative ImportError: scipy/sparse/linalg/isolve/_iterative.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/special/__init__.py", line 586, in <module> from ._ufuncs import * ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset ====================================================================== ERROR: Failure: ImportError (scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 411, in loadTestsFromName addr.filename, addr.module) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "scipy/stats/__init__.py", line 338, in <module> from .stats import * File "scipy/stats/stats.py", line 184, in <module> import scipy.special as special File "scipy/special/__init__.py", line 586, in <module> from ._ufuncs import * ImportError: scipy/special/_ufuncs.so: undefined symbol: __intel_avx_rep_memset ====================================================================== FAIL: test_qhull.TestDelaunay.test_incremental('pathological-1-chunk-1',) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/spatial/tests/test_qhull.py", line 507, in check np.arange(points.shape[0])) File "/usr/local/numpy-1.9.2/lib/python2.7/site-packages/numpy/testing/utils.py", line 739, in assert_array_equal verbose=verbose, header='Arrays are not equal') File "/usr/local/numpy-1.9.2/lib/python2.7/site-packages/numpy/testing/utils.py", line 620, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not equal (shapes (79,), (81,) mismatch) x: array([ 0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,... y: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,... ====================================================================== FAIL: test_qhull.TestUtilities.test_degenerate_barycentric_transforms ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/local/numpy-1.9.2/lib/python2.7/site-packages/numpy/testing/decorators.py", line 146, in skipper_func return f(*args, **kwargs) File "/nfs/galaxy/unixhome/jmiller/scipy-to-copy/lib/python2.7/site-packages/scipy/spatial/tests/test_qhull.py", line 296, in test_degenerate_barycentric_transforms assert_(bad_count < 20, bad_count) File "/usr/local/numpy-1.9.2/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_ raise AssertionError(smsg) AssertionError: 26 ---------------------------------------------------------------------- Ran 1760 tests in 13.580s
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Jonah,
What are your c and fortran compiler you are using? The symbol __intel_avx_rep_memset look like from one of Intel library, but i haven't found it so far. Anyway, could you please try
in the site.cfg file, you must link with mkl_rt only and any other linking method will not work.
Export the compiler flags as:
1 |
$ export CFLAGS= "-fopenmp -m64 -mtune=native -O3 -Wl,--no-as-needed" |
2 |
$ export CXXFLAGS= "-fopenmp -m64 -mtune=native -O3 -Wl,--no-as-needed" |
3 |
$ export LDFLAGS= "-ldl -lm" |
4 |
$ export FFLAGS= "-fopenmp -m64 -mtune=native -O3 -Wl, --no-as-needed" |
Best Regards,
Ying
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, Jonah
Did try to build Numpy/Scipy with 16.0 Intel compiler, but run with 15.0 libraries?
I noticed both 16.0 and 15.0 update 3 are installed on your system.
The "__intel_avx_rep_memset" is available in 16.0 libirc.so, but not in 15.0.
Thanks
Yolanda
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ying,
Thanks for the reply. I have tried exporting the values you suggest. Now scipy fails in teh build stage with the following error:
creating build/lib.linux-x86_64-2.7/scipy/fftpack /opt/intel/compilers_and_libraries_2016.0.109/linux/bin/intel64/ifort -shared -ldl -lm build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/_fftpackmodule.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/drfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zrfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zfftnd.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/src/dct.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/src/dst.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/fortranobject.o -Lbuild/temp.linux-x86_64-2.7 -ldfftpack -lfftpack -o build/lib.linux-x86_64-2.7/scipy/fftpack/_fftpack.so ifort: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance ld: build/temp.linux-x86_64-2.7/libdfftpack.a(zffti1.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC build/temp.linux-x86_64-2.7/libdfftpack.a: error adding symbols: Bad value error: Command "/opt/intel/compilers_and_libraries_2016.0.109/linux/bin/intel64/ifort -shared -ldl -lm build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/_fftpackmodule.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/drfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zrfft.o build/temp.linux-x86_64-2.7/scipy/fftpack/src/zfftnd.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/src/dct.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/scipy/fftpack/src/dst.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/fortranobject.o -Lbuild/temp.linux-x86_64-2.7 -ldfftpack -lfftpack -o build/lib.linux-x86_64-2.7/scipy/fftpack/_fftpack.so" failed with exit status 1ifort: warning #10315: specifying -lm before files may supersede the Intel(R) math library and affect performance ld: build/temp.linux-x86_64-2.7/libdfftpack.a(zffti1.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC build/temp.linux-x86_64-2.7/libdfftpack.a: error adding symbols: Bad value
Thank you for your help!
Best,
Jonah
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Yolanda,
Thank you for your reply! Yes, this is possible. To test this, I reset my LD_LIBRARY_PATH to exclude the 2015 compilers before and after building. However, when I do this, the code segfaults when loading fftw, all else remaining the same. I'm not sure how to interpret that.
Best,
Jonah
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Jonah,
Let us clarify,
1) when you try 2015 compiler and MKL + the original setting, what is the error?
2) when you try 2015 compiler and MKL + the setting in post #2, what is the error?
Regarding the last error itself, relocation R_X86_64_32S against `.rodata' can not be used itself, it seems ask recompile -fPIC
so may you add
$export FFLAGS="-fopenmp -m64 -mtune=native -O3 -Wl, --no-as-needed -fPIC"
and let us know the result.
Regards,
Ying
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ying, Yolanda,
Thank you for your help. It seems the error may have only been the library path. I have did the following to reset my path:
export LD_LIBRARY_PATH= source /opt/intel/bin/compilervars.sh intel64
resulting in the following library path:
/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/ipp/../compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/ipp/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64:/opt/intel/compilers_and_libraries_2016.0.109/linux/tbb/lib/intel64/gcc4.4:/opt/intel/debugger_2016/libipt/intel64/lib:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2016.0.109/linux/daal/../compiler/lib/intel64_lin:
Then I compiled as before and ran the tests. Numpy (of course) passed all tests. Scipy had 13 errors, which seemed to be genuine errors:
====================================================================== FAIL: test_definition (test_basic.TestIfftnSingle) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/fftpack/tests/test_basic.py", line 657, in test_definition assert_array_almost_equal_nulp(y,direct_idftn(x),self.maxnlp) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 1358, in assert_array_almost_equal_nulp raise AssertionError(msg) AssertionError: X and Y are not equal to 3500 ULP ====================================================================== FAIL: test_lorentz (test_odr.TestODR) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 293, in test_lorentz 3.7798193600109009e+00]), File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal precision=decimal) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal to 6 decimals (mismatch 100.0%) x: array([ 1.000000e+03, 1.000000e-01, 3.800000e+00]) y: array([ 1.430678e+03, 1.339051e-01, 3.779819e+00]) ====================================================================== FAIL: test_multi (test_odr.TestODR) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 190, in test_multi 0.5101147161764654, 0.5173902330489161]), File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal precision=decimal) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal to 6 decimals (mismatch 100.0%) x: array([ 4. , 2. , 7. , 0.4, 0.5]) y: array([ 4.379988, 2.433306, 8.002885, 0.510115, 0.51739 ]) ====================================================================== FAIL: test_pearson (test_odr.TestODR) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 236, in test_pearson np.array([5.4767400299231674, -0.4796082367610305]), File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal precision=decimal) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal to 6 decimals (mismatch 100.0%) x: array([ 1., 1.]) y: array([ 5.47674 , -0.479608]) ====================================================================== FAIL: test_qhull.TestDelaunay.test_incremental('pathological-1-chunk-1',) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/spatial/tests/test_qhull.py", line 507, in check np.arange(points.shape[0])) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 739, in assert_array_equal verbose=verbose, header='Arrays are not equal') File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 620, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not equal (shapes (79,), (81,) mismatch) x: array([ 0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52,... y: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,... ====================================================================== FAIL: test_qhull.TestUtilities.test_degenerate_barycentric_transforms ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/decorators.py", line 146, in skipper_func return f(*args, **kwargs) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/spatial/tests/test_qhull.py", line 296, in test_degenerate_barycentric_transforms assert_(bad_count < 20, bad_count) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_ raise AssertionError(smsg) AssertionError: 26 ====================================================================== FAIL: test_bdtrc (test_basic.TestCephes) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_basic.py", line 118, in test_bdtrc assert_equal(cephes.bdtrc(1,3,0.5),0.5) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 334, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 0.50000000000000011 DESIRED: 0.5 ====================================================================== FAIL: test_bdtri (test_basic.TestCephes) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_basic.py", line 115, in test_bdtri assert_equal(cephes.bdtri(1,3,0.5),0.5) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 334, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 0.49999999999999994 DESIRED: 0.5 ====================================================================== FAIL: test_cosm1 (test_basic.TestCephes) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_basic.py", line 214, in test_cosm1 assert_equal(cephes.cosm1(0),0.0) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 326, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: -0.0 DESIRED: 0.0 ====================================================================== FAIL: test_data.test_boost(<Data for eval_legendre_ld: legendre_p_ipp-legendre_p>,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_data.py", line 481, in _test_factory test.check(dtype=dtype) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/_testutils.py", line 292, in check assert_(False, "\n".join(msg)) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_ raise AssertionError(smsg) AssertionError: Max |adiff|: 7.49401e-16 Max |rdiff|: 7.92476e-14 Bad results (1 out of 140) for the following points (in output 0): 18.0 -0.8049191832542419 => -0.009106206044325216 != -0.009106206044324494 (rdiff 7.924759910919454e-14) ====================================================================== FAIL: test_data.test_boost(<Data for gammaincc: igamma_int_data_ipp-igamma_int_data>,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_data.py", line 481, in _test_factory test.check(dtype=dtype) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/_testutils.py", line 292, in check assert_(False, "\n".join(msg)) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_ raise AssertionError(smsg) AssertionError: Max |adiff|: 1.21569e-14 Max |rdiff|: 4.27465e-14 Bad results (1 out of 140) for the following points (in output 0): 36.5 73.0 => 8.570090512692315e-07 != 8.570090512692681e-07 (rdiff 4.274654382528594e-14) ====================================================================== FAIL: test_data.test_boost(<Data for gdtrc_: igamma_int_data_ipp-igamma_int_data>,) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_data.py", line 481, in _test_factory test.check(dtype=dtype) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/_testutils.py", line 292, in check assert_(False, "\n".join(msg)) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 53, in assert_ raise AssertionError(smsg) AssertionError: Max |adiff|: 1.21569e-14 Max |rdiff|: 4.27465e-14 Bad results (1 out of 140) for the following points (in output 0): 36.5 73.0 => 8.570090512692315e-07 != 8.570090512692681e-07 (rdiff 4.274654382528594e-14) ====================================================================== FAIL: test_orthogonal.test_js_roots ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_orthogonal.py", line 371, in test_js_roots verify_gauss_quad(roots(68.9, 2.25), evalf(68.9, 2.25), 5, atol=2e-14) File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/special/tests/test_orthogonal.py", line 283, in verify_gauss_quad assert_allclose(vv, np.eye(N), rtol, atol) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 1297, in assert_allclose verbose=verbose, header=header) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-15, atol=2e-14 (mismatch 100.0%) x: array([[ 1.000000e+00, -2.932083e-14, 1.628550e-14, -5.973276e-15, -1.360751e-15], [ -2.932083e-14, 1.000000e+00, -1.985946e-14, 1.119072e-14,... y: array([[ 1., 0., 0., 0., 0.], [ 0., 1., 0., 0., 0.], [ 0., 0., 1., 0., 0.],... ---------------------------------------------------------------------- Ran 17005 tests in 117.640s FAILED (KNOWNFAIL=97, SKIP=1181, failures=13) <nose.result.TextTestResult run=17005 errors=0 failures=13>
I tried again changed the "-O3" flag to "-O1" in both in both "numpy/distutils/intelcompiler.py" and "numpy/distutils/fcompiler/intel.py". And now both libraries pass all tests.
The error in post 4 was based on using 2015 compiler and MKL + the setting in post #2.
If I also add the fPIC command and use -O3, the code compiles and scipy.test9) returns only three errors:
====================================================================== FAIL: test_lorentz (test_odr.TestODR) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 293, in test_lorentz 3.7798193600109009e+00]), File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal precision=decimal) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal to 6 decimals (mismatch 100.0%) x: array([ 1.000000e+03, 1.000000e-01, 3.800000e+00]) y: array([ 1.430678e+03, 1.339051e-01, 3.779819e+00]) ====================================================================== FAIL: test_multi (test_odr.TestODR) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 190, in test_multi 0.5101147161764654, 0.5173902330489161]), File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal precision=decimal) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal to 6 decimals (mismatch 100.0%) x: array([ 4. , 2. , 7. , 0.4, 0.5]) y: array([ 4.379988, 2.433306, 8.002885, 0.510115, 0.51739 ]) ====================================================================== FAIL: test_pearson (test_odr.TestODR) ---------------------------------------------------------------------- Traceback (most recent call last): File "/nfs/galaxy/unixhome/jmiller/scipy-test/lib/python2.7/site-packages/scipy/odr/tests/test_odr.py", line 236, in test_pearson np.array([5.4767400299231674, -0.4796082367610305]), File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 842, in assert_array_almost_equal precision=decimal) File "/nfs/galaxy/unixhome/jmiller/numpy-test/lib/python2.7/site-packages/numpy/testing/utils.py", line 665, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal to 6 decimals (mismatch 100.0%) x: array([ 1., 1.]) y: array([ 5.47674 , -0.479608]) ---------------------------------------------------------------------- Ran 17005 tests in 114.097s FAILED (KNOWNFAIL=97, SKIP=1181, failures=3) <nose.result.TextTestResult run=17005 errors=0 failures=3>
It looks like I can use a lower optimization setting with this configuration to get everything working. Thank you for your help!
Best,
Jonah
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Jonah,
Thanks a lot for the report and verification.
It seems the O3 optimization will cause the numerical error in Scipy 15.1 as earlier version
Appendix 1:
Known issues:
When the -O3 or -O2(default) compiler flags (more aggressive compiler optimization flags) used for the ifort, one of the scipy tests may fail and it is a known corner case, to avoid this, as a workaround you can use -O1.
would you please help to submit the issue to premier.intel.com to intel fortran or C compiler or their forum:
https://software.intel.com/en-us/forums/intel-fortran-compiler-for-linux-and-mac-os-x. ;
Thanks
Ying
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ying,
Okay, I've submitted the bug to the fortran compiler forum. The link is here:
https://software.intel.com/en-us/forums/intel-fortran-compiler-for-linux-and-mac-os-x/topic/593518
Thank you for your help.
Best,
Jonah

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page