ifort 14.0.2 bug prevents scipy tests to pass

[Vladimir_Rapatskiy]

I've tried to use Intel MKL with numpy 1.8.0 and scipy 0.13.3 python packages.

When I use -O2 option for ifort scipy tests in scipy.odr subpackage fails everytime.

Using -O1 instead of -O2 results in passing the tests.

Here is detailed description of bug: http://software.intel.com/en-us/forums/topic/505993

[TimP]

You don't offer much to indicate this is a bug specific to the latest ifort update, as your title implies.  This may have discouraged someone with more knowledge of python and mkl from looking at your post.   As the legacy i8 option has been replaced by -integer-size 64, and the old one is no longer documented, if you think there is a bug in backwards compatibility of that option, or it is somehow failing at -O2, you should try the current one.  That might be considered a bug in the instructions to which you refer.

[Ron_Green · Accepted solution]

Looking at this it appears to be a simple case of balancing precision with optimizations.

Instead of -O1 I'd have tried   -O2 -fp-model precise  and/or -O2 -fp-model source

a lighter version:   -O2 -prec-div -prec-sqrt 

If I had more time I'd try it myself but just not enough hours in the day to pick up Scipy and give it a go.  There's a good read on balancing accuracy and optimizations here: http://software.intel.com/en-us/articles/consistency-of-floating-point-results-using-the-intel-compiler

be sure to open the attached PDF on this page.  Every person doing computational numerics needs to be aware of the tradeoffs between optimizations and accuracy.

ron

[Vladimir_Rapatskiy]

Ronald W Green (Intel) wrote:

Looking at this it appears to be a simple case of balancing precision with optimizations.

Instead of -O1 I'd have tried   -O2 -fp-model precise  and/or -O2 -fp-model source

a lighter version:   -O2 -prec-div -prec-sqrt 

Dear, Ronald

I tried your proposition with maximum number of options saving numerical precision. I used latest available NumPy 1.8.1 and SciPy 0.14.0rc1. I've got the same error during "scipy.odr" module test.

Test is OK with ifort compiler options: "-O1 -xhost -openmp -fp-model strict"

Testing command: python -c "import scipy; scipy.test('test_multi')"

Command output:

Running unit tests for scipy

NumPy version 1.8.1
NumPy is installed in /usr/local/lib/python3.3/dist-packages/numpy
SciPy version 0.14.0rc1
SciPy is installed in /usr/local/lib/python3.3/dist-packages/scipy
Python version 3.3.1 (default, Sep 25 2013, 19:29:01) [GCC 4.7.3]
nose version 1.3.1
..............
----------------------------------------------------------------------
Ran 14 tests in 1.982s

OK

Test FAILS with ifort compiler options: "-O2 -xhost -openmp -fp-model strict -no-ftz -no-fast-transcendentals -fp-speculation strict -mp1 -fimf-precision=high -fpe0 -prec-div -prec-sqrt"

Testing command: python -c "import scipy; scipy.test('test_multi')"

Command output:

Running unit tests for scipy
NumPy version 1.8.1
NumPy is installed in /usr/local/lib/python3.3/dist-packages/numpy
SciPy version 0.14.0rc1
SciPy is installed in /usr/local/lib/python3.3/dist-packages/scipy
Python version 3.3.1 (default, Sep 25 2013, 19:29:01) [GCC 4.7.3]
nose version 1.3.1
...FFF........
======================================================================
FAIL: test_lorentz (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.3/dist-packages/scipy/odr/tests/test_odr.py", line 293, in test_lorentz
    3.7798193600109009e+00]),
  File "/usr/local/lib/python3.3/dist-packages/numpy/testing/utils.py", line 811, in assert_array_almost_equal
    header=('Arrays are not almost equal to %d decimals' % decimal))
  File "/usr/local/lib/python3.3/dist-packages/numpy/testing/utils.py", line 644, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([  1.00000000e+03,   1.00000000e-01,   3.80000000e+00])
 y: array([  1.43067808e+03,   1.33905090e-01,   3.77981936e+00])

======================================================================
FAIL: test_multi (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.3/dist-packages/scipy/odr/tests/test_odr.py", line 190, in test_multi
    0.5101147161764654, 0.5173902330489161]),
  File "/usr/local/lib/python3.3/dist-packages/numpy/testing/utils.py", line 811, in assert_array_almost_equal
    header=('Arrays are not almost equal to %d decimals' % decimal))
  File "/usr/local/lib/python3.3/dist-packages/numpy/testing/utils.py", line 644, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([ 4. ,  2. ,  7. ,  0.4,  0.5])
 y: array([ 4.37998803,  2.43330576,  8.00288459,  0.51011472,  0.51739023])

======================================================================
FAIL: test_pearson (test_odr.TestODR)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.3/dist-packages/scipy/odr/tests/test_odr.py", line 236, in test_pearson
    np.array([5.4767400299231674, -0.4796082367610305]),
  File "/usr/local/lib/python3.3/dist-packages/numpy/testing/utils.py", line 811, in assert_array_almost_equal
    header=('Arrays are not almost equal to %d decimals' % decimal))
  File "/usr/local/lib/python3.3/dist-packages/numpy/testing/utils.py", line 644, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

(mismatch 100.0%)
 x: array([ 1.,  1.])
 y: array([ 5.47674003, -0.47960824])

----------------------------------------------------------------------
Ran 14 tests in 2.001s

FAILED (failures=3)

My conclusion that failures are related to sources of "scipy/odr/odrpack" module compiled by ifort. Failures comes not from calculations precision, but from somewhere else. Is it possible to see all optimization options enabled in -O2 level, that are disabled at -O1 level? I think it could be related to other optimization techniques that are enabled at -O2 level.

[mecej4]

I do not know any details of the tests, but my suspicions were raised when I saw that some tests "failed" because "Arrays are not equal to almost 6 decimals". If the floating-point calculations in the test were performed in single precision (23+9 bits), i.e., 7+ decimal digits precision, perhaps the acceptance criteria are too stringent. You may consider whether these "failures" are truly indicative of a failed build, a compiler error, or if the tests, themselves, are flawed.

[Ron_Green]

Interesting.  We can turn off vectorization and see if that is the problem:

-O2 -no-vec -openmp -fp-model strict

with this, you should not need the other options, though you could slowly add them 1 by 1 if the above does not give the answers you seek.  IF the above works, then try:

-O2 -no-vec -openmp -fp-model precise

then

-O2 -no-vec -openmp

And have you tried this with gfortran or PGI with full optimizations and vectorizations enabled?

ron