Issue with np.min

KOZHEVNIKOV__Artem — Wed, 16 May 2018 13:47:45 GMT

I'm using the last docker image of "intelpython/intelpython2_full:2018.0.2". And I've met a strange error with a basic "np.min" usage. I'm able to reproduce this on Amazon machine c5.2xlarge (see info below) but not on others. Can you reproduce this issue ? Is it just some kind of processor incompatibility ?

Python 2.7.14 |Intel Corporation| (default, Feb 12 2018, 06:28:32) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import sys
>>> print sys.version
2.7.14 |Intel Corporation| (default, Feb 12 2018, 06:28:32) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
>>> import numpy as np
>>> np.version.version
'1.14.0'
>>> np.min(np.array([1, -1, 1, 1, 1, 1, 1, 0, 1, 1]))
0  # should return -1

>>> np.show_config()
lapack_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/conda/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/conda/include']
blas_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/conda/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/conda/include']
lapack_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/conda/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/conda/include']
blas_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/conda/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/conda/include']
mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/conda/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/conda/include']

uname -a
Linux 3aeba6ed7b31 4.4.0-1057-aws #66-Ubuntu SMP Thu May 3 12:49:47 UTC 2018 x86_64 GNU/Linux


cpuinfo
Intel(R) processor family information utility, Version 2018 Update 2 Build 20180125 (id: 18157)
Copyright (C) 2005-2018 Intel Corporation.  All rights reserved.

=====  Processor composition  =====
Processor name    :    
Packages(sockets) : 1
Cores             : 4
Processors(CPUs)  : 8
Cores per package : 4
Threads per core  : 2

=====  Processor identification  =====
Processor   Thread Id.  Core Id.    Package Id.
0           0           0           0   
1           0           1           0   
2           0           2           0   
3           0           3           0   
4           1           0           0   
5           1           1           0   
6           1           2           0   
7           1           3           0   
=====  Placement on packages  =====
Package Id. Core Id.    Processors
0           0,1,2,3     (0,4)(1,5)(2,6)(3,7)

=====  Cache sharing  =====
Cache   Size        Processors
L1  32  KB      (0,4)(1,5)(2,6)(3,7)
L2  1   MB      (0,4)(1,5)(2,6)(3,7)
L3  24  MB      (0,1,2,3,4,5,6,7)


cat /proc/meminfo
MemTotal:       15842476 kB
MemFree:         8816108 kB
MemAvailable:   15272200 kB
Buffers:           87380 kB
Cached:          6365788 kB
SwapCached:            0 kB
Active:          1128312 kB
Inactive:        5425168 kB
Active(anon):     103096 kB
Inactive(anon):     8396 kB
Active(file):    1025216 kB
Inactive(file):  5416772 kB
Unevictable:        3652 kB
Mlocked:            3652 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:        104020 kB
Mapped:           112984 kB
Shmem:              8760 kB
Slab:             376280 kB
SReclaimable:     350120 kB
SUnreclaim:        26160 kB
KernelStack:        4112 kB
PageTables:         4164 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     7921236 kB
Committed_AS:     848880 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:     26624 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       92132 kB
DirectMap2M:     3504128 kB
DirectMap1G:    13631488 kB

Dear Artem,

Oleksandr_P_Intel — Wed, 16 May 2018 19:19:56 GMT

Dear Artem,

Thank you for the report.

I can reproduce the reported behavior on AVX-512 enabled machines, and will begin analyzing the culprit.

Sincerely,
Oleksandr

Dear Artem,

Oleksandr_P_Intel — Fri, 18 May 2018 17:41:52 GMT

Dear Artem,

I am happy to report that the issue behind this erroneous result has been fixed, an will be released at the scheduled cadence.

Regrettably the only work-around is to either downcast to 32-bit integers, or ensure that array's size is a multiple of 8:

In [1]: import numpy as np

In [2]: def safe_amin(x):
   ...:     len_x = len(x)
   ...:     s8 = (len_x // 8) * 8
   ...:     return np.min([np.min(x[:s8]), np.min(x[s8:])]) if s8 > 0 and s8 < len_x else np.min(x)
   ...:
   ...:

In [3]: x = np.array([1, -1, 1, 1, 1, 1, 1, 0, 1, 1])

In [4]: np.min(x) # wrong as reported
Out[4]: 0

In [5]: safe_amin(x)
Out[5]: -1

Similar approach can be taken if minimum is to be computed along a specific axis, or collections of axis.

Sorry about the inconvenience. Please let me know if you need help implementing a more robust work-around.

topic Issue with np.min in Intel® Distribution for Python*

Issue with np.min

Dear Artem,

Dear Artem,