- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We work with camera images from CMOS sensors which often use 16Bit image formats (int16, uint16). For some image sizes the implementation of percentile() causes memory corruptions/segfaults. This not observed for other data types (8, 32 or 64Bit) and is present at least in numpy version 1.13 ... 1.17 (releases 2018-1 through 2019-5). Other python distributions than intel behave normal.
We could boil down the problem to the attached 3-liner 'minimal-iP.txt':
import numpy as np
test = np.array([np.arange(i, 44+i, dtype=np.int16) for i in range(56)])
np.percentile(test, (0.1, 0.99))
$: python3 minimal-iP.txt
The resulting crash log is also attached. Other image sizes might work, (43x47) is also known to trigger the crash.
With best regards,
Stephan
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Stephan,
Thank you for providing a clean reproducer. I was able to reproduce the crash. The crash is due to aggressive compiler optimization of {{dumb_select_short}} routine in numpy/core/src/npymath/selection.c.src, which implements quadratic complexity selection.
The fix is suppress vectorization there for short integers, just like it is done for bytes already.
https://github.com/IntelPython/numpy/blob/intel/1.17.3/numpy/core/src/npysort/selection.c.src#L258
The issue will be brought up to the compiler's team attention. I will report here once the binary of numpy 1.17.3 with the fix is published.
Sorry for the inconvenience,
Oleksandr

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page