Intel® C++ Compiler
Support and discussions for creating C++ code that runs on platforms based on Intel® processors.
7727 Discussions

ICC compiler bug building Python sha3

Ed1
Beginner
230 Views

Hi,

When trying to build Python 3.6.3 (and 3.6.4) using icc 2018.1 and .2 the compiler segfaults.

It is essentially a problem with icc blowing up with a very long line of code generated by a macro.

building '_sha3' extension
icc -pthread -fPIC -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3
-Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-parameter
-Wno-missing-field-initializers -fp-model strict -I./Include -I.
-I/usr/local/include
-I/derp/Python-3.6.4/Include
-I/derp/Python-3.6.4 -c
/derp/Python-3.6.4/Modules/_sha3/
sha3module.c -o
build/temp.linux-x86_64-3.6/derp/Python-3.6.4/Modules/_sha3/sha3module.o
": internal error: ** The compiler has encountered an unexpected problem.
** Segmentation violation signal raised. **
Access violation or stack overflow. Please contact Intel Support for assistance.

Full details of the problem are here: https://bugs.python.org/issue33174

My hacky solution to get it to build was to expand the macro into multiple lines, but even then it will only build with "-O0".

0 Kudos
1 Reply
Ed1
Beginner
230 Views

Updates from the Python bug report:

 

msg320100 - (view)

  Author: Kenneth Hoste (boegel) Date: 2018-06-20 17:47
After a bit of back and forth, we have figured out the underlying cause for the problem with the Intel compiler: if the stack limit is not set to a specific value (see output of "ulimit -s"), the Intel compilers fail with a segfault or Interal Compiler Error (ICE).

But if the stack limit is set to 'unlimited' (e.g. via "ulimit -s unlimited"), then the compilation works fine...

See also https://github.com/easybuilders/easybuild-easyconfigs/issues/6484 and our (optional) workaround in https://github.com/easybuilders/easybuild-easyblocks/pull/1441
msg320104 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-06-20 18:52
Thanks for all the investigate work and thanks for the finding what appears to be an easy workaround, @boegel.  Clearly, though, this is not a Python problem and, with an external workaround (e.g. increasing the process stack size via ulimit), there is no need for a workaround in Python itself so I'm closing this issue.  Suggest making sure that the workaround is brought to the Intel compiler team's attention.
msg320106 - (view) Author: Kenneth Hoste (boegel) Date: 2018-06-20 19:01
Fully agreed that this is not something that should be fixed in the Python code base.

Moreover, the problem has been fixed in Intel compilers 2018 update 3 as far as I can tell (tested with both Python 3.6.4 and 3.7.0rc1).

I can't reproduce the problem anymore with that version of the Intel compilers, even after setting a hard stack limit (e.g. "ulimit -s 10240"), while it was easy to reproduce with the 2018 update 1 version.
Reply