mcpcom floating point exception compiling GROMACS 2020 for Intel compiler >= 19 update 5

[Kenway__Owain]

I've been trying to install the latest release of GROMACS (2020) (from ftp://ftp.gromacs.org/pub/gromacs/gromacs-2020.tar.gz) with the Intel compiler suite on RHEL 7.x and have run into the following problem:

[ 47%] Building CXX object src/gromacs/CMakeFiles/libgromacs.dir/pulling/pullutil.cpp.o
icpc: error #10106: Fatal error in /lustre/shared/ucl/apps/intel/2020/compilers_and_libraries_2020.0.166/linux/bin/intel64/mcpcom, terminated by floating point exception
compilation aborted for /tmp/gromacs-build.LsXlUP0elB/gromacs-2020/src/gromacs/pulling/pullutil.cpp (code 1)
make[2]: *** [src/gromacs/CMakeFiles/libgromacs.dir/pulling/pullutil.cpp.o] Error 1
make[1]: *** [src/gromacs/CMakeFiles/libgromacs.dir/all] Error 2
make: *** [all] Error 2

I get this failure regardless of whether I try to compile with icpc that comes with the 2020 or 2019 update 5 versions of Intel Cluster Studio and regardless of whether I compile on a RHEL 7.4 or 7.6 host.

I am using GCC 7.3.0 as the version of GCC to get the C++ standard libraries from.

Interestingly, it *does* compile with the version of icpc that comes with the Intel Cluster Studio 2019 update 4, so this bug was presumably introduced between update 4 and update 5.

Compiler version info:

Doesn't work (floating point exception):

$ module load compilers/intel/2020/release
$ icpc -v
icpc version 19.1.0.166 (gcc version 7.3.0 compatibility)

$ module load compilers/intel/2019/update5
$ icpc -v
icpc version 19.0.5.281 (gcc version 7.3.0 compatibility)

Works (compiles):

$ module load compilers/intel/2019/update4
$ icpc -v
icpc version 19.0.4.243 (gcc version 7.3.0 compatibility)

If it matters, I've tested with both Skylake and Cascade Lake hosts.
 

[Kenway__Owain]

Just to add, in dmesg there's a bit more detail about the error:

[9937246.824662] traps: mcpcom[204416] trap divide error ip:74c178 sp:7ffd43eb7870 error:0 in mcpcom[400000+2ac8000]

[GouthamK_Intel]

Hi Owain,

Thanks for reaching out to us. 

We tried to reproduce this issue in ubuntu 18.04, icpc -19.1 version and we didn't get any errors. Please find the screenshot below for reference. 

Now we are working on RHEL distribution, and we will provide you with further updates. 

Regards

Goutham

[Kenway__Owain]

That's interesting.

In the meantime, the GROMACS developers have released 2020.1 (and 2019.6) which explicitly includes a work-around for the compiler issue by dropping the optimisation level for that file with a pragma:

$ diff gromacs-2020/src/gromacs/pulling/pullutil.cpp gromacs-2020.1/src/gromacs/pulling/pullutil.cpp
6c6
<  * Copyright (c) 2013,2014,2015,2016,2017,2018,2019, by the GROMACS development team, led by
---
>  * Copyright (c) 2013,2014,2015,2016,2017,2018,2019,2020, by the GROMACS development team, led by
519a520,524
> // Compiler segfault with 2019_update_5 and 2020_initial
> #if defined(__INTEL_COMPILER) \
>         && ((__INTEL_COMPILER == 1900 && __INTEL_COMPILER_UPDATE >= 5) || __INTEL_COMPILER >= 1910)
> #    pragma intel optimization_level 2
> #endif

This does indeed work and allows me to compile 2020.1 on our systems with the Intel 19.1 C++ compiler which dramatically decreases the urgency but still points to a bug somewhere (i.e. enough other people had the issue for the GROMACS developers  to diagnose and put a fix in).

Oh, in case it matters in reproducing my issue I'm issuing the following steps to build GROMACS:

$ mkdir build
$ cd build
$ cmake ..     -DGMX_GPU=OFF     -DGMX_FFT_LIBRARY=mkl     -DGMX_X11=ON     -DCMAKE_INSTALL_PREFIX=../../install

<cut for brevity>

$ make

Cheers,

Owain

[GouthamK_Intel]

Hi Owain,

We are glad that your issue got resolved with GROMACS 2020.1. 

Please confirm whether we can close this thread. 

Regards

Goutham

[GouthamK_Intel]

Hi Owain,

We are closing this thread. 

Please feel free to raise a new thread in case of any further issues.

Thanks

Goutham