OMP: Error #29: Unable to set OMP thread stack size

[rmf166]

Hello-

I'm running into some trouble after compiling a very simple fortran code with a few OpenMP directives on RHEL 4 using the Intel Fortran 11.1.075 compiler. Using the following compile flags

ifort  -O -static -openmp -openmp-link static -o test_omp.exe main.f90

I encountered the following error messages

>./test_omp.exe
  THREADS      =  2
  OMP_STACKSIZE= 16M
OMP: Error #29: Unable to set OMP thread stack size to 16777216 bytes:
OMP: System error #22: Invalid argument
OMP: Hint: Try changing OMP_STACKSIZE and/or the shell stack limit.
forrtl: error (76): Abort trap signal
Abort

However, if I recompile with (dropped -static)

ifort  -O -openmp -openmp-link static -o test_omp.exe main.f90

The program terminates correctly

>./test_omp.exe
  THREADS      =  2
  OMP_STACKSIZE= 16M
  NORMAL TERMINATION

Evidently there is a conflict between -static and -openmp-link static.

Is -openmp-link static sufficient for a static compile? Can I verify this somewhere?

I would like to compile in RHEL 4 and execute the program in RHEL 6.4

Thanks in advance

[Casey]

You can use the program 'ldd' to list dynamic library dependencies. 

[rmf166]

Addendum:

If I switch the -static flag with -static-intel, then the program runs correctly, i.e. the OMP_STACKSIZE for each thread is set to 16 MB and terminates normally. What are the differences between -static and -static-intel? Thanks again.

[rmf166]

Casey,

Thanks for the response. Compiling without -static, on RHEL4

>ldd -v test_omp.exe
    libm.so.6 => /lib64/tls/libm.so.6 (0x0000003da0a00000)
    libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000003da1200000)
    libc.so.6 => /lib64/tls/libc.so.6 (0x0000003da0700000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003da2e00000)
    libdl.so.2 => /lib64/libdl.so.2 (0x0000003da0c00000)
    /lib64/ld-linux-x86-64.so.2 (0x0000003da0500000)

    Version information:
    ./test_omp.exe:
        libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
        libgcc_s.so.1 (GCC_3.0) => /lib64/libgcc_s.so.1
        libpthread.so.0 (GLIBC_2.3.2) => /lib64/tls/libpthread.so.0
        libpthread.so.0 (GLIBC_2.2.5) => /lib64/tls/libpthread.so.0
        libc.so.6 (GLIBC_2.3.2) => /lib64/tls/libc.so.6
        libc.so.6 (GLIBC_2.3) => /lib64/tls/libc.so.6
        libc.so.6 (GLIBC_2.2.5) => /lib64/tls/libc.so.6
    /lib64/tls/libm.so.6:
        libc.so.6 (GLIBC_2.2.5) => /lib64/tls/libc.so.6
    /lib64/tls/libpthread.so.0:
        ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
        ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
        libc.so.6 (GLIBC_2.3.2) => /lib64/tls/libc.so.6
        libc.so.6 (GLIBC_PRIVATE) => /lib64/tls/libc.so.6
        libc.so.6 (GLIBC_2.2.5) => /lib64/tls/libc.so.6
    /lib64/tls/libc.so.6:
        ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
        ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
        ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
    /lib64/libgcc_s.so.1:
        libc.so.6 (GLIBC_2.2.5) => /lib64/tls/libc.so.6
    /lib64/libdl.so.2:
        libc.so.6 (GLIBC_2.2.5) => /lib64/tls/libc.so.6
        libc.so.6 (GLIBC_PRIVATE) => /lib64/tls/libc.so.6
        ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2

whereas compiling on RHEL6.4

>ldd -v test_omp.exe
    linux-vdso.so.1 =>  (0x00007fff34f3f000)
    libm.so.6 => /lib64/libm.so.6 (0x00007fe885372000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe885154000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fe884dc1000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fe884bab000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007fe8849a6000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fe885601000)

    Version information:
    ./test_omp.exe:
        libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
        libgcc_s.so.1 (GCC_3.0) => /lib64/libgcc_s.so.1
        libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
        libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
        libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
        libpthread.so.0 (GLIBC_2.3.2) => /lib64/libpthread.so.0
        libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
    /lib64/libm.so.6:
        libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
        libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /lib64/libpthread.so.0:
        ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
        ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
        ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
        libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
        libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
        libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /lib64/libc.so.6:
        ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
        ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
    /lib64/libgcc_s.so.1:
        libc.so.6 (GLIBC_2.4) => /lib64/libc.so.6
        libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /lib64/libdl.so.2:
        ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
        libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
        libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6

So there is an additional linux-vdso.so.1 =>  (0x00007fff5f3ff000) in RHEL 6.4 (Is this a clue?). Note that if I do compile with -static on RHEL 6.4, the program runs correctly in RHEL 6.4, but the kernel is too old to run on RHEL 4. If I compile with -static on RHEL 4, the program aborts in the same fashion as in my first message on both RHEL 4 and RHEL 6.4.

Thanks in advance.

[TimP]

Dynamic linking against linux shared objects is recommended for compatibility across a wider range of releases.

-static-intel shouild pick static links against Intel compiler libraries (static libiomp has gone away from more recent releases).

[Steven_L_Intel1]

Static libomp is still there for Linux. It did go away on Windows. Dynamic linking is, I agree with Tim, the better choice, but it also requires redistributables.

It is generally recommended to link against the oldest system you want to run on. Especially on Linux.

[rmf166]

Tim and Steve,

If I compile the test code on RHEL 4 with -static and run the executable on the same Linux version and the same machine, I would expect it to behave correctly, which means run until normal termination. Instead, I get the following abort

>./test_omp.exe
  THREADS      =  2
  OMP_STACKSIZE= 16M
OMP: Error #29: Unable to set OMP thread stack size to 16777216 bytes:
OMP: System error #22: Invalid argument
OMP: Hint: Try changing OMP_STACKSIZE and/or the shell stack limit.
forrtl: error (76): Abort trap signal
Abort

However, without the -static flag, I get the normal execution, again on the same Linux version and machine

>./test_omp.exe
  THREADS      =  2
  OMP_STACKSIZE= 16M
  NORMAL TERMINATION

I understand that compiling with -static and using the executable across different Linux version is not safe, but at least I would expect the -static flag to create an executable which runs to normal termination on the same machine and Linux version it was compiled on. Perhaps I didn't quite understand your comments.

Thanks again

[TimP]

I no longer have a copy of ifort 11.1 installed to look at the release notes to see which Red Hat versions were tested, but evidently 11.1 doesn't include any changes to accommodate RHEL 6.4, which wasn't available during the 11.1 release cycle.

According to https://access.redhat.com/support/policy/updates/errata/ RHEL4 is in "extended life phase" which seems to mean a special "ELS" contract is needed for any support beyond access to past updates.  I suppose it means Intel teams don't have sufficient access to it for problem investigations.

[rmf166]

Tim,

I looked up the ifort 11.1 release notes for Linux

http://software.intel.com/sites/default/files/article/164251/release-notesf-111-l-en-u4.pdf

On page 5,

One of the following Linux distributions (this is the list of distributions tested by Intel; other distributions may or may not work and are not recommended -please refer to Technical Supportif you have questions):

o Asianux* 3.0

o Debian* 4.0

o Fedora* 10

o Red Hat Enterprise Linux* 3, 4, 5

o SUSE LINUX Enterprise Server* 9, 10, 11

o TurboLinux* 11

o Ubuntu* 9.04

Perhaps there is a problem with the installation of the ifort 11.1 on my RHEL machine?

-Best Regards

[TimP]

Yes, I saw those release notes are posted on line; RHEL 3 was "deprecated" and 4 should have been still supported by 11.1.

I was trying to point out the difficulty of supporting a wide range of distros with a compiler which was released before the newer distros you want to support.  Support for -static seems difficult on current Red Hat as I, for one, don't have access to many of the Red Hat static libraries.  I suppose it will be difficult to fix any bugs with support of the old version now that RHEL4 is no longer supported by current compilers and a new major version is in development for release later this year.