ifort/ifx BUG: A serial code compiled using `/Qopenmp` runs with an error

[zoziha]

Description

Hello, Intel Fortran compiler developers,

I recently had a piece of code that was supposed to be parallel and got the correct result using `gfortran`, but `ifort/ifx` gave unexpected results. After checking, it should be clear that an `ifort/ifx` bug was triggered.

My environment is (for this purpose, I upgraded to the latest oneAPI 2023.2.0 ) :

Windows-10 OS, AMD R5 2500U;
ifort (Intel(R) 64, Version 2021.10.0 Build 20230609_000000)
ifx (Intel(R) 64, Version 2023.2.0 Build 20230627)

Serial code with `/Qopenmp`

The serial code is `serial.f90` in the attachments, note that the above is serial code without any `OpenMP` syntax, and compiling it using `ifort/ifx` with `/Qopenmp` causes problems (wrong results) when run:

> ifort /Qopenmp serial.f90; ./serial
> ifx /Qopenmp serial.f90; ./serial
indexing...
searching...
i,j,k= 1 9 118 index= 0
index out of range, bug occurred, using `/libs:dll` option would solve this problem

Run this normally with the following command (note `/libs:dll`) :

> ifort serial.f90; ./serial
> ifx serial.f90; ./serial
> ifort /Qopenmp /libs:dll serial.f90; ./serial
> ifx /Qopenmp /libs:dll serial.f90; ./serial
indexing...
searching...

Parallel code with `/Qopenmp`

Add `OpenMP` syntax to the main program (line 228), the parallel code is `openmp.f90` in the attachments.

After adding the `OpenMP` syntax, using the corresponding command-line compilation is consistent with the serial code, and using `ifort/ifx` with `/Qopenmp` runs into problems:

> ifort /Qopenmp openmp.f90; ./openmp
> ifx /Qopenmp openmp.f90; ./openmp
 indexing...
 searching...
 (key)i,j,k=         255         115         155 (value)index=           0
index out of range, bug occurred, using `/libs:dll` option would solve this problem, output file is `debug.txt`

Run this normally with the following command (note `/libs:dll`) :

> ifort openmp.f90; ./openmp
> ifx openmp.f90; ./openmp
> ifort /Qopenmp /libs:dll openmp.f90; ./openmp
> ifx /Qopenmp /libs:dll openmp.f90; ./openmp
indexing...
searching...

This code is the smallest replicated example I could find, and it's still 254 lines of code

1. I believe that code compiled without `openmp` syntax should behave the same with and without `/Qopenmp`, I think this bug is related to the `/Qopenmp` option;
2. Using `/Qopenmp` and `/libs:dll` seems to work, indicating either static linking or incorrect calls to system APIs;
3. This is a hash table program. When you run the program, it generates `debug.txt `, which stores the `key/value` pairs output by the program when it runs.

serial.f90 runs online see link: https://godbolt.org/z/Tbq6f79bG.

[Barbara_P_Intel]

There was a recent discussion about what happens when you compile with -qopenmp (/Qopenmp). There is more than meets the eye. You can read it here.

[zoziha]

Thank you for your reply @Barbara_P_Intel , the question in your link may be an OpenMP-biased discussion of that issue, but the question in this issue may go beyond the basic linking issue with OpenMP, and it may be that OpenMP triggers a related bug about `allocatable` variables in types (maybe it has something to do with heap or stack allocation).

Enabling `/Qopenmp` causes links to change, I can't comment on that.

But for specific code, I enable `/Qopenmp` and the code is good, error-free logic and parallelizable, I store data like `key=1,9,118;value=1` into a hash container, but when I go to access the value with `key=1,9,118`, surprisingly, I find that its value is missing! Instead of `value=1`, it could be an arbitrary value determined by memory! This behavior that results in a missing value should be unacceptable.

[Barbara_P_Intel]

@zoziha, you are correct about the link discussion. There is one detail buried in there that could be relevant to you.

/Qopenmp sets /Qauto automatically making all variables into stack variables by default, instead of declaring them essentially on the heap.

[zoziha]

Sorry for this, I have only the most superficial knowledge of computers, I'm just a student of mechanics.
What I mean is that whether the compiler's compilation options `/Qopenmp` cause variables to be placed on the heap or the stack should not affect the result of the computer's calculations, is it right?
The result of the calculation in this post is weird, implying some kind of bug? I would actually want to get confirmation from the compiler developers as to whether there is indeed a bug here :), thanks!

[Barbara_P_Intel]

I compiled the serial.F90 like this, ifx -what -g -O0 -check all serial.F90. The output indicates there's an uninitialized variable. That could be a reason for the run failure.

This is a bit of the output. I admit the output is AWEFUL to understand. The first place to look is line 114 in serial.F90, the push_back routine.

  #7 0x49b3fe in nnps_key_value_mp_push_back_ /nfs/site/home/b/quad/05928696/serial.F90:114:13
  #8 0x4a7186 in nnps_spatial_hashing_mp_set_ /nfs/site/home/b/quad/05928696/serial.F90:185:14
  #9 0x4af17d in MAIN__ /nfs/site/home/b/quad/05928696/serial.F90:217:14

Yes, ifort does not report this uninitialized variable.

[zoziha]

The `serial.f90` behaves normally without `/Qopenmp`, and only when `/Qopenmp` is enabled I get a runtime error (missing values). I've checked that the values are stored deterministically when pushing values into the hash table, but after storing a large number of values, querying the hash table key-value reveals that values are missing for almost all keys.

Line 114 is the following code, where adding a key-value pair to the end of an `allocatable` array uses the constructor, which should be a false positive warning here.

self%items = [self%items, key_value_constructor(key, value)]

 The main thing is that the code works fine with `gfortran` and special `ifort/fix` compilation options (e.g. `/Qopenmp /libs:dll`), whereas the hash table values are lost when using `/Qopenmp` alone.

[Barbara_P_Intel]

Thank you for taking the time to look at the uninitialized variable reported.

I filed a bug report, CMPLRLLVM-49983. I'll let you know when there's a fix.

This fails the same way on Linux.

[Barbara_P_Intel]

The compiler team continues the triage of this issue using valgrind. Have you used that? It's used to detect memory mismanagement. serial.f90 compiled without /Qopenmp is showing issues with the memory management. We are investigating to find the source of the problem.

[zoziha]

Sorry, I have relatively average performance needs, I use Fortran mostly under Windows, I haven't used valgrind, but I have heard of it.
My impression of Fortran's semantic environment is that memory problems generally only occur at allocatables and pointers, and this code I think has a high probability of not having memory problems, although I can't 100% guarantee it.
Since the code runs normally with `gfortran` and `ifort` without `openmp`, the stored data won't and shouldn't be lost, but I'm surprised that you talk about memory management problems in code (`serial.f90`) without `/Qopenmp`.

Even with `ifort` enabled `/Qopenmp`, by inserting `print` statements locally, it is possible to see that the data is stored in the data container, but after returning from the storage action and waiting for the next access, the target data is missing.

(The root cause of this problem here could be very well hidden )

[Barbara_P_Intel]

I have an update for you on this issue. There is a fix for it in the ifx compiler 2024.1.0 that will be released in a couple of months.

Since ifort is deprecated and will no longer be available by the end of 2024, I recommend that you use ifx when the next release is available.