Segfault with ifx with openmp, recursion and allocatable

[martinmath]

The following code and variations of it often but not always abort with a segfault if compiled with option -qopenmp in Linux. Using valgrind I always see invalid reads and sometimes invalid writes and conditional jumps depending on uninitialised data, depending on the code variation I run.

Without command line option -qopenmp, everything looks fine. However, note that there is no parallel region. Traceback by valgrind points to the memory allocator.

Anyway, in order to get invalid reads (writes...), I also need recursion and a local variable with allocatable component. Interestingly, if I put the local variable tmp in a block section (and the code of the subroutine as well, obviously) then it works without complains from valgrind.

As I mentioned I have tried some variants, all of which failed. In particular I have tried it with a character(len=:), allocatable variable and some character actions. I actually see this problem in recursive parser code, where it always crashs at some point. This looks like a big show-stopper for any modern code which uses openmp, where recursion and allocatable components are common.

module mod

implicit none
public

type :: t
   integer, dimension(:), allocatable :: a
end type t

contains

recursive subroutine rec(c, n, x)
   integer, dimension(:), intent(in) :: c
   integer, intent(in) :: n
   type(t), intent(out) :: x
   type(t) :: tmp
   if (n > 0) then
      call rec(c, n-1, tmp)
      x%a = tmp%a
      x%a(n) = x%a(n) + 1
   else
      x%a = c
   end if
end subroutine rec

end module mod


program alloc_rec

use mod
implicit none

type(t) :: s
call rec([1,2,4], 2, s)
print *,s%a

end program alloc_rec

[jimdempseyatthecove]

While this is a compiler bug, see if the following changes resolve the issue:

   type(t), automatic, :: tmp

Note, automatic is (supposed to be) implicit with -qopenmp and/or recursive procedures.

Older Fortran standards (older versions of Intel's Fortrans), for non -qopenmp and/or recursive procedures, implicitly had UDT's and arrays SAVE.

IIF this corrects the problem, then it is likely that there is an improper IF (...) THEN ... ELSE ... ENDIF in the compiler code. IOW it is erroneously making the UDT's  static.

Please report back if this corrects the problem.

Jim Dempsey

[martinmath]

Thanks for the suggestion. With the block statement there is a work around. Anyway I was just checking out ifx and there are still too many bugs with more modern language features. But it looks promising.

[Ron_Green]

  I am investigating the bug.

[Ron_Green]

actually, @jimdempseyatthecove  you can change tmp to save to work around the issue

type(t), save :: tmp

still, appears to be a bug that you have to do that with IFX.  I'm checking with our OMP architect on this issue.  

IFX has to allow OMP offload so, yes, -qopenmp takes different code paths in IFX than IFORT.  IFX took a wrong turn at Albuquerque. 

[JohnNichols]

Albuquerque 2000.

I went there once for four days.  Starting raining as I landed, taxi driver said never rains here, four days later as I got on the plane it was still raining, taxi driver on way to airport said never seen this before. 

London 1969

Sir, we have never known a period of 3 weeks without rain to my father, wonderful weather. 

Never trust a weatherman, statistically they are wrong about 30% of the time. 

[jimdempseyatthecove]

>>...you can change tmp to save to work around the issue

No, Ron, it is the opposite. The point is to force tmp (actually the array descriptor for tmp) to be stack allocated.

My suspicion is the error is induced in the user's program by the array descriptor for tmp being (implicitly) SAVE as would be for the older implementations of Fortran, and by explicitly tagging it with automatic (unnecessary with newer versions of Fortran) is a means to workaround this bug.

Jim Dempsey

[Ron_Green]

Bug ID is CMPLRLLVM-43741

[Steve_Lionel]

Marking the variable AUTOMATIC has no effect on where the compiler allocates its descriptor.

[Barbara_P_Intel]

@martinmath, this runtime error is fixed in the version of ifx that is planned to be available in mid-year.

[martinmath]

That's great, thanks for rapidly fixing this. Hopefully I can check and run our real code with the upcoming ifx. Currently it crashs right after start due to this issue.

[Ron_Green · Accepted solution]

This bug was fixed in the 2023.2.0 compiler.

[martinmath]

Thanks. I just checked also with some similar small character based parser snippets and it works fine. I still need to check the real code with the newest ifx and the various bugfixes.

However, I got a bit confused. Keyword "recursive" is not required anymore as it is now the default in F18, right? But ifx is not yet using F18 as it complained about missing recursive in one of my testcases. However, adding compiler option -std18 does not make a difference, ifx is still complaining about recursion.

(For years I always use -recursive compiler option in makefiles, so do not much care about this).

[Barbara_P_Intel]

I checked with the Fortran team about the keyword "recursive". This is the reply:

By default, ifort/ifx do not assume a subprogram is recursive; we generate non-recursive entry sequences and code.  Specifying -assume recursion, -standard-semantics or -recursive will get the compilers to generate recursive entry sequences and code without specifying the recursive keyword.

FYI: -stand fxx only diagnoses features that are not part of the standard at the level specified by fxx.

[martinmath]

---

@Barbara_P_Intel wrote:

By default, ifort/ifx do not assume a subprogram is recursive; we generate non-recursive entry sequences and code.  Specifying -assume recursion, -standard-semantics or -recursive will get the compilers to generate recursive entry sequences and code without specifying the recursive keyword.

---

This is a somewhat surprising statement. Even 15 years ago (with ifort10) the generated code with or without recursive keyword or compiler option was always exactly the same. I came upon code which used a detour to avoid the recursion error from the compiler, as the developer was unaware of both keyword and compiler option. Back then it just worked even in openmp parallel blocks.

Just to make sure, I tested with the rather smart code below for determining whether a number is even or odd. Due to the detour the compiler does not see its recursive nature and it just always compiles. Comparing assembler output of ifort as well as ifx does not show any difference, whether '-recursive' is specified or not and with or without -qopenmp (8 variants!). I cannot see that there are really different entry sequences. I mean, the keyword exists only because back in the day there was very little memory, thus no stack and hence no recursivability. Once stacks were established and became the default for passing arguments and storing local variables there was no hindrance for recursive calling and the keyword became kind of obsolete (except for documenting that a procedure is recursively called). I would actually love to hear about historical details of how that evolved.

Here is the code I used to compare compiler output of ifort and ifx with and without -recursive and -openmp. (I also checked with and without local variable m). In all 16 cases I compiled with O2, but there does not seem to be any optimisation step using the (non)recursive information.

module rec_m

implicit none
public

contains

subroutine even(n, isEven)
   integer, intent(in) :: n
   logical, intent(out) :: isEven
   integer :: m
   if (n > 0) then
      m = n - 1
      call odd(m, isEven)
   else
      isEven = .true.
   end if
end subroutine even

subroutine odd(n, isOdd)
   integer, intent(in) :: n
   logical, intent(out) :: isOdd
   integer :: m
   if (n > 0) then
      m = n - 1
      call even(m, isOdd)
   else
      isOdd = .false.
   end if
end subroutine odd

end module rec_m


program rec

use rec_m
use omp_lib
implicit none

integer :: i, j
logical :: isEven, isOdd

!$omp parallel default(private)
i = 300
!$ i = i + omp_get_thread_num()
do j = 1,1000000
   call even(i, isEven)
   call odd(i, isOdd)
end do
if (((modulo(i,2) == 0) .eqv. isEven) .and. (isEven .neqv. isOdd)) then
   print *, 'SUCC: ', i-300, i, isEven, isOdd
else
   print *, 'FAIL: ', i-300, i, isEven, isOdd
end if
!$omp end parallel

end program rec

 The big j-do-loop is there to make sure that any reentrancy issue should pop up in multi-threaded execution.

[jimdempseyatthecove]

Barbara, check with your team as to if the subprograms are default reentrant (IOW safe for use within an OpenMP parallel region).

Jim Dempsey

[jimdempseyatthecove]

Barbara,

There is a difference between recursive and reentrant.

Reentrant means multiple threads my safely enter the procedure concurrently.

Recursive means a single thread within its procedure can call its own procedure (either directly or at some deeper call stack level).

Though the generated code may be the same in many cases, in some cases it may not.

A recursive procedure is generally reentrant-safe.

Jim Dempsey

[Barbara_P_Intel]

The reply is:

-q openmp and -recursive (-assume recursion) all set the -auto command line option, which puts all unsaved scalars and arrays on the stack. By default, scalar intrinsic types get stack allocated, but some derived types and arrays may be statically allocated.  -auto ensures everything that does not have the SAVE attribute (or is in COMMON or EQUIVALENCED) or ALLOCATABLE is allocated on the stack.