Unexpected stack overflow: array temporaries?

[Arjen_Markus]

I encountered a peculiar problem with a small program - for a case with large arrays the program crashed because of stack space problems. It was solved by adding the option -heap-arrays, but as the program merely uses pointers to array sections I was surprised that temporary arrays were involved.

Is there a way to get the compiler to show where these are created? 

I have attached the program source and the input file (rename it to .inp).

Further information: I also have a version that uses ASSOCIATE instead of pointers and it gives the same problem. With the same solution. It is part of a series of programs to explore what variations are possible. 

[andrew_4619]

I recall the optimiser reports show some detail on temp creation but that is a while since I looked at that,

[garraleta_fortran]

do this

[Arjen_Markus]

I did use that option to get around the problem, but the question is: why would that be necessary? What temporary arrays are used? 

[JohnNichols]

using stock standard console1 Fortran copy your code in no changes in VS 2025 Preview oneapi 2025.1 using ifx in debug mode, compiles fine.  

Need input file to test it.  

[JohnNichols]

using your exact code.  NO tweaking 

[Arjen_Markus]

I supplied a sample input file as "poisson_island_pointer.txt" since the forum does not accept files with ".inp" as the extension. The matrix size 1000x1000 causes the stack overflow, both with the debug (-Od) and the optimised version (-O2). My current compiler version is 2025.0.0, perhaps I should upgrade :).

[garraleta_fortran]

I have successfully received the INP file.
This is the result of the process.

[Arjen_Markus]

So, you succeeded in running the program. Unfortunately, I removed a part of the program that resides in another source file, so that you did not get the compiler information. What compiler and compile options did you use? 

[garraleta_fortran]

see file p.jpg 

[Arjen_Markus]

Ah, yes, of course, that will solve it :). See also Ron Green's detailed answer. 

That said, it helps (me) to read such explanations as I am not a compiler writer or very knowledgeable about them (beyond having used them a lot over the years :)). 

[Ron_Green]

it's interesting that you posted this question at this time.  Not 2 weeks ago we started discussions about the capability to alert when array temps are used.  Several customers have recently asked for such a new feature.

There are 2 possibilities to this feature request:  

One is a compile time check, perhaps a new -warn keyword to issue a compile time warning for array temp need.  The trouble here is that sometimes the array temp is only determined to be needed at runtime.  So the compiler warning would have to be something like "array temporary creation possible <here>".  Other cases are certain "array temporary creation <here>".  

Another possibility for this feature request is runtime checking.  Perhaps something under the -check <keyword> option.  This would be more exact, as the warning would emit only when array temp creation is encountered.    This could cause a WHOLE LOT of warnings if the creation occurs countless times in a run.  Which is quite possible, it could be millions of such occurences.  Runtime checks are sent to STDERR so in theory one could redirect STDERR to a file so as not to clutter the actual runtime output stream in STDOUT.  Still, I would not like to filter millions of lines of warnings to find all the locations of array temp usage. 

Of the 2 solutions proposed, I prefer the compile time check.  I think a runtime check would just flood the user with too much output to be practical.  What do you think? 

As you found, the -heap-arrays option can help avoid the default stack allocation for array temps.  This could be added to your ifx.cfg file to apply to all compilations if you so desire.   We have been asked "why not make some stack/heap threshold default"?  The answer - performance.  Stack allocation is much faster than tracking things in heap.  So if we changed the compiler to auto detect large array temps and automatically do those in heap then performance would be impacted.  And I think you will agree that this would cause an uproar.  

I will resume the discussion of a possible new -warn feature to flag possible array temp 

Now in your particular example it is quite obvious to people familiar with Fortran compiler so spot your array temp need.  It is this 

pderiv = diffw * pwest  + &
                 diffe * peast  + &
                 diffn * pnorth + &
                 diffs * psouth   &
                 - (diffw + diffe + diffn + diffs) * pcentre + pforce

real, dimension(:,:), pointer             :: pcentre, pwest, peast, pnorth, psouth, pforce, pderiv, &
                                                 diffw, diffe, diffn,  diffs

look at the left-hand side of the assignment and the right-hand side:  pderiv is a pointer to real, dimension(:,:)  targets.  Just like the LHS.  So pderiv COULD possibly alias the same memory as any of the pointers on the RHS.  Sure, YOU can see that it doesn't but the compiler is not aware of how you have assigned the pointers and thus makes the safe assumption to make an array temp for the RHS calculation then store the results with a mem move into the data pointed to by pderiv.  Compilers err on the side of safety. 

[Arjen_Markus]

Ah, thanks for your enlightening answer! In more detail:

• I agree that a compile-time check on the creation of temporary arrays is more useful. The loop in my small test program would probably become more involved and I am thinking of transferring the inner part to a subroutine, so that more customisation is possible, Like I said: this is an investigation into the possibilities.
• Such a check is useful as a prognosis tool. In my case, the program stops immediately, but it might also occur after several hours of hard labour (by the computer) when it enters the output phase of the program :).
• The stack overflow also occurred in the version that uses association in stead of pointers. I would assume that the compiler is then capable of detecting that pderiv is not aliased to anything involving the right-hand side. There the code looks like this:
  
  

    !
    ! Set up the associations to the respective array sections
    !
    associate(                             &
        pcentre => u(2:nx+1,2:ny+1),       &
        pwest   => u(1:nx  ,2:ny+1),       &
        peast   => u(3:nx+2,2:ny+1),       &
        psouth  => u(2:nx+1,1:ny  ),       &
        pnorth  => u(2:nx+1,3:ny+2),       &
        pforce  => force(2:nx+1,2:ny+1),   &
        pderiv  => du(2:nx+1,2:ny+1),      &
        diffw   => diffx(1:nx,2:ny+1),     &
        diffe   => diffx(2:nx+1,2:ny+1),   &
        diffn   => diffy(2:nx+1,2:ny+1),   &
        diffs   => diffy(2:nx+1,1:ny)      )

        do while ( error > eps .and. iter < max_iter )

            pderiv = diffw * pwest  + &
                     diffe * peast  + &
                     diffn * pnorth + &
                     diffs * psouth   &
                     - (diffw + diffe + diffn + diffs) * pcentre + pforce

            iter  = iter + 1
            error = deltt * sum(abs(du)) / (nx-2) / (ny-2)

            u = u + deltt * du

            write(  *, * ) iter, error
            write( 20, * ) iter, error
        enddo

    end associate

​

• I found that gfortran handles this differently and actually of the four versions of the program the "associate" version is faster.

[Arjen_Markus]

Actually, knowing the reason for the compiler to create such a temporary array, I now see some more variations :). They might be able to persuade the compiler not to do this, thus probably enhancing the performance.

[jimdempseyatthecove]

>>This could cause a WHOLE LOT of warnings...

In my experience with ifort, the warning appeared once for any statement. iow by use of a once-only flag for each allocation.

Jim Dempsey

[Ron_Green]

Feature Request ID is CMPLRLLVM-68444

[SteveSuehs]

Ron, 
Thanks for creating that Feature Request.

I'd like to offer another compelling variant  on the current Runtime check.   If we turn on [temp array detection] or other options covered by /check:all, IFORT would pop up a dialog on the desktop, which is not much fun when running a few thousand input cases.  If we could configure the handlers it would be helpful, as we could then route them to log files instead of popping up a user-facing dialog, etc.

-Steve

[andrew_4619]

The Intel Fortran environment variable FOR_DIAGNOSTIC_LOG_FILE controls where diagnostic output from the Fortran runtime system is written. If this variable is set, the runtime system will direct its diagnostic messages, including error messages and stack traces, to the file specified by the variable's value, rather than displaying them on the console. 

[SteveSuehs]

thank you!  I am now digging through the documentation to read more on this.