Bad codegen for automatic reallocated arrays results in stack overflow

[tobias-loew]

The following code generates a stack overflow on Windows (for a standard 1MB stack) even when compiled with default "Release" settings.

    module test
    
    contains
    subroutine add_lines()
    character(500):: line
    integer:: loop_file
    character(500),dimension(:),allocatable:: c

    allocate(c(0))
    c = ''

    do loop_file=1,10000
        line = 'abcd'
        c = [character(len=500) :: c , line]
    end do

    end subroutine
    end module test

    
    program Console1
    use test
    implicit none

    
    call add_lines()

    end program Console1
    

[Ron_Green]

This example is curious.  I also see the stack exhaustion.  I can prevent it by adding -heap-arrays.  But it still should not need this option.

I need to investigate this more.  it's not obvious to me why the constructor would blow up stack.  It should only use 500 chars of storage.  Might be that the RHS constructor is not being freed up after the copy to the LHS.  Just a suspicion. 

[tobias-loew]

Yes, I also just found out that -heap-arrays (even with sizes far greater than the stack size) prevents it.

But without -heap-arrays:

While debugging through the loop, the stack-pointer is growing all the time. After leaving "add_lines" the stack-pointer has the correct value again (when the loop-counter is below the stack-overflow limit, of course).

[Ron_Green]

BugID is CMPLRLLVM-69147

[Ron_Green]

Let me explain in more detail what is happening in this case.

First, 'c' is a local variable for subroutine add_lines.  Most compilers except Intel will allocate c from heap.  As you saw, Intel Fortran uses stack for local allocations by default.  So our behavior is different than other compilers by default.  

'c' is a array of rank 1, each element is 500 characters. 

This code is creating 500 characters for each of the 10,000 elements that you create for c.   If loop_file is larger, then c has more elements.   the number of elements of c is the number of iterations of loop_file.  In effect, you are doing this:

character(500), dimension(:), allocatable :: c

allocate ( c(10000) ) 
c(1:10000) = 'abcd'     ! each element 500 chars beginning with abcd

But your code, which I hope is not in any real application, does this :

do loop_file=1,10000
        line = 'abcd'
        c = [character(len=500) :: c , line]
end do

On line 3, the following happens:

   for the constructor, a temporary is created for c.  C is growing in each iteration.  for loop_file=1, C has 1 element ( C(1:1) ).  At each iteration C grows by another element (row or line if you want to think of it this way).  So the constructor is making a temporary each time for C, and C is growing in size by 500 x value of loop_file.   Gfortran, NAG, others allocate that temp in heap.  Intel is growing the stack.

After the temp is created, the LHS old data in C is deallocated, and reallocated for the size of the C-temporary on the RHS. 
Then the elements/data of C-temporary are copied into this new empty allocation of C.  
Since stack is limited, you eventually blow out of the stack with Intel.  In gfortran and NAG, you are just sloppily growing heap.  And all along the way you are doing deallocates and reallocates and copies on each iteration.  Horrible coding.  

Fine, but if you want to do this filling in C with a loop going over each line that is your call.  Most of us would do 1 allocation of C with the number of element (rows or lines) that you want and do this just once.  Then fill the elements or lines one by one without the need for reallocation.

So you want Intel Fortran to behave like gfortran or NAG or any other Fortran compiler?  Do this change for the declaration of C

character(500),dimension(:),allocatable,save:: c

the SAVE attribute will cause Intel Fortran to use heap instead of stack.  Just like the -heap-arrays option.  And just like the other compilers.  Keep in mind, this is horrifically inefficient coding.  Nonetheless, it will give you the results you seek.  Of these 2 options, the SAVE attribute will work without the need for compiler option -heap-arrays.  Options are notorious in that sometime in the future you or someone following you will forget to add that option.

We did find a small efficiency in our own creation of the C-temporary for the constructor.  we were able to reduce our temp size usage by 1/2.  Still, if you increased loop_file you will eventually hit stack limit without the 2 workarounds described.