Stack overflow when writing large unformatted vector to file

[Petros]

Hi, I met a small problem. During a simulation, I have an internal buffer that I save the results and write them all together at the end. The code reads:

double precision, dimension(:), allocatable :: observBuffer
integer(kind=8) :: observBufferPntr, sizeOfobservBuffer
integer :: iostatus
sizeOfobservBuffer = 492840000 ! This is the size of the matrix
observBufferPntr = 1 ! shows the highest value in the buffer
allocate(observBuffer(sizeOfobservBuffer),STAT=AllocStatus)
open(newunit=traj,file=filenm,form='unformatted',status='replace',iostat=iostatus)
call simulate(observBufferPntr,observBuffer) ! This is the suroutine that "fills" the observBuffer up to 
write(traj)observBufferPntr-1
write(traj)(observBuffer(1:observBufferPntr-1))
close(traj)

It crashes with:

forrtl: severe (170): Program Exception - stack overflow

It flags the line write(traj)(observBuffer(1:observBufferPntr-1)). The same code in GFortran works fine. Also, for smaller sizes of matrix, the above code works fine with Intel as well. It seems I am hitting a size limit somewhere. I tried (without result) enabling/disabling the assume buffered io as well as increasing the stack size. Any help is welcome.

I am using Intel® Parallel Studio XE 2015 Update 2 Composer Edition with Microsoft Visual Studio* 2013, Version 15.0.0118.12.

[IanH · Accepted solution]

Perhaps the compiler is creating a temporary for the array slice.  This might seem silly, but try with the superfluous outer parentheses around the output item removed.  You could also try iterating over the array elements using an implied do.

[Petros]

ianh wrote:

Perhaps the compiler is creating a temporary for the array slice.  This might seem silly, but try with the superfluous outer parentheses around the output item removed.  You could also try iterating over the array elements using an implied do.

If actually worked :o Removing the parenthesis worked! Also, when increasing the stack size by a lot. But, I prefer your solution :)

Any ideas why?

[IanH]

If X is a variable, (X) is an expression, which when evaluated as the same value as X, but it is no longer the same thing as X.  That's not so relevant here, but it matters in some contexts, particularly argument association.  In those other contexts the compiler often has to create a copy of the thing inside the parenthesis for things to work correctly.  I suspect that the compiler is treating the output list items in the same manner as it treats an actual argument.

It is a bit silly that the compiler is creating the copy here, as it doesn't need to, but on the other hand the extra parentheses are a bit odd, so it is not surprising that this silliness has been overlooked or is known but tolerated because there are bigger fish to fry.

[Petros]

Thanks, I'll keep that in mind next time!

ianh wrote:

If X is a variable, (X) is an expression, which when evaluated as the same value as X, but it is no longer the same thing as X.  That's not so relevant here, but it matters in some contexts, particularly argument association.  In those other contexts the compiler often has to create a copy of the thing inside the parenthesis for things to work correctly.  I suspect that the compiler is treating the output list items in the same manner as it treats an actual argument.

It is a bit silly that the compiler is creating the copy here, as it doesn't need to, but on the other hand the extra parentheses are a bit odd, so it is not surprising that this silliness has been overlooked or is known but tolerated because there are bigger fish to fry.