Stack overflow when copying a large array

[Arjen_Markus]

Hello,

we have encountered a problem with copying large arrays. The following program illustrates
this. Some details: we use Intel Fortran 11.1.065 on Windows XP (it happens with 11.0.72 and
11.1.54 as well, but not with 11.0 on Linux).

The output from the program is:

forrtl: severe (170): Program Exception - stack overflow

Image PC Routine Line Source

bigarrays.exe 0042A157 Unknown Unknown Unknown

bigarrays.exe 00401296 Unknown Unknown Unknown

bigarrays.exe 00440183 Unknown Unknown Unknown

bigarrays.exe 0042A303 Unknown Unknown Unknown

kernel32.dll 7C817027 Unknown Unknown Unknown

The problem occurs at the statement "xr =x" in the program below.

There are several ways of solving it:
- Use explicit loops
- Use an associate block

Still, this is an annoying phenomenon.

Regards,

Arjen

----

! bigarrays.f90 --

! Try and reproduce a stack overflow problem with large

! arrays

!

module griddata

implicit none

integer, parameter :: double = kind(1.0d0)

type grid_t

real(double), dimension(:,:), pointer :: x

real(double), dimension(:,:), pointer :: y

end type grid_t

type(grid_t) :: grid

contains

subroutine copy_grid_arrays( grid )

type(grid_t), target :: grid

real(double), dimension(:,:), pointer :: x, y

real(double), dimension(:,:), pointer :: xr, yr

integer :: m, n

x => grid%x

y => grid%y

m = size(grid%x, 1)

n = size(grid%x, 2)

allocate( xr(m,n) )

allocate( yr(m,n) )

xr = x

yr = y

end subroutine copy_grid_arrays

end module griddata

program bigarrays

use griddata

implicit none

integer :: m, n

m = 1500

n = 700

allocate( grid%x(m,n) )

allocate( grid%y(m,n) )

grid%x = 1.0_double

grid%y = 1.0_double

call copy_grid_arrays( grid )

end program bigarrays

[mecej4]

I wonder if, after the user increases the stack to allow

xr = x

to function without causing stack overflow, the memory allocated for the copy operation is released/reused for the next statement:

yr = y

[TimP]

I suppose the stack overflow is incurred by an intermediate temporary array, thus may depend on your /link /stack settings. On linux, if similar code is generated, it would also depend on stack settings. As you hint, usage of unexpected temporaries is provoked by array assignment, and might be avoided by explicit loops.
In my viewpoint, if another compiler (e.g. gfortran) is able to compile the code without using a temporary (thus consuming more cache), it is a worthwhile challenge for ifort to eliminate the temporary.

I don't have a copy of the compiler you used. The latest ifort for Win64 has solved some such problems; as far as I can see, this case may be one of those which is fixed.

[Arjen_Markus]

I can confirm that gfortran on Windows XP runs the program without complaints.
On Linux I noticed messages from the Intel compiler about loops being vectorised that I do
not see on Windows (they correspond to the array assignments).

It may have to do with the allocation of temporary arrays - ISRT gfortran always uses the
heap and only recently acquired the possibility to use the stack instead for performance
reasons.

The issue is solved if I use the -heap-arrays option.

Regards,

Arjen

[Steven_L_Intel1]

There are several things going on here. On Windows, the default stack size is established by the linker and is a puny 1MB. The Intel compiler, by default, puts temporary arrays on the stack, as this is faster. Some other compilers put them on the heap by default.

Also, as of 11.1 the Intel compiler no longer issues vectorization messages by default.

[Arjen_Markus]

The unexpected thing is that it uses temporaries at all. Is that because of all the pointering?
To prevent aliasing to wreck havoc? (I probably found the reason myself while formulating my
question ;))

Regards,

Arjen

[Steven_L_Intel1]

Right - the compiler assumes that pointers are aliased, since pointers implicitly have the TARGET attribute. If you use ALLOCATABLE instead, it doesn't.

[Arjen_Markus]

The code looks so innocent that I would never have thought temporaries are required. Luckily
we have compilers to do the tough work :).

Regards,

Arjen