topic Re: segfault with reshaping and transposing big arrays in Intel® Fortran Compiler

segfault with reshaping and transposing big arrays

wim_van_hoydonck — Sat, 09 Feb 2008 22:57:09 GMT

Hi,

For big arrays ( > 1048 by 3000), the following line gives a segfault (complete test program at the end):

b = reshape( transpose(a) , shape(b) )

whereas splitting this line in two, does not result in a crash:

at = transpose(a)
b = reshape( at , shape(b) )

This happens with:
ifort (IFORT) 10.1 20080112
and
ifort (IFORT) 10.1 20070913

with or without optimization.

Compiling with gfortran or g95 doesn't result in a segfault.

For integer(1) arrays, the segfault occurs when the array is approximately 4 times as big. Memory problem?

valgrind reports this (amongst other things):
==5654== Invalid write of size 1
==5654== at 0x402FA0: MAIN__ (test.f90:15)
==5654== Address 0x7FE48E380 is on thread 1's stack
==5654== Can't extend stack to 0x7FE48DA90 during signal delivery for thread 1:
==5654== no stack segment
==5654==
==5654== Process terminating with default action of signal 11 (SIGSEGV)
==5654== Access not within mapped region at address 0x7FE48DA90
==5654== at 0x402FA0: MAIN__ (test.f90:15)
==5654==
==5654== Invalid write of size 8
==5654== at 0x4802338: _vgnU_freeres (vg_preloaded.c:56)
==5654== Address 0x7FE48E2F8 is on thread 1's stack
==5654==
==5654== Process terminating with default action of signal 11 (SIGSEGV)
==5654== Access not within mapped region at address 0x7FE48E2F8
==5654== at 0x4802338: _vgnU_freeres (vg_preloaded.c:56)

Greetings,

Wim

!*test.f90************************************
program test

integer , parameter :: nx = 1049 , ny = 3000
integer , allocatable :: a(:,:) , at(:,:) , b(:)
integer :: status

allocate ( a(nx,ny) , b(nx*ny) , at(ny,nx) , stat=status )
print *, status

forall(i=1:nx,j=1:ny) a(i,j) = i*j

at = transpose(a)
b = reshape( at , shape(b) )

! ifort segfaults here
b = reshape( transpose(a) , shape(b) )

deallocate( a , b , at )

end program test
!********************************************************

Re: segfault with reshaping and transposing big arrays

oh_moose — Sun, 10 Feb 2008 00:12:55 GMT

Looks like a shameless stack abuse case to me.
Try to compile your software with the qualifier -heap-arrays 0

Re: segfault with reshaping and transposing big arrays

wim_van_hoydonck — Sun, 10 Feb 2008 12:55:26 GMT

Thanks,

That works :)

Now I have to figure out why...

Re: segfault with reshaping and transposing big arrays

oh_moose — Mon, 11 Feb 2008 22:37:08 GMT

On Linux/x86 the stack size is limited to 8 MiB. That is a pretty large stack for calling routines. Unfortunately some languages put the actual values of function arguments onto the stack for a function call. Fortran puts references (pointers) to the arguments onto the stack. And to make matters worse, some compilers put all local variables of functions onto the stack, too. At least the Intel Fortran compiler for Linux allows you to set the maximum size of arrays on the stack. Any array above the specified size ends up on the heap (see woman ifort). With -heap-arrays 0 you tell the compiler to put all arrays (including temporary copies and intermediate results) onto the heap. I am sure there is a performance penalty for doing the right thing. You could tune this option, but with large arrays on the stack you may just postpone the crash.

Re: segfault with reshaping and transposing big arrays

Steven_L_Intel1 — Tue, 12 Feb 2008 15:04:49 GMT

My understanding is that on Linux the stacksize is initially set by a kernel parameter. You can raise the stacksize limit with commands such as:

ulimit -s
or
limit stacksize unlimited

where "unlimited" really means "maximum as defined when the kernel was built". The stack can't exceed 1GB on 32-bit, I believe.

Regarding the -heap-arrays switch - you can leave off the 0 as that is the default. There is a small performance penalty due to the allocate and deallocate calls but my feeling is that this is typically swamped by the routine execution time. You can specify a non-zero parameter, so that allocations under the specified size are still done on the stack, but this affects only those cases where the size is known at compile-time.

If using -heap-arrays means that the program runs whereas without it it doesn't, the performance difference is not relevant.

Re: segfault with reshaping and transposing big arrays

wim_van_hoydonck — Tue, 12 Feb 2008 17:28:10 GMT

Thanks for the answers.

Stack size was 10240 kbytes (Fedora 8 x86_64). Setting it to unlimited (ulimit -s unlimited) solved the problem, as does the switch -heap-arrays.

Greetings,

Wim

Re: segfault with reshaping and transposing big arrays

kristipati — Tue, 18 Nov 2008 03:01:54 GMT

Greetings,

i have a similar problem. I am getting the following error when I compile my code with ifort with optimization -03. It is a fortran code.

I re-ran with unlimited stack size but, this time, the problem fails for larger arrays (larger grids).

Can't extend stack to 0x7FE7BE2B0 during signal delivery for thread 1:
==2255== no stack segment
==2255==
==2255== Process terminating with default action of signal 11 (SIGSEGV)
==2255== Access not within mapped region at address 0x7FE7BE2B0
==2255== at 0x42E7C3: global_mp_read_data_

When I ran with using -heap-arrays 0, it does not compile and throws an error. What is the correct way of using this option -heap-arrays 0 ?

$ mpif77 -O3 -heap-arrays 0 -o 500iterV0.exe V0.f90
ld: 1: No such file: No such file or directory
mpif77: No such file or directory

$ mpif77 -heap-arrays 0 -O3 -o 500iterV0.exe V0.f90
ld: 0: No such file: No such file or directory
mpif77: No such file or directory

I know fortran compiler exists as I am able to compile the code without using -heap-arrays 0 as shown below:

$ mpif77 -O3 -o 500iterV0.exe V0.f90

Pl advise. Thank you.

Quoting - wim.van.hoydonck@gmail.com

Thanks for the answers.

Stack size was 10240 kbytes (Fedora 8 x86_64). Setting it to unlimited (ulimit -s unlimited) solved the problem, as does the switch -heap-arrays.

Greetings,

Wim

Re: segfault with reshaping and transposing big arrays

kristipati — Tue, 18 Nov 2008 03:03:29 GMT

OS Info:

Linux 2.6.12.6 #1 Thu Dec 22 19:30:31 EST 2005 x86_64 x86_64 x86_64 GNU/Linux

Re: segfault with reshaping and transposing big arrays

Steven_L_Intel1 — Tue, 18 Nov 2008 06:49:17 GMT

Just leave off the 0.

Re: segfault with reshaping and transposing big arrays

kristipati — Tue, 18 Nov 2008 14:45:33 GMT

I did that and I was able to compile the code.

But, when I run the valgrind, I still get the stack error.

$ mpif77 -heap-arrays -O3 -o 500iterV0.exe V0.f90

$ mpirun -np 1 valgrind --tool=cachegrind ./500iterV0.exe
==6685== Cachegrind, an I1/D1/L2 cache profiler.
==6685== Copyright (C) 2002-2005, and GNU GPL'd, by Nicholas Nethercote et al.
==6685== Using LibVEX rev 1471, a library for dynamic binary translation.
==6685== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==6685== Using valgrind-3.1.0, a dynamic binary instrumentation framework.
==6685== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==6685== For more details, rerun with: -v
==6685==
g0_1.dat
==6685== Can't extend stack to 0x7FE7BE000 during signal delivery for thread 1:
==6685== no stack segment
==6685==
==6685== Process terminating with default action of signal 11 (SIGSEGV)
==6685== Access not within mapped region at address 0x7FE7BE000
==6685== at 0x42E7C3: global_mp_read_data_ (in /scratch/pavan/thesis/600/500iterV0.exe)
==6685==
==6685== I refs: 958,292
==6685== I1 misses: 3,741
==6685== L2i misses: 3,092
==6685== I1 miss rate: 0.39%
==6685== L2i miss rate: 0.32%
==6685==
==6685== D refs: 368,659 (248,427 rd + 120,232 wr)
==6685== D1 misses: 17,562 ( 10,986 rd + 6,576 wr)
==6685== L2d misses: 11,220 ( 5,126 rd + 6,094 wr)
==6685== D1 miss rate: 4.7% ( 4.4% + 5.4% )
==6685== L2d miss rate: 3.0% ( 2.0% + 5.0% )
==6685==
==6685== L2 refs: 21,303 ( 14,727 rd + 6,576 wr)
==6685== L2 misses: 14,312 ( 8,218 rd + 6,094 wr)
==6685== L2 miss rate: 1.0% ( 0.6% + 5.0% )
-----------------------------------------------------------------------------
One of the processes started by mpirun has exited with a nonzero exit
code. This typically indicates that the process finished in error.
If your process did not finish in error, be sure to include a "return
0" or "exit(0)" in your C code before exiting the application.

PID 6685 failed on node n0 (127.0.0.1) due to signal 11.
-----------------------------------------------------------------------------

Quoting - Steve Lionel (Intel)

Just leave off the 0.