Segmentation fault when reading in a large array from a text file

[classic35mm]

Hi,

I have compiled the following code using ifort (with the default settings). It appears to compile fine; however, when I run the resulting file (a.out), I get a Segmentation fault. Can you please help me think why this happens?

I also compiled it using another brand of Fortran compiler, and it compiled and ran fine there. To me, this seems to suggest that there is probably an option in ifort that I am not using but that I should be. Can you please help me think what option this might be?

Below is my complete code. I am trying to read in an external file (testdata.txt) to an array, and then write this array to an unformatted Fortran stream file called ustream.demo. The file testdata.txt is very large (~78 MB); it contains a 5001-by-2287 array of real numbers--each row printed on a single line of text--delimited by spaces.

I am usingIntel Fortran Intel 64 Compiler Professional for applications running on Intel 64, Version 11.0 Build 20081105.

Thank you very much for your time and any help that you can give. I truly appreciate it.

Andrew DeYoung

Carnegie Mellon University

[fortran]MODULE july28subs

  IMPLICIT NONE

CONTAINS

SUBROUTINE tick(t)
  INTEGER, INTENT(OUT) :: t
  CALL system_clock(t)
END SUBROUTINE tick

! returns time in seconds from now to time described by t
REAL FUNCTION tock(t)
  INTEGER, INTENT(IN) :: t
  INTEGER :: now, clock_rate

  call system_clock(now,clock_rate)

  tock = real(now - t)/real(clock_rate)
END FUNCTION tock

FUNCTION loaddata(fname, numrows, numcols)
  CHARACTER(LEN=14) :: fname
  INTEGER :: numrows, numcols
  REAL, DIMENSION(numrows,numcols) :: loaddata
  INTEGER :: i, j

  OPEN(UNIT=11, FILE=fname)
  DO i=1,numrows
    READ(11,*) (loaddata(i,j), j=1,numcols)
  END DO
  CLOSE(UNIT=12)

END FUNCTION loaddata

END MODULE july28subs

PROGRAM july28
  USE july28subs
  IMPLICIT NONE

  ! Declarations
  INTEGER, PARAMETER :: numrows=5001, numcols=2287
  INTEGER :: i, j, myclock
  REAL :: mytime
  REAL, DIMENSION(numrows,numcols) :: data, dataoutput
  REAL, DIMENSION(numrows) :: time
  CHARACTER(LEN=14) :: fname="testdata.txt"
  
  CALL TICK(myclock)
  data=loaddata(fname,numrows,numcols)
  PRINT *, "loaded"

  OPEN(UNIT=12, FILE="ustream.demo", STATUS="REPLACE", ACCESS="STREAM", FORM="UNFORMATTED", ACTION="WRITE")
  WRITE(12) data
  CLOSE(UNIT=12)
  PRINT *, "Finished writing."

  OPEN(UNIT=13, FILE="ustream.demo", STATUS="OLD", ACCESS="STREAM", FORM="UNFORMATTED", ACTION="READ")
  READ(13) dataoutput
  CLOSE(UNIT=13)
  PRINT *, "FINISHED READING OUTPUT."

  PRINT *, data(1,1)
  PRINT *, dataoutput(1,1)

  mytime=tock(myclock)
  PRINT *, mytime, " sec"

END PROGRAM july28[/fortran]

[mecej4]

Your function LOADDATA may, depending on the compiler options used, cause a large (over 140 Mbytes) local array to be allocated on the stack. A stack overflow can occur here, if the program is not linked with an option to specify the required stack size or if the OS imposes a default stack limit that is smaller than the needed amount.

After the data is read into that variable, the statement that assigns the function return value to the array DATA may cause an array copy to be performed. Unless you have specific reasons to make the function LOADDATA to return a huge array, one solution is to convert the function to a subroutine with the currently returned array as an additional local argument. For example:

call loaddata(fname,data,numrows,numcols)

in place of

data = loaddata(fname,numrows,numcols)

[Steven_L_Intel1 · Accepted solution]

You can also try the compiler option -heap-arrays

This tells the compiler to allocate temporary array copies on the heap rather than on the stack.

[Ron_Green]

more details on -heap-arrays and seg fault is here: http://software.intel.com/en-us/articles/determining-root-cause-of-sigsegv-or-sigbus-errors/

[Izaak_Beekman]

If you're on a Linux machine (like RHEL 5) you can view the system limits with the `ulimit -a` command. Specifically looking at a limit on OS imposed stack size: `ulimit -s` You can now change this by typing `ulimit -s ` or `ulimit -s unlimited` to set the stack limit to be unlimited. This may very well work under other *NIX systems too, like mac OS. Add a line to your .bashrc file to make the changes permenent. Some may consider this to be an ugly hack but since this creates a rather cryptic error message I find it saves me hours of frustration.