fortran 9.0 for linux fail to read binary file

[luxiaod]

Hi,

I have a fortran 90 code which runs well when use other compiler and also the older version of intel fortran. I am also using mpich2.

The code reads the binary data file saved from matlab. The binary data file is

about 32 MB, which is supposed to be read to several arrays, some of them are integer arrays and some are double precision arrays.

The segment of the code like this:

open(IO_inputData,FILE=inputData,IOSTAT=IERR,FORM='BINARY')

read(IO_inputData,IOSTAT=IERR) ...some integersdefine array length...ok

ALLOCATE some memory for the arrays ...ok

read(IO_inputData,IOSTAT=IERR) ...some other integer arrays...ok

fine, I can get the integer arrays as expected, then something went wrong

read(IO_inputData,IOSTAT=IERR) ...some double precision arrays

"Return code = 0, signaled with Segmentation fault"

This code worked on other compilers and older version 8.0 e.g. ifort.

I checked all the arrays are within the bounds.

Did version 9.0 change some default on read binary or something?

Any suggestions please...

[Steven_L_Intel1]

Try raising your stacksize limit and be sure that you are using a current compiler (9.0.033).

[luxiaod]

Hi, steve, I use ulimit to change the stack size from 10MB to 10GB. It works. Thanks.

I don't quite understand this stack size issue. My system has 128 GB memory and it is installed GNU/Linux 2.6.9-SMP with 8-way dual-core processors. Does it mean that each processor will have 10GB stack size to hold the user data? then I think it will be too much, 16x10GB =160 GB>128GB. How much stack size I should use?Should it be bigger than my application data?

Another thing is that I already used the allocate() for the array, like:

INTEGER(4),POINTER :: array1(:), array2(:) ...

REAL(8), POINTER :: array3(:), array4(:)

ALLOCATE(array1(...))

ALLOCATE(array2(...))

....

why I still need to use the stack? In my situation, the arrays are allocated on the stack?

Thanks a lot for your help!

[Steven_L_Intel1]

Actually, the stack can't grow to more than 1GB, no matter what you set it to, and it is just a limit. The actual stack usage (which is virtual memory, not physical) will depend on what the process is doing at the time.

You did not show your actual code. If you put something like array(:) in the READ, that could cause the compiler to think it needed to create a temporary copy. We've been working at eliminating some of these unnecessary copies, and newer versions of the compiler are better.

[luxiaod]

Steve, I am confused. We you say stack, is itreferring tostack and heap memory which are physical or those swap memory which are virtual?

I thought I used the Allocate() and it should be in heap and should be physical.

I checked my system and I found the following when I use ''top":

Mem: 131893864k total, 40119220k used, 91774644k free, 522960k buffers
Swap: 32009504k total, 0k used, 32009504k free, 32748564k cached

It seems the Swap is created on physical memory also. Am I correct?

My code segment:

INTEGER(4),POINTER :: themask(:,:,:),themask2(:,:,:)

REAL(8), POINTER :: dijs(:)

INTEGER (4) xx, yy, zz

open(IO_inputData,FILE=inputData,IOSTAT=IERR,FORM='BINARY') ...OK

read(IO_inputData,IOSTAT=IERR) numdijs,xx,yy,zz ...OK

! numdij = 1065755, xx=180,yz=180,zz=85

ALLOCATE(themask(xx,yy,zz),STAT=tt) ...OK

ALLOCATE(themask2(xx,yy,zz),STAT=tt) ...OK

ALLOCATE(dijs(numdijs),STAT=tt) ...OK

read(IO_inputData,IOSTAT=IERR) dijs, themask, themask2 ...FAILED

! the inital stack size is 10MB. I set it 100MB, code works.

I am not sure what happens when I read the data from a binary file (34MB).

Does it need the virtually memory to do the read? I have much free memory to use. Still confused.

Thanks, steve.

[Steven_L_Intel1]

From the perspective of your application, all memory is virtual. The OS manages the physical memory - more physical memory improves performance.

See thi s article I wrote on the difference between heap and stack.

Your ALLOCATABLE variables are always allocated on the heap. Sometimes, though, the compiled code wants to make a temporary copy of some data - for example, an array slice used in an expression. In the current implementation, it always creates these temporary copies on the stack. We know that this can cause problems, especially as problem sizes get bigger, so we have work underway to allow larger temps to be allocated on the heap.

In the case of I/O statements, sometimes the compiler thinks it has to make a copy of an array (for example, if you READ into an array slice). Sometimes it does this even when it really doesn't need to - we try to identify such cases and skip making the copy if possible. Over the years, the compiler has gotten better about this, but I have seen that some cases where if you say A(:) instead of A, you get a copy. This seems obvious to humans, but the compiler represents these in different ways (semantically they ARE different) and it's sometimes hard to notice that they can be treated the same. It's a work in progress, and if you have an example that shows us getting it wrong, please send it to Intel Premier Support.

[sabdel]

Hi,
I have a similar probem to the yours, concerning the stack size limit (my code crashs when I rise some array dimension with the error message "erreur de segmentation") , in order to resolve the problem I have to increase the stack size limit, could you tel me how do you do to do that? I appreciate your help.
thanks.