Dynamic allocation help

dajum · ‎05-26-2006

That last one fired off by itself 3 times somehow. I'm getting error 41 while trying to allocate an array of Real*4 of about size 230,000,000 This has been working fine in CVF and 3 other compilers. But it isn't under IVF. I have /STACK set to 100,000,000 due to an issue I can't recall, and couldn't look up cause I can't get into premier support. The error 41 description is vague. What exactly is the "data limit". I don't want to try and guess the answer to this by playing with options since the problem takes over an hour to get to the point where the error occurs. Is there somewhere to look to see why this is failing without running the program. I couldn't really see it in the .map, but maybe I'm not looking in the right place. Thanks in advance.

Dave

jimdempseyatthecove · ‎05-26-2006

Depending on the version of IA32Windows your application may have 2GB or 3GB of virtual memory available. Assume the lesser. 230 million real 4's are almost 1GB. If this array is a growing array where you allocate a larger array and then copy the older smaller array, and delete the older smaller array,then due to memory fragmentation (from allocation, deallocation), you would expect to see a failure when you reach 1/2 of the available virtual memory.

Pre-allocating the data early on in the program, to the maximum required size, will help to alleaviate this problem.

dajum · ‎05-26-2006

Thanks, I've already accounted for all of those issues. I find now that I can only

get an array of 194M entries (real*4) at the start of the program now. But I can't see where this limit comes from when I look at the .bss size and other addresses. Even a simple 10 line program without any data is limited to

a 332M Real*4 array. These sizes don't add up in my mind. They are consistent once the program has been compiled, but I don't see where the relationship comes from that limits the size of the allocatable array to about 1.2 GB - whatever size the .bss segment is. Surely there isn't 700MB of overhead. So I'm still looking for the majic word that increases this limit.

Dave

Steven_L_Intel1 · ‎05-27-2006

There's no magic word. Sometimes the library/OS can't find that much contiguous memory. Or, Windows has decided that your system configuration can't support that much virtual memory (how big is the pagefile? How much RAM do you have?)

As an experiment, try calling the Win32 API routine VirtualAlloc and see how much you can get. I just did an experiment and was able to allocate 1,900,000,000 bytes without error.

If you still need help, send a sample to Intel Premier Support.

dajum · ‎05-29-2006

My swap file is set to the max of 4095 MB, I have 2.5 GB of RAM. If I watch the process on the task manager ( I run WIN2K), it always seems to peak at a VM size of a little more than 700MB. I can still add a common block of 290M entries, and then dynamically allocate a 55M word array. But my base project seems to be limited to allocating 194M array, which seems to tie in to the VM size.

jimdempseyatthecove · ‎05-30-2006

Code:

    program Alloc

    implicit none

    ! Variables
    real(4), allocatable :: array(:)
    integer :: iSTATUS
    ! Body of Alloc
!    400000000 works
!    allocate(array(400000000), STAT=iSTATUS)
!    500000000 fails
!    allocate(array(500000000), STAT=iSTATUS)
!    450000000 works
!    allocate(array(450000000), STAT=iSTATUS)
!    475000000 works
!    allocate(array(475000000), STAT=iSTATUS)
    allocate(array(475000000), STAT=iSTATUS)
    write(*,*) istatus
    end program Alloc

475 million works on my system. Did not wish to take the time to find the uppermost limit.

ifort version:

Jugoslav_Dujic · ‎05-31-2006

A long shot: start the debugger, having placed the breakpoint somewhere near the start, and open Debug/Windows/Modules. That will give you the load map of modules. It's just possible that you have some dll's sticking in the middle of the address space, making the gaps smaller than they should be.