Memory usage decreases when I call a dll by VC++

[Intel_C_Intel]

In my Fortran program (testlarge_only.f as attached) I can allocate a complex array with 248M elements (248M * 8 = 1.984 GB).

allocate(A(length)) ! length can go up to 248*1000000

However,there is a functionGetId ina reg.dll file(The source codes with MS Visual Studio solution is in the folder Reg_Test attached). The function is defined as:

BOOL _stdcall WINAPI GetId(LPTSTR lpOutBuffer, LPCTSTR lpRegisterCode)

{

return TRUE;

}

If I call this function in my Fortran program, and use

ifort testlarge.f entestf.f reg.lib

The generated testlarge.exe can only allocate 161M-element array, occupying only 1.288 GB!

Do you know why there is a discount of the memory usage?

Please note that this test is done in a Pentium 4 machine with 1 GB or 2 GB memory.

Message Edited by jingz on 07-25-2005 09:07 AM

[jim_dempsey]

It is likely due to your heap getting fractured. Can you allocate your largest blocks of memory prior to calling external routines? If you cannot allocate complex array prior to calling GetID then allocate 248MB (+ fudge) to temporary array, call GetID, thendeallocate the temporary arrayjust prior to allocating your complex array.

Depending on your operating system you may only have 2GB of working storage (per application).

Jim Dempsey

[Intel_C_Intel]

If I use

complex

A(248000000) ! Does this mean(+ fudge)?

and call GetId later, itworks.

However, I need to dinamically allocate the array. If I use

complex A(:) allocatable :: A

...

allocate

(A(170000000))

call

GetId

Itdeos NOT work.

I need to allocate the array dynamically in my program. Why is dynamic allocation even worse?

[Steven_L_Intel1]

Jim is most likely on target when he refers to memory fragmentation. C and Fortran use different memory allocators (Fortran sometimes calls the Win32 API directly to avoid some restrictions in C's malloc). He is also correct in saying that the maximum process address space on 32-bit Windows is 2GB (except under unsuual circumstances.)

[Intel_C_Intel]

Many thanks to Jim and Steve.

Now I can: 1)allocate 2 GB if I don't use the C code; 2) allocate no more than 1.3 GB if I use the C code and do dynamic allocation in Fortan; 3) allocate 2 GB if I use static allocation when I use the C code.

Is there any solution of point 2),i.e.,if I have both C and Fortran codes, and I could remain the amount of dynamic allocation up to 2GB?

[Steven_L_Intel1]

I'm not sure I should recommend this, but... You could use the "integer POINTER extension" and the malloc intrinsic. For example:

real bigarray(*)
pointer (p_bigarray, bigarray)
p_bigarray = malloc (number_of_elements*4)

This will use C's malloc (indirectly) and you should not get the pool fragmentation. Maybe. You can now refer to bigarray normally, though you don't get bounds checking.

I will comment that pushing the 2GB limit can cause unpredictable results - your program may work on some systems and not others, or work one day and not another. It all depends on what else is loaded into your address space by Windows.

You may also want to experiment with the order in which you do the allocations, which may make a difference.

Or consider using the Win32 API VirtualAlloc for both C and Fortran.

[jim_dempsey]

What I am suggesting you use is

complex A(:)

allocatable :: A

real(8), allocatable :: dummy(:)

program YourProgramName

! in beginning of program before any other calls

! e.g. first execuitable line in main

allocate(dummy(248000000+256))

call InitializeProgram ! Including Getting Array Size

... ! other startup but pre-allocation code

! Now allocate big honk'n array to BigHonknSize

deallocate(dummy) ! free up humoungous memory block

allocate(A(BigHonknSize))

[jim_dempsey]

Additional note:

Now after thinking about it the following is my assumption of what is going on with your "missing" memory.

Prior tothe first call to a C routine that allocates memory the C heap manager has not yet allocated memory from Windows. At this point your FORTRAN program could allocate all of the virtual address space remaining. However, instead you call a C routine that in turn allocates memory from the C heap (heaps). The C heap manager, finding no memory available in its heap (heaps) allocate what appears to be 700MB for a heap. Then the C memory allocation obtains memory from this heap. This C memory heap is now memory that is not available to your FORTRAN program.

However, now note that if your FORTRAN program gobbles up enough memory for its future needs _prior_ to calling the C routine that allocates memory then returns this memory just prior to it's major allocation then your allocation will succeed. Do not be too greedy in your dummy allocation as the C modules will require some working memory (but hopefuly not 700MB).

Also, depending on the C heap manager it may use an environment variable to restrict the maximum C heap size.

Jim Dempsey