Another memory leak

[qolin]

This one is probably my fault, but let's see what others think. See attached.

The constructor declares its argument as intent(out). When you run it, the result is a leak that (on my PC at least) quickly exhausts available memory. Changing it to intent(inout) causes error 151, allocatable array is allready allocated.

[mecej4]

Perhaps the phrase "memory leak" should be applied only in situations where there is no possibility to staunch the leak, i.e., an irreparable memory leak. This used to happen in Fortran 90, and can still happen when (i) memory is allocated using a pointer variable, and (ii) the pointer variable is subsequently associated with something else or becomes dis-associated before the associated memory is deallocated.

The variable stuff is a formal argument to subroutine construct_stuff; it is not a local variable. Therefore, the rules regarding automatic deallocation of local allocatable variables do not apply; it is the programmer's responsibility to manage the allocation of variables that are passed back and forth to subroutines, either as arguments or through host association.

In fact, the compiler has no way of determining when variables such as stuff  are.safe to deallocate except, of course, when the main program or the subprogram where stuff  is a local variable executes a RETURN or STOP statement.

The rules regarding what INTENT(IN), INTENT(OUT) and INTENT(IN OUT) are a bit subtle in the context of allocatable arguments. In particular, does INTENT refer to the allocation status of the variable or to the value(s) of the variable? Please read a Fortran manual about the details.

[qolin]

I have a number of Fortran manuals and books available to me here (yes real paper ones as well as disk files) but alas none of them go into sufficient detail on the workings of INTENT(OUT) allocatable arguments.

Your posting therefore prompted me to do an internet search, and I found the draft of the Fortran 2003 standard at http://www.j3-fortran.org/doc/year/04/04-007.pdf. On page 116 of this I see:

9 When a procedure is invoked, any allocated allocatable object that is an actual argument associated with

10 an INTENT(OUT) allocatable dummy argument is deallocated; any allocated allocatable object that is

11 a subobject of an actual argument associated with an INTENT(OUT) dummy argument is deallocated.

[John_Campbell]

If you read the preceeding lines 5 to 8, my interpretation of this would be the INTENT(OUT) argument is deallocated before entry to the procedure, not on return. This refers to there is no IN intent and would not occur with INTENT (INOUT).
I find that changeing from the default of INTENT (INOUT) should be used with caution.
A solution to your problem might be to use the default INOUT and test if the argument is allocated then respond accordingly.

John

[IanH]

(As far as I can tell this is another "WTF??" class bug (assuming the struct allocatable argument is INTENT(OUT), otherwise the program is broke as per the runtime error message).  I'm mildly stunned that this bug (if my understanding is right), and the previous one, have been hidden for so long - I'm sure I've used similar constructs in my code.)

John's description is correct, but a quibble - INTENT(INOUT) isn't quite the same as having no intent (if that's what was meant by "default"), there are differences - such as an INTENT(INOUT) actual argument must be definable - you can't pass a constant to an INTENT(INOUT) argument (the compiler will more than likely give you a compile time error), but you may be able to pass a constant to an argument with no intent specified (you are unlikely to get an error message if a constant is not appropriate). 

Consequently if INTENT(INOUT) fits the situation, it's a good idea to explicitly specify it.

[qolin]

Anyone from Intel got anything to say about this?

[Steven_L_Intel1]

As I was at the Supercomputing show last week, I have not had a chance to look at this yet. I will do so soon. It's on my list.

[qolin]

Steve, any advance on this?

[jimdempseyatthecove]

qolin,

The standard C heap manager (the underliying heap manager for Fortran), does perform memory consolidation of returned nodes. If you want this feature, you must enable the "Low Fragmentation Heap" (LFH). You can find information on this on MSN.

In looking at your reproducer, the allocation and deallocation pattern is:

ii=1
allocate one stuff_type (includes array descriptor, for your example depending on version ~9*sizeof(void*)): 36 or 72 bytes
allocate 800 bytes

ii=2
deallocate 800 bytes
deallocate 36 or 72 bytes
allocate 36 or 72 bytes (likely get same address as when ii=1)
allocate 1600 bytes (***starting after freed node of 800 bytes)

...

Each allocation will creep along the heap, thus exhausting virtual memory.

The solution to this is to enable the Low Fragmentation Heap (if this is possible for the IVF).

*** however, the information on the LFH states when enabled, it is not used for allocations larger than 16KB. Therefore, your reproducer program may always exhibit this symptom... unless you replace the heap manager used by IVF. You can Google for this on the internet (look at Stackoverflow.com and sourceforge.com).

Also note, if you were to run the program's loop in reverse (largest allocation to smallest), the program would likely complete the loop the first time run, but fail on the second time run (in same run of application).

Jim Dempsey

[Steven_L_Intel1]

Sorry for not getting back to this. I tried the program in 14.0.1. As written, with intent(inout), it properly gives an "already allocated" error as noted by mecej4 and Ian. But with intent(out), it would appear that struct%v isn't being deallocated when struct is deallocated due to the intent(out). I will let the developers know about this.

[qolin]

Thankyou Steve.

[Steven_L_Intel1]

Issue ID is DPD200253045.

The workaround is to make the argument INTENT(INOUT) and to explicitly deallocate it if allocated. I know you shouldn't have to do this....

[jimdempseyatthecove]

One of the other threads on IDZ indicates, for now, when deallocating a type with nested types and allocations, that you would be safer to manually deallocate from inner to outer nest levels. This is supposed to work automagicly, apparently, in some situations it does not.

You might be able to fix this easiest with a FINAL routine in the types with allocatables.

Jim Dempsey

[IanH]

I don't understand how this can have gone undetected... did this break when the fix for the corresponding issue for polymorphic INTENT(OUT) things was added a year or two back?

[Steven_L_Intel1]

I was wondering the same thing - there have in fact been two (that I know of) similar bugs with polymorphic INTENT(OUT) arguments that needed separate fixes. It does seem odd that this didn't get noticed sooner. Maybe when the bug is analyzed there will be some clue about that. I will also try some much older compilers and see what they did.

[Steven_L_Intel1]

We've fixed this and I expect the fix to appear in update 3. This problem related to a scalar allocatable INTENT(OUT) dummy argument of derived type - in some cases, allocatable subcomponents did not get deallocated.