Memory leak - array of derived types and type-bound procedures

[ZlamalJakub]

I am using arrays of derived types for computation. I prepared routines for addition of two arrays of derived types

interface operator(+)
   module procedure dar_add_dar,&   ! scalar

        dar_add_dar_v   ! vector version
end interface

when I run test code on scalar derived types, it is OK. There is a memory leak (the number of allocation of memory is not the same as number of deallocations) only when vector versions of operators are used.

subroutine RunAddArrayMemoryLeak  cause memory leak.

Can someone help me to correct my code?

Attached is Visual Studio 2022 project (Windows)

Problem is when compiling using  IFX 2024.0.2 and  IFORT 2021.11.0 too.

[andrew_4619]

    function dar_add_dar_v(a,b) result(apb)   ! vector version for arrays
        implicit none
        type(DAr),intent(in) :: a(:), b(:)
       ! type(DAr) :: apb(size(a))
        type(DAr),allocatable :: apb(:)
        integer :: i, istat
        write(*,*) 'dara_add_dar_v'
        allocate( apb(size(a)), stat = istat )
        if(istat /= 0 ) write(*,*) 'vecalloc fial',istat
        do i=1,size(a)
            write(*,*) 'i=',i,'   before add'
            apb(i) = dar_add_dar(a(i),b(i))
            write(*,*) 'i=',i,'   after add    apb(i)cnt',apb(i)%order
        end do
        write(*,*) 'dara_add_dar_v finished'
    end function

I made apb allocatable in dat_add_v rather than an automatic array after the function end and the assignment is made it will be deallocated automatically.  I did global edit out all non-standard tabs and integer*4  ( integer(4)  ) and set standard checking on

[ZlamalJakub]

Dear andrew_4619,

   thank you very much for fast solution of my problem.

But when I try to increase size of array from 1 to n, then memory leak appears. And it seems that (n-1) array items is not deallocated. Can you help me also with this problem?

subroutine RunAddArrayMemoryLeak()
! derived type is an array
use types
implicit none
    integer(4), parameter :: n =3
    type (dar)  y1(n),y2(n)
	 type(dar), allocatable :: y3(:)
    integer(4) i
    
	 do i=1,n
      call y1(i)%init()
      call y2(i)%init()
    enddo
    y3=y1+y2                    ! add types as vectors
    write(*,*) 'y3',y3(1)%order ! value should be 4

    do i=1,n
       call y1(i)%deinit()
       call y2(i)%deinit()
       call y3(i)%deinit()
	 enddo
    write(*,*) 'end of routine'
end subroutine

[andrew_4619]

If i substitute the new RunAddArrayMemoryLeak then it doesn't run "forrtl: severe (408): fort: (8): Attempt to fetch from allocatable variable Y3 when it is not allocated" at line 14 which suggests the "=" assignment does not allocate LHS. If I pre-allocate y3 it runs but we get "test is not OK" I did not dig and further at this point.

EDIT I am note sure if this program is conforming or not, or if there is any compiler bug or not. I am also not sure what this program is really trying to test. If you have all your locals of type(dar) as allocatable when you exit the routine they will automatically be deallocated and any allocatable components likewise. So I am not sure you actually need the destructor in that case. Also any automatic deallocations will not call your destructor so the CNT variable can then do wrong?

[JohnNichols]

I have tidied up your output statements so they make some modicum of sense.  

I added in the memory leak test and got the following 

 integer(4), parameter :: n =3
 type (dar)  y1(n),y2(n)
	 type(dar), allocatable :: y3(:)
 integer(4) i
 
	 do i=1,n
   call y1(i)%init()
   call y2(i)%init()
 enddo
 y3=y1+y2                    ! add types as vectors
 write(*,*) 'y3',y3(1)%order ! value should be 4

The error would appear to be between line 3 and line 10.   It is somewhat self explanatory.  

[ZlamalJakub]

I am sorry I forgot that I made another change in code so it does not work as I expected. I am including whole project now. Sorry for wasting your time with a broken example. 

My test code is very simplified part of code with much more complicated derived types which are nested one in another as allocatable arrays. Code works, but causes memory leaks so it cannot be run for long time. Code itself does not use destructors but when I implemented it, memory leaks were smaller.

My intention is to find what is wrong in the current implementation and fix problems. May be it is compiler problem. Maybe someone from Intel will know what the problem could be. I'm not an expert in this area and rather try different changes.

[JohnNichols]

subroutine RunAddArrayMemoryLeakB()
! derived type is an array
use types
implicit none
    integer(4), parameter :: n =3
    type (dar)  y1(n),y2(n)
	 type(dar), allocatable :: y3(:)
     
    integer(4) i
    
     allocate(y3(3))
	 do i=1,n
      call y1(i)%init()
      call y2(i)%init()
    enddo
    y3=y1+y2                    ! add types as vectors
    write(*,*) 'y3',y3(1)%order ! value should be 4

    do i=1,n
       call y1(i)%deinit()
       call y2(i)%deinit()
       call y3(i)%deinit()
	 enddo
    write(*,*) 'end of routine'
end subroutine

Without line 11 your program does not execute properly

It is much easier to follow with the cleaned up write statements.  When you get to the last return CNT, then nothing exists in the locals watch, I think you are miscounting, although I am not going to chase it through.  

Why you developed 300 lines of code for really only a few dozen lines with complexity beyond the earth is beyond me.  

Code should be simple and readable - Fortran is a gift, please don't throw it off the cliff. 

I think you are mistaken. 

[ZlamalJakub]

Dear JohnNichols, thank you for your effort.

When I run your code in Intel Inspector, it shows memory leak at line 16. So I suppose that counting allocation and deallocation work and there is a problem in the code.

My code is only important part dealing with allocation and deallocation of derived types, there is a lot of math implementing differential algebra computations in each derived type. I removed it for simplicity.

The complexity of original code is also beyond me. But this implementation using derived types allows computation with differential algebras using simple mathematic operators. Here I implemented only addition.

[JohnNichols]

I played with it a bit this morning, I can see what you mean, but I think your problem with y3 = y1+y2 is buried in the functions that this + calls. 

No code is too complex, it is just poorly written, if it cannot be read simply, if you are a Lisper you understand that point.  There is a special place in hell for the people in who revel in the complex C constructs that take 20 minutes to denut in your head.

I suggest you put in three log files as output.  Write the stuff I did to one, write the y1 - order and a count of where it comes from for the second and you should add some real values to coefficients and they get tracked in log 3.  It is likely to be a miscounting or a dealloc mistake somewhere.   You can find it, but you need more output to pin down the exact line and it is not the line y3=y1+y2, it is within the stuff that creates it.  

Plus please make the output nice to read, your stuff only yields headaches, it has to have a logic to it.

Does the whole code run?

[andrew_4619]

indeed the essence of the code is in the + and = operators.  In reality  after the y3=y1+y2 the y' s are all x3 arrays of type dar so we should have a net 9 allocations after this add but we have more if we follow the cnt variable. It is a logic bug I think but I don't have any time this week to figure it out..... 

[jimdempseyatthecove]

>>...after the y3=y1+y2 the y' s are all x3 arrays of type dar so we should have a net 9 allocations after this add...

Fortran requires the rhs to be completely evaluated before lhs is set, so would this not require a temporary be created (12 allocations)?

(While compiler optimizations could reduce this to 9, it may also not do so, especially in debug build).

Jim Dempsey

[JohnNichols]

There is a logic error somewhere, if the programmer uses the logs and we can see the output, it should become obvious.  

But I agree the addition is more complex than coded. 

[andrew_4619]

That isn't the point Jim, the code is instrumented to track  alloc/deallac then so the net result of the count should be 9 and isn't

It then dellocates 9 items and says they are not all done....

[Steve_Lionel]

This code is more complex than I want to try to figure out, but I'll mention, not being sure if it is relevant, that you don't get automatic allocation on assignment with defined assignment.

[JohnNichols]

The alloc and dealloc works until you add in the addition, something is happening inside the addition to play with the alloc's.  The programmer needs to put in some effort to learn, we do not do them any favours by fully solving the problem.  

Let us not take away the fun of discovery. 

[JohnNichols]

subroutine dar_to_dar_v(lhs,rhs) ! vector version for arrays of dar
    type(DAr),intent(in) :: rhs(:)
    type(DAr),intent(out) :: lhs(size(rhs))
    integer::i
    write(*,*) '              Vector version for dar_to_dar_v'
    do i=1,size(rhs)
        write(*,*) '              i=',i,'   Position before dar to dar'
        call dar_to_dar(lhs(i),rhs(i))
        write(*,*) '              i=',i,'   Position after dar to dar'
    end do
    write(*,*) '              Routine dara_to_dar_v  finished'
end subroutine

line 3 assigns a size of 1 to lhs instead of a size of three.  I have no idea why size(rhs) does not come back as three, but single stepping through it shows this error, I checked it more than once,  you should be able to fix it from here.  

@mecej4 taught me to use a module with a set of integer variables that you use for sizing arrays so they are only in one spot.  I would suggest you remove all size spots and do 

integer parameter  mn_3 = 3 

or along those line.  No idea why @mecej4 used mn_, but I have stuck with through thick thin and the odd dropped coke.  

[andrew_4619]

@JohnNichols John, way back up thread after I pointed out some problems the OP reposed the whole project with some corrections. The code snippet you made above is from an older version with errors.

@ZlamalJakub  why not removed the defined assignment, intrinsic assignment should work OK in my opinion. If it doesn't then  there is a compiler bug. Also be cautious with Inspector it sometimes gives false positives for memory leak. You could instrument the code with some system calls to actual track memory usage. 

[JohnNichols]

@andrew_4619 , sorry mate, missed that one.  I am really busy at the moment, so I will add you corrections to the code and try again, but it will not be before Saturday, just a bit busy with accelerometers and 200 yr old gunboats.  

It is an interesting little problem.  

[ZlamalJakub]

Thanks to everyone for the ideas and comments. I won't be able to deal with the problem until the end of the week. I'll let you know what I find out.

[ZlamalJakub]

The results of my investigations of calculations and assignments with derived types:

1.Using

interface assignment (=)​

and defining assignment functions for derived types (at least in my case) does not work for arrays of derived types. It leads to the allocation of the left side inside of assignment function and copying RHS to LHS. But FORTRAN then uses its own management for the assignment of result anyway, so the derived type LHS is allocated unnecessarily in function.

In addition, I have not been able to find a way to define the assignment function in such a way that it does not lead to a memory leak for arrays of derived types.

2. To assign LHS=RHS in code FORTRAN uses the following process (observed in disassembly):

1. calls the destructor (for_finalize function) for LHS, so if it is already allocated, it is deallocated.
2. the assignment itself is then performed by calling the for_alloc_assign_v2 function. This function does not call the LHS constructor but apparently copies the RHS. Therefore, it makes no sense to count constructor and destructor calls because they cannot match.

Attached is my test code (with commented write statements) which does not lead to memory leaks.

Thanks again for your comments on my test code and also for the time spent debugging it.

[andrew_4619]

Interesting I will look at this when I have time. Do you think the compiler behaviour is wrong or it is non standard coding. It seemed ok to me BTW.