Finalization is not deallocating everything as intended

[AThar2]

Hello there,

I have tried for sometime to analyze what's going on regarding my finalization subroutine - with no luck. Let me start showing the bit of the code you need to understand what I am trying to and hopefully spot my bug. So firstly here are my declared types, (i have made them a bit simpler just to illustrate the point)

      type root_node
         type(nodes), pointer   :: root => null()
         contains
         final                          :: DESTROY_ROOT
      end type root_node

      type nodes
         type(nodes), pointer  :: parent      => null()
         type(nodes), pointer  :: children(:) => null()
         type(nodes), pointer  :: leaf        => null()
      contains
         final                         :: DESTROY_node
      end type nodes

      type leaf
         integer, allocatable          :: id(:)
         integer                       :: num_point
      contains
         final                         :: DESTROY_LEAF
      end type leaf

Now I will show the the three final routines, the main is DESTROY_ROOT which will be calling the other two, and eventually destroy everything.

      subroutine DESTROY_ROOT( this )

      implicit none

      type(ROOT)               :: this
      integer                      :: i

      if(.not. associated(this% root% children) )RETURN
      deallocate(  this% root )


      end subroutine DESTROY_ROOT

      recursive subroutine DESTROY_NODE ( this )

      implicit none

      type(t_octree_node)           :: this
      type(t_octree_node), pointer  :: node => null(), next => null()
      integer                       :: i
      integer, save ::debug=0


      if( associated(this% leaf) )then
         deallocate( this% leaf )
         nullify(this% parent)
         RETURN
      endif

      do i = 1, 8
         node => this% children(i)
         debug = debug + 1
         print*, i, debug
         deallocate( node )
      enddo
      nullify( this% children )
     ! deallocate(node%children) commented as it does not work  - crashes!
     end subroutine DESTROY_NODE


      subroutine DESTROY_LEAF ( leaf )

      implicit none

      type(LEAF)        :: leaf
      deallocate( leaf% id )

      end subroutine DESTROY_LEAF

When I use my monitoring tool on my laptop to check the memory consumption it is observed that the memory declines when the deallocation process begins, however, it seems like a lot is still left without being cleared. I tried to compare this to a normal subroutine which clears the memory and it was clear that the entire memory allocated was being freed again.  The only difference with those two example were that the leaf type was not considered, instead all children had a "id" allocatable. So that clearing is basically done by 

  recursive subroutine DESTROY_NODE2(node)

    type(node_type), intent(inout) :: node

    integer i

    if (associated(node%children)) then
      do i = 1, 8
        call clean_node(node%children(i))
        deallocate(node%children(i)%id)
      end do
      deallocate(node%children)
    end if

  end subroutine DESTROY_NODE2

As you observe there is an deallocate(node%children), which does not work if I place it my DESTROY_NODE routine. Note again that DESTROY_NODE2 is not a final routine, but just a normal routine called by  " call DESTROY_NODE2 ( node )". 

Does anybody see what I am doing wrong please?

Happy new year !!

[FortranFan]

@AT90,

You may want to put together a *reproducible* example that clearly shows the memory leak using some analysis tool e.g., valgrind.  You can then also submit a support request at the Intel Online Service Center, https://supporttickets.intel.com/?lang=en-US

I, for one, am unable to follow your code snippets; they also appear incorrect perhaps due to incomplete/inaccurate copy-paste steps from your actual code e.g., your first type has a different 'end type' name

[AThar2]

@FortranFan I am still working on it, I made a small code but that one seemed to deallocate the stuff as intended, i.e. it deallocates everything, and by using the exact same final routines shown. The difference is that I manually just create one leaf with many empty children instances. So the bug seems to be somewhere in the construction of the data tree. I am trying to reproduce a small case with that one involved. If it remains problematic I will utilize the support service. I don't use any of these tools, and don't know if my Intel license give me access to those analysis tools. For now I am just using my the "activity monitor" on Mac to check how the memory varies and using the pause features in fortran to make sure that I follow the updates in the monitor window precisely. 

You are right about the typo,  

[Steve_Lionel]

You can't rely on operating system reports of memory used. Deallocating an item doesn't return memory to the OS - it goes back to the available pool for that process. What you should do is instrument your finalizer routine and have it record/print what is being deallocated. Do the same for where you allocate and see if anything is missed. Note that objects allocated in the main program don't get finalized.

Intel Inspector XE has a "memory leak" tool. If you don't have a Parallel Studio XE Professional Edition license, you can get a 30-day free trial.

[AThar2]

Hello Steve,

Thanks for your answer. I will try the intel Inspector to see. My concern was that when I did the simplified example I saw my OS system reporting that the memory were deallocated (what I saw was that the job id consumed the same memory as before I did allocate the objects - while for the other case it wasn't). But as say if I can't rely on it, then I should rely on this difference observation as well. 

[AThar2]

Seems like it is not support for Mac IOS, is that correct 

[Steve_Lionel]

Could be - I'm not as familiar with the MacOS products. 

[AThar2]

In my DESTROY_NODE routine I do this : 

      do i = 1, 8
         node => this% children(i)
         debug = debug + 1
         print*, i, debug
         deallocate( node )
      enddo

When I put a 

print*, associated(this%children) 

right after the loop (i = 1,8) I still get "T", i.e. true. Does it mean that this% children has still not been deallocated. 

I have tried the following:

1) instead of using an auxiliary pointer "node" I directly do the deallocation by :

deallocate(this%children(i))

But that gives me an error saying " Not a proper allocate-object for the DEALLOCATE statement. [THIS]" 

2) 

I did monitor how many instances of all declared types I allocated, and I get the exact same number of the times I invoke the final routine for each respective object. 

[AThar2]

Note that the way I allocate "this% children" is buy doing "allocate(this% children(8))", and each of those children can also have a allocate with always 8 instances. 
3) I tried to place a deallocate(this% children) after the loop but then getting was message from malloc saying I am trying to free memory that has already been freed, however, the "this% children" is still associated. 

Please notice I tried to make a program that keep allocate and deallocate the process over multiple times, and eventually my program crashed due to memory problems. So, it seems that my deallocation is not functioning. 

[Steve_Lionel]

You can't deallocate this%children(i) as that is not a pointer. this%children is a pointer to an array, but you'll have to chase down the array and deallocate each component bottom-up.

At this point you keep showing us pieces but not a whole program. Come up with a complete program we can build and run that doesn't do what you want, and we'll look at it. Right now you're just fighting with yourself.

[jimdempseyatthecove]

You really need a working  (non-working) reproducer. Your sample code has naming inconsistencies and/or missing information.

*** Making assumptions based on missing information consider the following:

In post #1 you have:

type nodes
  ...
  type(nodes), pointer  :: leaf        => null()
       ^^^^^

should this be:


  type(leaf), pointer  :: p_leaf        => null() ! or use type(leaf_t) with leaf type name changed

       ^^^^^



subroutine DESTROY_ROOT( this )
  implicit none
  type(root_node)          :: this    ! **** you had type(ROOT)
  if(associated(this% root) ) then
    deallocate( this% root )
    nullify( this% root )
  endif
end subroutine DESTROY_ROOT

recursive subroutine DESTROY_NODE ( this )
  implicit none
  type(node)           :: this ! *** was type(t_octree_node)
  type(node), pointer  :: node => null(), next => null()
  integer                       :: i
  integer, save ::debug=0
  if( associated(this% leaf) )then
    deallocate( this% leaf )
    nullify(this% leaf)
  endif
  if(associated(this% children)) then
    do i = 1, size(this% children)
      if(associated(this% children(i)) then
        debug = debug + 1
        print*, i, debug
        deallocate( this% children(i) )
        nullify(this% children(i))
      endif
    enddo
    deallocate( this% children )
    nullify( this% children )
  endif
  if(associated(this% parent)) nullify(this% parent)
end subroutine DESTROY_NODE

subroutine DESTROY_LEAF ( leaf )
  implicit none
  type(leaf)        :: p_leaf !  *** or type(leaf_t)        :: leaf
  deallocate( p_leaf% id )    ! *** or deallocate( loeaf% id )
end subroutine DESTROY_LEAF

*** the above may have incorrectness due to incomplete information posted earlier.

Jim Dempsey

[jimdempseyatthecove]

Also, if your tree has a very large number of branches, your stack must be sufficiently large enough to recurse the worst case tree branch.

And, your tree should be properly maintained such that it does not multiply reference the same node(s).

Jim Dempsey

[AThar2]

Hi Jim, Thanks for your reply and advice.
What is exactly meant by that it does not multiply reference the same node(s)

[jimdempseyatthecove]

Your code (as was) and my code (sketch) assumes that (at the point of deletion) that any and all pointer members are either NULL or unique. If a non-NULL pointer is not unique then you will be delallocating it multiple times. Keeping the pointers unique is outside the scope of the above code. Note, an additional requirement may be that all allocated nodes/leaf/other are represented by the tree. (You may have multiple trees and thus sets of nodes).

An additional thing to consider is what is the computational nature of the leafs?

IOW are the leafs suitable for aggregate computation? i.e. could, when organized properly, take advantage of vectorization?

I think there is an error in my code example due to:

type(nodes), pointer  :: children(:) => null()
 

This is a "pointer to an array of nodes" and not a "pointer to an array of node pointers". My deallocation code is incrorrect with this respect.. If you need an array of pointers then something like:

program pointerpointer
    implicit none
    integer :: i
    type node
        integer :: something
    end type node
    
    type node_pointer
        type(node), pointer :: p
    end type node_pointer
    
    type(node_pointer), pointer :: array(:)
    
    allocate(array(10))
    do i=1,size(array)
        allocate(array(i)%p)
    end do
end program
 

Unfortunately Fortran (at least V17.0) does not support

     type(node), pointer, pointer :: array(:) ! *** not supported
     ...
     array(index)%something ! *** not supported

So please adjust the code accordingly

Jim

[jimdempseyatthecove]

Also, whenever possible, avoid using pointers.

Jim Dempsey