memory management

[franzw82]

I am trying to implement a suballocator for managing the memory consumption of an application.

http://www.memorymanagement.org/articles/alloc.html#suballocator

Thus I want to associate variables (arrays, derived data types, etc.) with specific memory locations.

In C the location of a variable (scalar, array, struct, etc.) in memory (i.e. the address) can be changed by adjusting the pointer.

I am not sure how this could be realized in Fortran? I tried to use cray pointers, but these seem not to work correctly with derived data types. Any suggestions would be greatly appreciated.

[Jugoslav_Dujic]

If I were you, I would try to override the compiler's ALLOCATE and DEALLOCATE routines. In this way, you don't change the user code at all, just the allocation algorithm. Of course, that only applies to ALLOCATABLE variables.

This is not so hard as it seems. The trick is to write your own versions of library-routines for_alloc_allocatable and for_deallocate. If you put them in a source file within the project, and link the rest of the application with Dll version of run-time library, linker will find your "cuckoo eggs" first, and use them instead of library routines, whenever an ALLOCATE statement is executed (or DEALLOCATE).

Here are few old posts of mine, which describe the necessary bits and pieces.

1) Prototypes of for_alloc_allocatable and for_deallocate. There, you can find the routine interfaces, and declarations of Intel Fortran array descriptors Array descriptors are described here, and a more up-to-date version is in your help. Working with array descriptors is rather delicate and error-prone. However, I think (but did not verify) that the compiler fills in all the pieces except the address.

2) A sample implementation of for_alloc_allocatable. This is, however, very simple-minded, because it just forwards the call to for_allocate and fills the returned allocated memory with a bit-pattern; it does not manipulate array descriptors.

Note that memory manager algorithms can be mind-boggling if you try some non-trivial cases. I would rather stick to standard library memory management unless you exactly know what you're doing.

http://origin-software.intel.com/en-us/forums/showpost.php?p=11514

[Steven_L_Intel1]

I do not recommend overriding the compiler-supplied routines. The interfaces do change over time. Using C_F_POINTER in conjunction with TRANSFER to convert an integer of the correct size to type C_PTR seems the best approach to me.

[jimdempseyatthecove]

C_F_POINTER works as Steve suggests.

However, if you are trying to manage memory consumption in a general way, well that is what the heap is used for.

If you are trying to improve the performance and/or avoid fragmentation at the expense of some memory consumption then consider allocating arrays of structs (user derived types) and managing pointers to these. This can be done entirely in FORTRAN or you can use a C/C++ scalable allocator together with the C_F_POINTER.

However, using the C/C++ side allocation may introduce problems if your FORTRAN derived types require constructors (e.g. contains an allocatable array who's descriptor requires initialization). Other than for a wipe to 0 initialization, C/C++ allocation may present a problem.

In FORTRAN you can allocate to a pointer to an array of objects and then maintain a pool of these arrays of objects.

something like the following:

[fortran]module YourModule
type YourType
  real :: stuff
  integer :: otherStuff
end type YourType

integer, parameter :: YourTypePoolSize = 100

type YourTypePool
    type(YourTypePool), pointer :: next
    type(YourType) :: objects(YourTypePoolSize)
    integer :: freeCount
    logical :: IsFree(YourTypePoolSize)
end type YourTypePool

type(YourTypePool), pointer :: YourTypePoolHead => NULL()

contains
function getYourType()
type(yourType), pointer :: getYourType
    type(YourTypePool), pointer :: pool

    pool => YourTypePoolHead
    do while(associated(pool))
        if(pool%freeCount .gt. 0) then
            do i=1,YourTypePoolSize
               if(pool%IsFree(i)) then
                  pool%IsFree(i) = .false.
                  pool%freeCount = pool%freeCount - 1
                  getYourType => pool%objects(i)
                  return
               endif
               pool%freeCount = 0 ! ??
            end do
        endif
        pool => pool%next
    end do
    allocate(pool)
    pool%IsFree(1) = .false.
    getYourType => pool%objects(1)
    pool%freeCount = YourTypePoolSize - 1
    do i=2,YourTypePoolSize
        pool%IsFree(i) = .true.
    end do
end function getYourType

subroutine returnYourType(obj)
    type(yourType), pointer :: obj
    type(YourTypePool), pointer :: pool

    pool => YourTypePoolHead
    do while(associated(pool))
        if((loc(obj) .ge. loc(pool%objects(1))) .and. (loc(obj) .le. loc(pool%objects(YourTypePoolSize)))) then
          do i = 1, YourTypePoolSize
            if(loc(obj) .eq loc(pool%objects(i))) then
                pool%IsFree(i) = .true.
                pool%freeCount = pool%freeCount + 1
                return
            endif
          end do
        endif
    end do
    ! error
end subroutine returnYourType

subroutine cleanupYourType
    type(YourTypePool), pointer :: pool

    pool => YourTypePoolHead
    do while(associated(pool))
        YourTypePoolHead = YourTypePoolHead%next
        deallocate(pool)
    end do
end subroutine cleanupYourType

end module YourModule
[/fortran]

Jim Dempsey

[Jugoslav_Dujic]

Quoting Steve Lionel (Intel)

I do not recommend overriding the compiler-supplied routines. The interfaces do change over time. Using C_F_POINTER in conjunction with TRANSFER to convert an integer of the correct size to type C_PTR seems the best approach to me.

While I generally agree with that, the approach I suggested has the advantage that it keeps all the dirty low-level stuff at one place, leaving user code generally unaffected. If you stick to (semi-)standard approach as you suggested, you'll get C_F_POINTERS all over the code, and still don't have particularly good portability.

[franzw82]

Thank you for your suggestions. I will have a look at C_PTR pointers and the C_F_POINTER routine.