Memory leaks with polymorphic allocatable objects and components

IanH · ‎02-01-2017

With 17.0.1 (and perhaps earlier, I didn't check) the compiler does not correctly handle the deallocation of polymorphic objects, where there is some sort of inheritance hierarchy and the deallocation is triggered by associating an allocatable object with an INTENT(OUT) dummy, or by using MOVE_ALLOC.

For example:

MODULE m
  IMPLICIT NONE
  
  TYPE, PUBLIC :: Parent
    INTEGER :: tag_parent = TRANSFER('prnt', 1)
  END TYPE Parent
  
  TYPE, PUBLIC :: Element
    CLASS(Parent), ALLOCATABLE :: item
    INTEGER :: tag_element = TRANSFER('elem', 1)
  END TYPE Element
  
  TYPE, PUBLIC, EXTENDS(Parent) :: Extension
    INTEGER :: tag_extension = TRANSFER('extn', 1)
    TYPE(Element), ALLOCATABLE :: array(:)
  END TYPE Extension
  
  TYPE, PUBLIC, EXTENDS(Extension) :: ExtExt
    INTEGER :: tag_extext = TRANSFER('exex', 1)
  END TYPE ExtExt
  
  TYPE, PUBLIC, EXTENDS(Parent) :: Other
    CHARACTER(:), ALLOCATABLE :: buffer
    INTEGER :: tag_other = TRANSFER('othr', 1)
  END TYPE Other
  
  PUBLIC :: Run
CONTAINS
  SUBROUTINE Run(option)
    INTEGER, INTENT(IN) :: option
    
    CLASS(Parent), ALLOCATABLE :: a
    CLASS(Parent), ALLOCATABLE :: not_allocated
    
    CALL create_extext(a)
    
    SELECT CASE (option)
    CASE (1)  ; DEALLOCATE(a)
    CASE (2)  ; CALL intent_out(a)
    CASE (3)  ; CALL MOVE_ALLOC(not_allocated, a)
    END SELECT
  END SUBROUTINE Run
  
  SUBROUTINE intent_out(arg)
    CLASS(Parent), INTENT(OUT), ALLOCATABLE :: arg
  END SUBROUTINE intent_out
  
  SUBROUTINE create_extext(arg)
    CLASS(Parent), INTENT(OUT), ALLOCATABLE :: arg
    
    TYPE(ExtExt), ALLOCATABLE :: tmp
    INTEGER :: i
    
    ALLOCATE(tmp)
    ALLOCATE(tmp%array(10))
    DO i = 1, SIZE(tmp%array)
      CALL create_other(tmp%array(i)%item)
    END DO
    
    CALL MOVE_ALLOC(tmp, arg)
  END SUBROUTINE create_extext
  
  SUBROUTINE create_other(arg)
    CLASS(Parent), INTENT(OUT), ALLOCATABLE :: arg
    
    TYPE(Other), ALLOCATABLE :: tmp
    ALLOCATE(tmp)
    tmp%buffer = REPEAT('x',1024)
    CALL MOVE_ALLOC(tmp, arg)
  END SUBROUTINE create_other
END MODULE m

PROGRAM p
  USE m
  IMPLICIT NONE
  INTEGER :: i
  
  continue        ! Get heap stats here.
  
  DO i = 1, 1000 ! 1000000 - use larger number to force crash.
    CALL Run(2)
  END DO
  
  continue        ! Get heap stats here.
END PROGRAM p

(The components initialized by transfer are just to provide identifiable tags for each type of object when examining the allocated blocks on the heap.)

On x86, if the upper limit of the do loop in the main program is increased to one million or so, the program will crash at run time - (the runtime is unable to report anything meaningful due to memory exhaustion). With more nominal values for the upper limit (e.g. as in the code above), the leak can be observed by using your heap analysis tools of choice. For example (on x64):

>ifort /check:all /warn:all /standard-semantics /debug "2017-02-02 memleak.f90"
Intel(R) Visual Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 17.0 Build 20161005
Copyright (C) 1985-2016 Intel Corporation.  All rights reserved.

2017-02-02 memleak.f90(44): remark #7712: This variable has not been used.   [ARG]
  SUBROUTINE intent_out(arg)
------------------------^
Microsoft (R) Incremental Linker Version 14.00.24215.1
Copyright (C) Microsoft Corporation.  All rights reserved.

"-out:2017-02-02 memleak.exe"
-debug
"-pdb:2017-02-02 memleak.pdb"
-subsystem:console
"2017-02-02 memleak.obj"

>"c:\Program Files (x86)\Windows Kits\8.1\Debuggers\x64\windbg.exe" "2017-02-02 memleak.exe"

then...

Microsoft (R) Windows Debugger Version 6.3.9600.17237 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

CommandLine: "2017-02-02 memleak.exe"

************* Symbol Path validation summary **************
Response                         Time (ms)     Location
Deferred                                       srv*c:\Users\ian\Symbols*http://msdl.microsoft.com/download/symbols
Symbol search path is: srv*c:\Users\ian\Symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
ModLoad: 00007ff7`c6960000 00007ff7`c6aee000   2017-02-02 memleak.exe
ModLoad: 00007ffe`a33f0000 00007ffe`a35c1000   ntdll.dll
ModLoad: 00007ffe`a11f0000 00007ffe`a129b000   C:\WINDOWS\System32\KERNEL32.DLL
ModLoad: 00007ffe`9fbf0000 00007ffe`9fe0d000   C:\WINDOWS\System32\KERNELBASE.dll
ModLoad: 00007ffe`a0cd0000 00007ffe`a0cec000   C:\WINDOWS\System32\imagehlp.dll
ModLoad: 00007ffe`9fe80000 00007ffe`9ff75000   C:\WINDOWS\System32\ucrtbase.dll
(22f0.3ebc): Break instruction exception - code 80000003 (first chance)
ntdll!LdrpDoDebuggerBreak+0x30:
00007ffe`a34c34e0 cc              int     3
0:000> bu `2017-02-02 memleak.f90:78`
*** WARNING: Unable to verify checksum for 2017-02-02 memleak.exe
0:000> g
ModLoad: 00007ffe`a1070000 00007ffe`a110e000   C:\WINDOWS\System32\msvcrt.dll
Breakpoint 0 hit
2017_02_02_memleak!P+0x4f:
00007ff7`c69692c4 48c7454800000000 mov     qword ptr [rbp+48h],0 ss:00000096`b3bffdb8=cccccccccccccccc
0:000> !heap -s
NtGlobalFlag enables following debugging aids for new heaps:
    tail checking
    free checking
    validate parameters
LFH Key                   : 0x3f9ea29935acbd81
Termination on corruption : ENABLED
          Heap     Flags   Reserv  Commit  Virt   Free  List   UCR  Virt  Lock  Fast 
                            (k)     (k)    (k)     (k) length      blocks cont. heap 
-------------------------------------------------------------------------------------
00000245043d0000 40000062    1020    148   1020     43     9     1    0      0      
00000245042b0000 40008060      64      4     64      2     1     1    0      0      
0000024504770000 40001062      60     36     60     13     2     1    0      0      
-------------------------------------------------------------------------------------
0:000> bu `2017-02-02 memleak.f90:84`
0:000> g
Breakpoint 1 hit
2017_02_02_memleak!P+0x8a:
00007ff7`c69692ff 48c7455000000000 mov     qword ptr [rbp+50h],0 ss:00000096`b3bffdc0=cccccccccccccccc
0:000> !heap -s
NtGlobalFlag enables following debugging aids for new heaps:
    tail checking
    free checking
    validate parameters
LFH Key                   : 0x3f9ea29935acbd81
Termination on corruption : ENABLED
          Heap     Flags   Reserv  Commit  Virt   Free  List   UCR  Virt  Lock  Fast 
                            (k)     (k)    (k)     (k) length      blocks cont. heap 
-------------------------------------------------------------------------------------
00000245043d0000 40000062   16364  13708  16364      3    14     5    0      0      
00000245042b0000 40008060      64      4     64      2     1     1    0      0      
0000024504770000 40001062      60     36     60     13     2     1    0      0      
-------------------------------------------------------------------------------------
0:000> WTF!!!
       ^ Syntax error in 'WTF!!!'

The differences in the record for the first heap for the process across the do loop show an increase of about 13MB of allocations, when the difference in allocations should be all but zero.

A slightly modified example is attached that uses the C runtime memory routines to illustrate the leak (must be compiled with /MTd).

>ifort /check:all /warn:all /standard-semantics /MTd /debug /exe:"2017-02-02 memleak2.exe" H:\Projects\Win32Lib\Other\CrtMemoryUtilities.f90 "2017-02-02 memleak2.f90" && "2017-02-02 memleak2.exe"
Intel(R) Visual Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 17.0 Build 20161005
Copyright (C) 1985-2016 Intel Corporation.  All rights reserved.

2017-02-02 memleak2.f90(44): remark #7712: This variable has not been used.   [ARG]
  SUBROUTINE intent_out(arg)
------------------------^
Microsoft (R) Incremental Linker Version 14.00.24215.1
Copyright (C) Microsoft Corporation.  All rights reserved.

"-out:2017-02-02 memleak2.exe"
-debug
-pdb:CrtMemoryUtilities.pdb
-subsystem:console
CrtMemoryUtilities.obj
"2017-02-02 memleak2.obj"

Explicit deallocate
0 bytes in 0 Free Blocks.
0 bytes in 0 Normal Blocks.
0 bytes in 0 CRT Blocks.
0 bytes in 0 Ignore Blocks.
0 bytes in 0 Client Blocks.
Largest number used: 13010 bytes.
Total allocations: 1301000 bytes.

End of procedure
0 bytes in 0 Free Blocks.
0 bytes in 0 Normal Blocks.
0 bytes in 0 CRT Blocks.
0 bytes in 0 Ignore Blocks.
0 bytes in 0 Client Blocks.
Largest number used: 0 bytes.
Total allocations: 1301000 bytes.

INTENT(OUT), ALLOCATABLE argument
0 bytes in 0 Free Blocks.
1289100 bytes in 2100 Normal Blocks.
0 bytes in 0 CRT Blocks.
0 bytes in 0 Ignore Blocks.
0 bytes in 0 Client Blocks.
Largest number used: 1276209 bytes.
Total allocations: 1301000 bytes.

MOVE_ALLOC with unallocated from
0 bytes in 0 Free Blocks.
1289100 bytes in 2100 Normal Blocks.
0 bytes in 0 CRT Blocks.
0 bytes in 0 Ignore Blocks.
0 bytes in 0 Client Blocks.
Largest number used: 1289100 bytes.
Total allocations: 1301000 bytes.

Kevin_D_Intel · ‎02-02-2017

Thank you for the complete, detailed analysis and convenient reproducer. I reported all of this to Development.

(Internal tracking id: DPD200417531)