Reallocation on assignment for derived type array with deferred-length components

[jwmwalrus]

Hi.

The following code aids in illustrating the problem.

implicit none

type :: sentence
    character(:), allocatable :: lower, upper
end type

type :: list
    type(sentence), allocatable :: list(:)
end type

integer :: i, n = 7
type(list) :: a

character(*), parameter :: LOWER = 'abcdefghijklmnopqrstuvwxyz'
character(*), parameter :: UPPER = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'

a%list = [sentence ::]

do i = 0, n
    if (MOD(i, 2) == 0) then
        a%list = [a%list, sentence(LOWER(:13+i), UPPER(:13-i))]
    else
        a%list = [a%list, sentence(LOWER(i+1:), UPPER(:i+1))]
    endif
enddo

do i = 1, SIZE(a%list)
    print *, i, ', lower=',a%list(i)%lower, ', upper=', a%list(i)%upper
enddo

end

From the code, it's clear that the a%list%lower component shouldn't have any uppercase characters, but I get the following with the ifort compiler:

$ ifort -g -O0 test.f90 && ./a.out
           1 , lower=abcdefghijklm, upper=ABCDEFGHIJKLM
           2 , lower=cdefghijklmnopqrstuvwxyzA, upper=AB
           3 , lower=abcdefghijklmno, upper=ABCDEFGHIJK
           4 , lower=ghijklmnopqrstuvwxyzABC, upper=ABCD
           5 , lower=abcdefghijklmnopq, upper=ABCDEFGHI
           6 , lower=klmnopqrstuvwxyzABCDE, upper=ABCDEF
           7 , lower=abcdefghijklmnopqrs, upper=ABCDEFG
           8 , lower=opqrstuvwxyzABCDEFG, upper=ABCDEFGH

Enabling the default optimizatiions mitigates the problem:

$ ifort test.f90 && ./a.out
           1 , lower=abcdefghijklm, upper=ABCDEFGHIJKLM
           2 , lower=cdefghijklmnopqrstuvwxyz, upper=AB
           3 , lower=abcdefghijklmno, upper=ABCDEFGHIJK
           4 , lower=ghijklmnopqrstuvwxyz, upper=ABCD
           5 , lower=abcdefghijklmnopq, upper=ABCDEFGHI
           6 , lower=klmnopqrstuvwxyz, upper=ABCDEF
           7 , lower=abcdefghijklmnopqrs, upper=ABCDEFG
           8 , lower=opqrstuvwxyzA, upper=ABCDEFGH

Using the ifx compiler with no default optimizations seems to match expected behavior, except for the last entry:

$ ifx -g -O0 test.f90 && ./a.out
           1 , lower=abcdefghijklm, upper=ABCDEFGHIJKLM
           2 , lower=cdefghijklmnopqrstuvwxyz, upper=AB
           3 , lower=abcdefghijklmno, upper=ABCDEFGHIJK
           4 , lower=ghijklmnopqrstuvwxyz, upper=ABCD
           5 , lower=abcdefghijklmnopq, upper=ABCDEFGHI
           6 , lower=klmnopqrstuvwxyz, upper=ABCDEF
           7 , lower=abcdefghijklmnopqrs, upper=ABCDEFG
           8 , lower=opqrstuvwxy, upper=ABCDEFGH

And the something similar happens for ifx with the default optimizations:

$ ifx test.f90 && ./a.out
           1 , lower=abcdefghijklm, upper=ABCDEFGHIJKLM
           2 , lower=cdefghijklmnopqrstuvwxyz, upper=AB
           3 , lower=abcdefghijklmno, upper=ABCDEFGHIJK
           4 , lower=ghijklmnopqrstuvwxyz, upper=ABCD
           5 , lower=abcdefghijklmnopq, upper=ABCDEFGHI
           6 , lower=klmnopqrstuvwxyz, upper=ABCDEF
           7 , lower=abcdefghijklmnopqrs, upper=ABCDEFG
           8 , lower=opqrstuvwxyzA, upper=ABCDEFGH

These are the compiler versions:

$ ifort -V && ifx -V
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.9.0 Build 20230302_000000
Copyright (C) 1985-2023 Intel Corporation.  All rights reserved.

Intel(R) Fortran Compiler for applications running on Intel(R) 64, Version 2023.1.0 Build 20230320
Copyright (C) 1985-2023 Intel Corporation. All rights reserved.

[Adelson_BR]

@Barbara_P_Intel ,

This code (teste_realoca_tipos.f90) is for reproducing the corruption of the content of an internal (allocatable) component (C(#)%B(#)%A) when B (C(#)%B) is reallocated. Showing this problem is all this code is for.

These are some lines that the code prints when A is allocatable.

1 Sum of A before reallocation of C(#)%B 15.00000
2 Sum of A before reallocation of C(#)%B 15.00000
3 Sum of A before reallocation of C(#)%B 15.00000
1 Sum of A after reallocation of C(#)%B -152.8812
2 Sum of A after reallocation of C(#)%B -300.7615
3 Sum of A after reallocation of C(#)%B -300.7615

Note that the sum of A is 15 (1+2+3+4+5) before reallocation and changes after the reallocation due to some memory invasion.

Also, if one comment line 11 and uncomment lines 12,  25, 42, and 51, to set A as pointer, the code also shows that the reallocation no longer affects the content of A.

And these are the same lines printed when A is a pointer:

1 Sum of A before reallocation of C(#)%B 15.00000
2 Sum of A before reallocation of C(#)%B 15.00000
3 Sum of A before reallocation of C(#)%B 15.00000
1 Sum of A after reallocation of C(#)%B 15.00000
2 Sum of A after reallocation of C(#)%B 15.00000
3 Sum of A after reallocation of C(#)%B 15.00000

It can be seen from above that the sum of A (and its content) is not altered after reallocation when A is a pointer.

No segfault happens here. Please, check if you follow the steps above for working A as a pointer.

I hope it helps.

Thanks in advance.

[Barbara_P_Intel]

Thank you. Those comments are MUCH to subtle to find without directions!

But I'm still missing something. I get identical results with both versions. Attached is the "working" version I created.

What compiler options are you using?

[jwmwalrus]

Hi @Adelson_BR .

If I try the code you provided, with the latest ifx, I get:

(ins)$ ifx -V teste_realoca_tipos.f90 && ./a.out 
Intel(R) Fortran Compiler for applications running on Intel(R) 64, Version 2024.0.2 Build 20231213
Copyright (C) 1985-2023 Intel Corporation. All rights reserved.

 Intel(R) Fortran 24.0-1238.2
GNU ld (GNU Binutils for Debian) 2.41.50.20231214
           3
 i=           1           3
 j=           1
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           2
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           3
   1.000000       2.000000       3.000000       4.000000       5.000000    
 i=           2           2
 j=           1
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           2
   1.000000       2.000000       3.000000       4.000000       5.000000    
 i=           3           1
 j=           1
   1.000000       2.000000       3.000000       4.000000       5.000000    
           3 before, size of C(#)%B
           1 Sum of A before reallocation of C(#)%B   15.00000    
           2 Sum of A before reallocation of C(#)%B   15.00000    
           3 Sum of A before reallocation of C(#)%B   15.00000    
           1 Sum of A after reallocation of C(#)%B   15.00000    
           2 Sum of A after reallocation of C(#)%B   15.00000    
           3 Sum of A after reallocation of C(#)%B   15.00000    
           5 after
           2 before, size of C(#)%B
           1 Sum of A before reallocation of C(#)%B   15.00000    
           2 Sum of A before reallocation of C(#)%B   15.00000    
           1 Sum of A after reallocation of C(#)%B   15.00000    
           2 Sum of A after reallocation of C(#)%B   15.00000    
           4 after
           1 before, size of C(#)%B
           1 Sum of A before reallocation of C(#)%B   15.00000    
           1 Sum of A after reallocation of C(#)%B   15.00000    
           3 after
           2 after           4
           1 after           5
           5
 i=           1           5
 j=           1
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           2
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           3
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           4
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           5
   1.000000       2.000000       3.000000       4.000000       5.000000    
 i=           2           4
 j=           1
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           2
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           3
  0.5000000       1.000000       1.500000       2.000000       2.500000    
 j=           4
  0.5000000       1.000000       1.500000       2.000000       2.500000    
 i=           3           3
 j=           1
   1.000000       2.000000       3.000000       4.000000       5.000000    
 j=           2
  0.3333333      0.6666667       1.000000       1.333333       1.666667    
 j=           3
  0.3333333      0.6666667       1.000000       1.333333       1.666667    
 i=           4           2
 j=           1
  0.2500000      0.5000000      0.7500000       1.000000       1.250000    
 j=           2
  0.2500000      0.5000000      0.7500000       1.000000       1.250000    
 i=           5           1
 j=           1
  0.2000000      0.4000000      0.6000000      0.8000000       1.000000    

So I guess that means the latest version fixes your problem, am I right?

Btw, at line 19 in your code, you're requesting the SIZE of a possibly unallocated variable, so if your actual code has a similar line, you might want to take care of that.

[Adelson_BR]

@Barbara_P_Intel  and @jwmwalrus ,

The compiler I had here was ifx and ifort 2023.  I got the binary with,

$ ifort -o teste_realoca_tipos -O3 teste_realoca_tipos.f90

Yes, you both could not reproduce the problem because you are using the new 2024 version of the compiler. I saw above in this thread that the newest compiler had addressed successfuly the issue. 

Good news!

I'll check for availability of this new compiler in my organization.

@jwmwalrus , thank you to remind me about line 19 in the code! But, please, don't mind about it since this code was extracted and dimmed so it is not surprise to have bugs as well. 

Thanks to everybody.

[Adelson_BR]

Hi,

I'm back here to report that in a RHEL 7.9, with

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/Pathto/oneapi/compiler/2024.0/lib; /Pathto/oneapi/2024.0/bin/ifx -o teste_realoca_tipos -O3 teste_realoca_tipos.f90

I reproduce the same memory problems I had with version 2023.

However, the problem is solved when I try with RHEL 8 in a docker, ifx version also 2024.0.2 2023.12.13.

Is the problem related to glibc version in RHEL 7.9?

Thanks again

[Barbara_P_Intel]

According to the Fortran System Requirements, RHEL 7.x is not supported with 2024.0.

>> Is the problem related to glibc version in RHEL 7.9?

I don't have an environment to test that idea.

[Adelson_BR]

Hi, @Barbara_P_Intel ,

I have compiled my testing code with RHEL 8 (glibc 2.28) and tried to run the binary in a RHEL 7.9 (glibc 2.17), with no warnings or dependence failures, just appending  LD_LIBRARY_PATH with the oneapi lib.

Under RHEL 8 I see no memory problems, its OK. But, again, with no warnings, in RHEL 7.9 there has been memory corruption.

IMHO, I think it would be nice if an execution aborting message could be issued to the programer pointing to the lack of proper conditions (old glibc?) to run the binary since the results may be unpredicatable. 

Thanks again

[Adelson_BR]

@Barbara_P_Intel , 

A colleague of mine, told me to compile the test program with options -Wl,--undefined=statx,

ifx -o teste_realoca_tipos -O3 teste_realoca_tipos.f90 -Wl,--undefined=statx

With this option, the binary generated in RHEL 8 (glibc 2.28) does not run on a RHEL 7.9:

$ testeealoca_tipos

teste_realoca_tipos: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by teste_realoca_tipos)

I will include this options from now on in future developments.

[Barbara_P_Intel]

Nice find! That's a Linux linker option.

[Barbara_P_Intel · Accepted solution]

Please try the latest ifx and ifort versions that were released last week. Looks to me like it's working now.

[jwmwalrus]

Hi @Barbara_P_Intel .

Thank you for the notification.

With the latest ifx I get the correct results for this issue, although I noticed what seems like a regression related to the structure constructor.

I just reported it here: https://community.intel.com/t5/Intel-Fortran-Compiler/Off-by-one-regression-with-deferred-length-character-string-in/m-p/1585935#M171556