Problems with using dynamic common blocks

[TScottDavis]

Fortran version
ifort (IFORT) 2021.10.0 20230609
GNU version (for C++ code)
g++ (Ubuntu 11.4.0-2ubuntu1~20.04) 11.4.0
Using WSL Ubuntu.

My application has a set of structures that get mapped to Fortran common blocks. A portion of the application loads Fortran shared objects (.so) using dlopen() and needs their common blocks to map to existing structures. I'm using dynamic common blocks to handle this and overriding the _FTN_ALLOC function. My custom _FTN_ALLOC function is called and provides the correct address for the common block. However, when the Fortran routine is entered, the variables in the common block don't have the expected values.

When debugging into _FTN_ALLOC(void **mem, int *size, char* name). After 'mem' is set, I get the expected values as follows:
*mem = 0xfa62280 <cb_shell_>
((double*)(*mem))[0] = 20.0
((double*)(*mem))[1] = 0.0
((double*)(*mem))[2] = 0.004166
The first three variables in the common block are REAL*8 values. Note that _FTN_ALLOC is C++ code.

Once I'm at the start of the Fortran routine, I get the following for the third variable:
1.2971498276818692e-315 (expected 0.004166)
The address of the common block at this point is given as:
0x7ffffd464070 <cb_shell.dyncom_ptr>

Fortran compiler flags are:
-extend-source 132 -f77rtl -intconstant -fpscomp logicals -real-size 64 -save -fpe1 -g -O0 -fPIC -dyncom cb_shell

The application also runs on Windows and the dynamic common blocks work as expected there.

Looking for any ideas what might be going on.

[TScottDavis]

I'm looking at this further and thought I'd share some more of what I'm seeing in the hope that someone will pickup on what may be happening.

As the previous post indicates, I'm using a custom _FTN_ALLOC to allocate a dynamic common block. The common block is setup as follows:

common/cb_shell/ shell(48)

byte shell

Then variables are setup using equivalent statements

equivalence( shell(17) ) dt  ! dt is the third REAL*8 in the common block

I don't think these details matter other than provide context for the names; cb_shell, shell, and dt.

The block is allocated at 0xfa62280.  It appears that Fortran creates a cb_shell.dyncom_ptr variable to hold this location of the common block cb_shell.  In gdb:

(gdb) p &shell
$16 = (PTR TO -> ( INTEGER*1 (48) )) 0x7ffffd464070 <cb_shell.dyncom_ptr>

(gdb) x 0x7ffffd464070

0x7ffffd464070 <cb_shell__dyncom_ptr>: 0x0fa62280

This looks like a stack variable pointing to the memory location provided by _FTN_ALLOC.

The 'shell' variable is given as:

(gdb) p shell

$13 = (-128, 34, -90, 15, 0, 0, 0, 0, -96, -109, -78, 15, 0, 0, 0, 0, -128, 36, -78, 15, 0, 0, 0, 0, 0, -4, -79, 15, 0, 0, 0, 0, -96, 56, -65, 15, 0, 0, 0, 0, 96, -48, -58, 15, 0, 0, 0, 0)

and the 'dt' variable as

(gdb) p dt

$14 = 1.2971498276818692e-315

The value of 'dt' in the allocated block is correct:

(gdb) x/gf 0x0fa62290
0xfa62290 <cb_shell_+16>: 0.0041666666666666666  (expected value)

Note that the first 4 bytes at 0x7ffffd464070 are

(gdb) x/4bd 0x7ffffd464070
0x7ffffd464070 <cb_shell__dyncom_ptr>: -128 34 -90 15

which is the same as the values in 'shell' above.  x/gf 0x7ffffd464070 shows the incorrect value of 'dt' above (i.e. 1.2971498276818692e-315).  Though this is odd as it doesn't appear to be offset in memory as I'd expect.

So it appears that a stack variable is created to hold the allocated common block, but instead of using the address held on the stack, the address OF the stack variable is being used as the common block address.  I can still be doing something wrong, but maybe this is a bug in some component of Intel Fortran?

[jimdempseyatthecove]

I would thing a better way (more portable) would be to create a user defined type and make an allocatable instance of that type .OR. create the user defined type, allocate (non-Fortran), return as C_PTR, and then use C_F_POINTER to construct the pointer to the Fortran object.

This does require you to use ObjectName%MemberVariable as opposed to MemberVariable.

Jim Dempsey

[TScottDavis]

Thanks for the reply.

As you may imagine, some of this code is not easy to rearchitect as that would impact multiple projects.   The dynamic common block approach worked well on Windows as it allowed minimal code changes.  I was hopefully it would work on Linux also.  I do have workarounds, but they limit the functionality of the application.

[Steve_Lionel]

It would be helpful if you could construct a small test case that demonstrated the problem, so we could try for ourselves. It is often the case that descriptions don't tell the whole story.

Dynamic common is an obscure and rarely used feature, and I've never heard of someone using it on Linux. That said, it sometimes is exactly what you need.

[TScottDavis]

That is a good idea.  I've always found that making a simple test case can help solve a problem.  I see that IanH has already done some work for me here and will investigate/expand what he put together.

[IanH]

Fortran only test cases constructed based on description - these work for me.

>ifort /Qdyncom "cb_shell" /dll "2023-08-19 common.f90"
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.10.0 Build 20230609_000000
Copyright (C) 1985-2023 Intel Corporation.  All rights reserved.

Microsoft (R) Incremental Linker Version 14.36.32532.0
Copyright (C) Microsoft Corporation.  All rights reserved.

"-out:2023-08-19 common.dll"
-dll
"-implib:2023-08-19 common.lib"
"2023-08-19 common.obj"
   Creating library 2023-08-19 common.lib and object 2023-08-19 common.exp

>ifort windows-main.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.10.0 Build 20230609_000000
Copyright (C) 1985-2023 Intel Corporation.  All rights reserved.

Microsoft (R) Incremental Linker Version 14.36.32532.0
Copyright (C) Microsoft Corporation.  All rights reserved.

-out:windows-main.exe
-subsystem:console
windows-main.obj

>windows-main.exe
Library loaded
Proc address obtained
In main program, address of stuff is 00007FF7493A9DA0 and d is 123.0000000000000
s1 entered - data is 00007FF7493A9DA0
Dynamic common set
d is 123.0000000000000

$ ifort -v
ifort version 2021.10.0
$ ifort -fpic -shared 2023-08-19\ common.f90 -dyncom "cb_shell" -o 2023-08-19\ common.so
2023-08-19 common.f90(19): remark #7841: DLL IMPORT/EXPORT is not supported on this platform.   [DLLEXPORT]
    !DEC$ ATTRIBUTES DLLEXPORT :: s1
---------------------^
$ ifort linux-main.f90 -o linux-main
$ LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH ./linux-main
Library loaded
Proc address obtained
In main program, address of stuff is 0000000000492700 and d is 123.0000000000000
s1 entered - data is 0000000000492700
Dynamic common set
d is 123.0000000000000

[TScottDavis]

Thanks IanH for putting these tests together.  Will take a look.

[jimdempseyatthecove]

RE: Modules and elimination of AllocatableUDT%var

If you use an allocatable of a User Defined Type, perhaps of the same name as the COMMON block name (e.g. cb_shell)

module mod_cb_shell
  type type_cb_shell
    sequence
    union
      map
         integer(1) :: shell(48)
      end map
      map
        real(8) :: X, Y, Z, DT, ...
      end map
    end union
  end type type_cb_shell
  type(type_cb_shell), allocatable :: cb_shell
  ...
end module mod_cb_shell
----------------
program X
  use mod_cb_shell, shell => cb_shell%shell
  ...
  allocate(cb_shell)
  ...
  read(ucb_shell, '(48I1)') shell
  ...
  call foo()
  ...
subroutine foo()
  use mod_cb_shell, X=>cb_shell%X,Y=>cb_shell%Y,Z=>cb_shell%Z,... 
  ...
  range = sqrt(X**2+Y**2+Z**2)
...

Note, you can place the use statement inside a source include file, say cb_shell.fi and then

subroutine foo()
  include 'cb_shell.fi'
  ...

This is possibly the same place as you currently have and INCLUDE for inserting the COMMON declaration(s).

Jim Dempsey

[TScottDavis]

Thanks for expanding on this idea and putting together the example code.  I'll have to think about this as a possible workaround.

[TScottDavis · Accepted solution]

After more investigation, the problem appears to be with gdb.  

I wrote a simple test case (code attached).  On both Windows and Linux, the code works as expected.  However, when debugging with gdb on Linux, I get similar results as I did with my application.  The variables in the dynamic common block show incorrect values.  The variables not in a dynamic common block, but set equal to variables in the block are displayed correctly.

I think I had a crash in the application and when debugging I saw incorrect values.  That set me off on a tangent that was not the actual issue causing the crash.  I was in the middle of getting the dynamic common block working in the first place so that was where I though the problem was.

Thanks everyone for the help.

NOTE: The test code includes two simple build scripts, common.f, ftn_alloc.cpp, main_linux.cpp, and main_win.cpp.  The common.f file represents the Fortran source that will be dynamically loaded and ideally would be "unmodified".  The ftn_alloc.cpp has the custom _FTN_ALLOC and an exported Setup routine.  The two main's create a data structure with test values, load the shared library, and get the exported functions.  Setup is used to pass the data structure and setup the custom allocator.  Then the foo function defined in common.f is called.

Though not previously discussed and not in the test code, a benefit for my application of using a custom _FTN_ALLOC is to detect new common blocks.  The application has the ability to load metadata for the common block based on its name.