adapting Windows .dll project to Linux .so

[rorban]

We are developing a realtime audio processor using Fortran for the DSP engine and C++ for the audio interface and GUI. So far, we have worked in Windows, and the Fortran generates a dll that is called by a C++ exe. We use ISO_C_BINDING.

The dll is quite complex, with tens of thousands of lines of code, dozens of modules, and hundreds of subroutines and functions. All subroutines and functions are in modules, with extensive use of PRIVATE and USE ONLY to enforce encapsulation. There are many subroutines and functions with the same name, but they are disambiguated by virtue of being private to a given module or contained within a higher-level subroutine.

The interface to the C++ is strictly limited to a few subroutines, using the following Intel Fortran extension to expose them:

DEC$ ATTRIBUTES DLLEXPORT::an_interface_subroutine

All other routines are private to the dll, 

The correct operation of this system depends crucially on optimization. The code executes too slowly with no optimization to permit realtime operation. We have compiled the dll with "whole program" optimization, and this works very well in Windows, speeding up the execution by about 3x compared to no optimization..

We are now considering porting this to Linux. I am an absolute novice in Linux, and other than being aware that Linux has a feature similar to dlls that uses the .so extension, I don't know anything about how our work with dlls in Windows can be ported to Linux .so files.

Can anyone point me towards documentation that explains this?

Is there an equivalent to DEC$ ATTRIBUTES DLLEXPORT in the Linux version of the compiler that enforces privacy of all routines in the .so except those I wish to expose to the C++ caller?

Will the .so version continue to support the data and procedure hiding enforced by modules, PRIVATE, and USE ONLY?

Does whole-program optimization work when creating Linux .so (as it does in Windows when creating dlls)?

Here is an shortened (but working) example of the module containing the public procedures that are called by the C++ caller.

MODULE Main_LoopModule

PRIVATE  
PUBLIC      audio_data_fort
    
CONTAINS 

! ========================================================================================

SUBROUTINE audio_data_fort     (                        IoBuffer      ,    &        
                                                        OutBuffer          &        
                               )  BIND(C,NAME="audio_data_fort") ! call this every nread samples. 
! The audio I/O to the C++ caller
                   
 ! Expose subroutine audio_data_fort to users of this DLL
 !DEC$ ATTRIBUTES DLLEXPORT::audio_data_fort
  
USE, INTRINSIC                  ::                      ISO_C_BINDING
                                                        
USE Audio_Processing_Module, ONLY:                      Audio_Processing
                                                        
USE WindowFunctions        , ONLY:                      nChannelsIO      , &
                                                        nread                        


 !------------BEGIN DUMMY VARIABLES-----------------------------------------------          
 REAL(C_FLOAT), DIMENSION(nChannelsIO * nread), INTENT(IN)  :: IoBuffer   
 REAL(C_FLOAT), DIMENSION(nChannelsIO * nread), INTENT(OUT) :: OutBuffer  
 !------------END DUMMY VARIABLES------------------------------------------------- 
 
CALL Audio_Processing (IoBuffer, OutBuffer)
  
RETURN

END SUBROUTINE audio_data_fort

! ========================================================================================
END MODULE Main_LoopModule 

[Steven_L_Intel1]

Linux doesn't have the concept of DLLEXPORT - when you link a .so, all of the global symbols are, in effect, exported (at least by default - I don't know if there's a way to limit that.) Intel Fortran on Linux will ignore DLLEXPORT attributes so you can leave them in if you want.  You can also use whole-program optimization (-ipo) when building the shared object.

[rorban]

Hi Steve -- Thanks for your quick reply.

In this case, what is a "global symbol?" Is it any function or subroutine declared PUBLIC in any module in the .so? Can any reader chime in regarding the challenge of preserving data hiding that is enforced by modules?

For example

MODULE A
PRIVATE
PUBLIC FOO
CONTAINS
SUBROUTINE FOO
END SUBROUTINE FOO
END MODULE A

MODULE B
PRIVATE
PUBLIC FOO
CONTAINS
SUBROUTINE FOO
END SUBROUTINE FOO
END MODULE B

MODULE C
USE MODULE A, ONLY : FOO
...
END MODULE C

MODULE D
USE MODULE B, ONLY: FOO
...
END MODULE D

shows a way of using two subroutines, both named FOO, in code without a naming conflict. So my question is this: what would happen in the .so? Would there be a name clash between the two FOOs? 

[Steven_L_Intel1]

It would be any module variable or module procedure, whether public or not, as accessibility concerns only references to names in source. The compiler still sometimes needs to know the linker-visible name of things even if private.

Note, though, that the global names all are qualified by the module name - on Linux, the default is modulename_mp_symbolname. So in your case above, the module procedure FOO in module A has a global symbol a_mp_foo and the one in module B has b_mp_foo, so there is no conflict.

Unless the programmer goes out of their way to access module entities by their global name, I wouldn't worry too much about hiding them.

[rorban]

Regarding C interoperability: In my original example, 

BIND(C,NAME="audio_data_fort")

would this subroutine appear as audio_data_fort in the .so, or would it be decorated as in the explanation you gave?

[Steven_L_Intel1]

It would appear exactly as you had it in the BIND.

[John4]

Just to clarify things a little bit for the OP:

The PRIVATE/PROTECTED/PUBLIC attributes in Fortran code are related to Fortran modules ---i.e., the linker, which is the one building the DLL or shared object, doesn't know about that at all.

On Windows, the DLLEXPORT (or the equivalent .def file) is actually an indirect instruction to the linker, so that it overrides the default, hidden, visibility for the given symbol.

On Linux, the visibility of symbols in an ELF (i.e., shared object or executable) defaults to public, but the strip utility can be used to remove all the symbols not needed for relocation purposes ---and most (binary) Linux distributions do just that with system libraries and executables, in order to reduce package size.

For C/C++, GCC has a similar concept to DLLEXPORT (i.e., __declspec(dllexport) ), in the forms __attribute__ ((visibility ("default"))) or __attribute__ ((visibility ("hidden"))).

In both Windows and Linux, the symbol can still be accessed by address, regardless of visibility, if you're into that.

 

[jimdempseyatthecove]

I haven't looked symbols in an unstripped ELF where you package your symbols into a user defined type, then declare an instance of that type.

IOW while the instance of the user defined type may be visible, the variables named therein might be hidden. It may be worth your effort to experiment with this to see if it gives you what you are looking for.

Jim Dempsey