Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.

dllexport: Exporting all module data automatically

Petros
Beginner
2,442 Views

I have a module with my global data in a DLL and I want to export it so that I can use it in other DLLs. I started by using:

!dec$ attributes dllexport :: a, b, c, ...

However, I have hundreds of variables in the module and I want to allow flexibility to the developer of the other DLL to access any of them (avoid having to expose one by one as needed).

I could write a script to parse the module file and generate the declarations but I would like a more elegant way. Is there a way to export the entire module?

0 Kudos
15 Replies
Steve_Lionel
Honored Contributor III
2,442 Views

No, there isn't. But you might consider putting all these variables in a COMMON and exporting that from the module.

0 Kudos
Petros
Beginner
2,442 Views

Steve Lionel (Ret.) wrote:

No, there isn't. But you might consider putting all these variables in a COMMON and exporting that from the module.

Thanks, Steve. I expected the answer but had hope. I actually spent quite some time removing all the commons and moving things into Modules in the past :-| I'll check which one is more convenient.

0 Kudos
Steve_Lionel
Honored Contributor III
2,442 Views

Put the COMMON in the module. It's really little different than module variables, and you can DLLEXPORT the common.

0 Kudos
Petros
Beginner
2,442 Views

Steve Lionel (Ret.) wrote:

Put the COMMON in the module. It's really little different than module variables, and you can DLLEXPORT the common.

Yes. However, I would still need to list all the variables in the COMMON and then DLLEXPORT the common. I can directly list the variables in the DLLEXPORT and skip creating the COMMON. Maybe there is something I'm missing.

0 Kudos
JVanB
Valued Contributor II
2,442 Views

If you compile the only to an *.obj file and then use dumpbin /symbols on the *.obj file at least you will have an easier task of parsing all the symbols so you could put them in a *.def file. Like suppose I had a Fortran source code:

module M
   integer a
   real x
   contains
      subroutine s
      end subroutine s
end module M

module N
   integer a
   real y
end module N

So now compile and DUMPBIN:

C:\>ifort /dll /c /nologo dll1.f90

C:\>dumpbin /symbols dll1.obj
Microsoft (R) COFF/PE Dumper Version 12.00.21005.1
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file dll1.obj

File Type: COFF OBJECT

COFF SYMBOL TABLE
000 00000000 SECT1  notype       Static       | .text
    Section length   30, #relocs    0, #linenums    0, checksum        0
002 00000000 SECT1  notype ()    External     | M.
003 00000010 SECT1  notype ()    External     | M_mp_S
004 00000020 SECT1  notype ()    External     | N.
005 00000008 UNDEF  notype       External     | M_mp_X
006 00000008 UNDEF  notype       External     | M_mp_A
007 00000008 UNDEF  notype       External     | N_mp_Y
008 00000008 UNDEF  notype       External     | N_mp_A
009 00000000 UNDEF  notype ()    External     | __ImageBase
00A 00000000 SECT2  notype       Static       | .drectve
    Section length   BB, #relocs    0, #linenums    0, checksum        0

String Table Size = 0x10 bytes

  Summary

          BB .drectve
          30 .text

Then our magic parser turns the output of DUMPBIN.EXE into dll1.def:

EXPORTS
   S = M_mp_S
   X = M_mp_X    DATA
   M_A = M_mp_A  DATA
   Y = N_mp_Y    DATA
   N_A = N_mp_A  DATA

Now we can compile and check that all symbols are exported:

C:\>ifort /dll dll1.obj dll1.def /nologo /exe:dll1.dll
   Creating library dll1.lib and object dll1.exp

C:\>dumpbin /exports dll1.lib
Microsoft (R) COFF/PE Dumper Version 12.00.21005.1
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file dll1.lib

File Type: LIBRARY

     Exports

       ordinal    name

                  M_A
                  N_A
                  S
                  X
                  Y

  Summary

          BA .debug$S
          14 .idata$2
          14 .idata$3
           8 .idata$4
           8 .idata$5
           A .idata$6

Looks good! I think gfortran exports all symbols automatically so this procedure might be unnecessary with that compiler.

0 Kudos
JVanB
Valued Contributor II
2,442 Views

A little experimentation showed that the *.lib file I made in Quote #6 was kind of useless. My first attempt used LoadLibrary and GetProcAddress to access the module variables and procedures and while it worked, it's a lot of effort to access more than a couple of variables by this method. My second attempt used BIND(C) to get at the symbols named in dll1.lib but instead of looking in dll1.lib to satisfy the names specified in the BIND(C) attributes the compiler made local space for them so when it wrote to variable X, for example, it didn't change the value of X as seen from the module. This could be fixed by using a !DEC$ ATTRIBUTES IMPORT statement but this gets us back to writing up all the stuff we wanted to avoid, just on the other side of the interface.

Finally I tried leaving the names unchanged in a new *.def file and using the modules M and N in the test program, but ifort refused to look for the symbols in the *.lib file I linked with. It could find them in an equivalent *.obj file, so I don't know what the problem is...

0 Kudos
LRaim
New Contributor I
2,442 Views

You can have hundredth of variables in a COMMON but one can easily get the length of the common by difference between the addresses of the first and last variable. So you can export all the variable with a single statement (WRITE or copy in memory).

You will never be able to do so using a module.

I consider a very bad habit to always replace a COMMON with a MODULE (think about this matter at least 10 times before doing so) and I would have placed such a programmer in a not-clever-programmer list.   

0 Kudos
FortranFan
Honored Contributor II
2,442 Views

@Petros, 

You can try looking into CMake 3.4 or later and check if you can use it with Intel Fortran compiler: 
https://blog.kitware.com/create-dlls-on-windows-without-declspec-using-new-cmake-export-all-feature/ 

Keep in mind what you're striving for here is rather poor library design and it's something Microsoft itself would not be keen to offer as a feature in its linker toward building DLLs.  And it has nothing to do with compilers such as Intel Fortran.   

JVanB
Valued Contributor II
2,442 Views

@FortranFan, in the article linked to it says For global data symbols, __declspec(dllimport) must still be used when compiling against the code in the DLL. So you still encounter the problem mentioned in the last paragraph of Quote#7. It seems that individual attention still has to be given to each variable that is ultimately imported by the end user's program. Given that, I would be inclined to create an INCLUDE file that has all the !DEC$ ATTRIBUTES DLLEXPORT statements in it and generate it automatically from the output of DUMPBIN /SYMBOLS. Would take three extra build steps every time global data was added or removed (compile the module, DUMPBIN to get symbol table, and then the parser to generate the INCLUDE file) and then there is the problem that the module procedures each have to have their DLLEXPORT declared within their bodies, although they could be handled via the *.def file solution if needed. Here is what I have gotten to work. My very crude parser:

! parse.f90
program parse
   implicit none
   character(132) line
   integer iunit
   integer junit
   integer vline
   character(8) UNDEF
   character(8) parens
   character(8) External
   character(100) name
   integer mp
   open(newunit=iunit,file='dll1.i90',status='replace')
   open(newunit=junit,file='dll1.def',status='replace')
   write(junit,'(a)') 'EXPORTS'
   do
      read(*,'(a)',advance='no',end=10,eor=5) line
5     continue
!      write(*,'(a)') trim(line)
      vline = scan(line,'|')
      if(vline == 0) cycle
      read(line,*) UNDEF, UNDEF, UNDEF
      if(UNDEF == 'UNDEF') then
         read(line,*) UNDEF, UNDEF, UNDEF, External, External
         if(External /= 'External') cycle
         mp = index(line(vline:),'_mp_')
         if(mp == 0) cycle
         read(line(vline+3+mp:),*) name
         write(iunit,'(a)') '!DEC$ ATTRIBUTES DLLEXPORT :: '//trim(name)
      else if(UNDEF(1:4) == 'SECT') then
         read(line,*) UNDEF, UNDEF, UNDEF, parens, parens, External
         if(parens /= '()' .OR. External /= 'External') cycle
         if(scan(line(vline+2:),'.') /= 0) cycle
         mp = index(line(vline:),'_mp_')
         if(mp == 0) cycle
         read(line(vline+2:),*) name
         write(junit,'(6X,a)') trim(name)
      end if
   end do
10 close(iunit)
   close(junit)
end program parse

This will write the required *.i90 and *.def files. Now we need a sample *.dll source file:

! dll1.f90
module M
   implicit none
include 'dll1.i90'
   integer a
   real x
   real y(10)
   contains
      subroutine s
         write(*,'(*(g0))') 'x = ',x
      end subroutine s
end module M

and a sample program that uses the *.dll:

! prog3.f90
program P
   use M
   implicit none
   X = 4*atan(1.0)
   call S
end program P

Finally a *.bat file that drives the build steps:

rem makedll.bat
@echo/ > dll1.i90
ifort /c /dll dll1.f90
dumpbin /symbols dll1.obj | parse
ifort /dll dll1.f90 dll1.def
ifort prog3.f90 dll1.lib

Having built the parser.f90 to parser.exe with ifort, typing makedll at the command line builds dll1.dll, dll1.lib, and prog3.exe, which actually works!

 

0 Kudos
Steve_Lionel
Honored Contributor III
2,442 Views

You can't use .def files for data in Fortran - the compiler has to see the DLLIMPORT directive for data (DLLEXPORT in a module turns into DLLIMPORT on USE) as the code it generates is different.

0 Kudos
JVanB
Valued Contributor II
2,442 Views

As I said in Quote #7, you can sort of use *.def files for data. That's what the DATA keyword in the EXPORTS section is there for. But as the reference cited in Quote #9 says, even in C you can't link to the data without extra pain such as DLLIMPORT or LoadLibrary/GetProcAddress/C_F_POINTER to get an alias for each data item.

That's why DLLEXPORT in the module seems to be the easiest way, because then the program that links to the *.dll can just USE the module and pick up the DLLIMPORT attribute from there. If the DLLEXPORT attribute were absent from the original module there is no way for a program that USEs the module to tell the compiler that the data item has the DLLIMPORT attribute because it is considered to be a redeclaration of the data item if you do so.

So just parsing the output of DUMPBIN /SYMBOLS and writing an INCLUDE file which will be realized in the specification part of the module works but the compiler doesn't allow you to do this for procedures because that's a redeclaration of the module procedure. Thus you need a *.def file for the procedures, so the parser writes both the INCLUDE file and a *.def file. On the end that USEs the module the compiler can figure out that the procedure comes from the appropriate module from the name mangling that was preserved in the *.def file.

Unfortunate that you need a different mechanism for code and data, but I guess that's just the Microsoft way. When you take this into account you can make it work with only one change to the code (the INCLUDE line in dll1.f90) and a parser (parser.f90 in Quote #10) and a couple of extra build steps (makedll.bat in Quote #10). Of course, if there were more than one module per file it might require a more sophisticated parser.

 

0 Kudos
Petros
Beginner
2,442 Views

@Luigi R. I do not understand the comment about the COMMON being better than using a Module. I find the module solution more friendly (readability) and also I can select what to import, etc. The fact that I can do the dllexport in one write is not crucial for me. Compilation time is very small.

@FortranFan I have been looking for a way for many DLL libraries to share a common memory. This is the best I found based on past comment. I put the global memory in a separate DLL and make it available through DLLEXPORT. Then, I link my other DLLs to it. Any better way is welcome.

@RepeatOffender Thanks! I'll try to modify your script to get the DLLEXPORTS, but I still prefer to use the MODULE to import them.

@Steve @RepeatOffender So, from what I got, if I dllexport the variables in the module and then have a "USE globals" in my DLL (with the appropriate .mod file and linking to the .lib), I do not need to explicitly write the dllimport in the DLL file, right?

Thanks to all!

0 Kudos
jimdempseyatthecove
Honored Contributor III
2,442 Views

A problem you may have with module data over COMMON is, from your specification of all variables to be exported, is it is .NOT. unusual for modules to have name conflicts with one another. With auto-exporting of all variables (as aliased names such as X as opposed to moduleName_mp_X) you will/may end up with name collisions. IOW be careful of what you ask for.

To accomplish what you want, it may be better to export (or make public "moduleName_mp_X") then construct a .h file (for use by C/C++/C#/other) that then produces a reference X to "moduleName_mp_X".

An alternative to this is to add a "get" function to the module that obtains the address of a named variable then your program's init function can produce the appropriate reference. Think of this as a C++ analog of the Fortran

    USE moduleName, ONLY : X
or
    USE moduleName, ONLY : UP=>X

implemented as

   double &X = moduleName.get("X");
or
   double &UP = moduleName.get("X");

(I will assume you are capable of interpreting and converting the sketch above to your needs)

Jim Dempsey

0 Kudos
JVanB
Valued Contributor II
2,442 Views

@Petros, I'm not sure what you mean in your comments to me in Quote #13. Have you tried downloading the 4 files in Quote #10 and then running makedll.bat? After that it should be more clear what I am doing in that example, because the prog3.exe that eventually gets built is able to access the data and procedures in module M, as can be seen from the fact the prog3.exe actually works. Also you can look at the dll1.i90 and dll1.def files that parser.exe makes from the output of DUMPBIN /SYMBOLS applied to dll1.obj. The only difference in your actual code from what would be the case if dll1.f90 were compiled to a *.obj file and prog3.f90 then linked to that *.obj file is the include 'dll1.i90' line in dll1.f90.

A problem with communicating is that you want to compile the file containing the global data module to a *.dll and then access that data in other *.dll files. So everything is a *.dll, but in your messages you are not very specific about which *.dll. When modifying my script all you should have to do is change 'dll1' to the name of the file that has the source code for your global data module. For example, if the file is called 'global_data.f90' you would change 'dll1' to 'global_data' throughout makedll.bat and parser.f90. I could have made this an argument to my makedll.bat file and then pass it along as a command line argument to parser.exe, but I wasn't sure whether it was just what you wanted and I didn't want to make my example any more complicated than it had to be.

I'm not sure about the comment about preferring to use a MODULE to import the data because that's just what my example does in that it inserts !DEC$ ATTRIBUTES DLLEXPORT statements into dll1.f90 through the INCLUDE file dll1.i90 which are then seen as DLLIMPORT statements in prog3.f90 through its USE of module M. You can't mix USE statements for a MODULE with hardwired DLLIMPORT statements for the module data because that's considered by the compiler to be a redeclaration of the variable names.

In your last comment I think when you say DLL you mean the source file for the *.dll that wants to access the global data. As remarked previously, not only will you not have to explicitly write the DLLIMPORT in the DLL, the compiler would reject it if you did.

@jimdempseyatthecove, exporting via DLLEXPORT or auto export (if possible) exposes the mangled names so that avoids collisions. Other languages can't read Fortran *.mod files so if it is desired to access the global data from another language, or for that matter from Fortran code compiled with another compiler, each variable is going to have to have effectively its own DLLIMPORT statement or acquire a pointer via LoadLibrary/GetProcAddress. Fortran seems unusual among languages that consumers of global data don't have to do this because the info is passed via the *.mod file. I suppose in C you could have a macro in a header file that evaluates to __dllexport or whatever in the source file the exports the global data and evaluates to __dllimport in the source files that access the global data.

0 Kudos
jimdempseyatthecove
Honored Contributor III
2,442 Views

RO>>auto export (if possible) exposes the mangled names

I am aware of this, however, Petros, appeared to indicate that he wanted the un-mangled name.

Jim Dempsey

0 Kudos
Reply