Avoiding accidental use of older Fortran run-time libraries - are manifest files the answer?

[Mick_Pont]

I want to build a Windows DLL composed of object files generated
by Intel Fortran. Naturally the object files contain dependencies
on Intel compiler run-time DLLs. These run-time DLLs evolve
over time, as new versions of Intel Fortran arrive, but they normally
have the same name from release to release; for example, one of the
run-time libraries is always named libifcoremd.dll.

Because the run-time libraries change from version to version of ifort,
there's a chance that the end users of my DLL will have an older version
of them on their systems. If my DLL needs some symbol or function that
only exists in my version of the run-time library, but the end user's
older version gets loaded first, then a likely outcome is that the
end user's program will crash at run time.

I'm aware that if I distribute libifcoremd.dll with my DLL,
and the PATH environment variable is set up correctly
to locate my version of libifcoremd.dll before any other, then
everything just works. But in my real application, my DLL is
being called from a Python program, and Python itself has
pre-loaded some older Intel run-time libraries that I cannot
override. That's the issue I'm trying to address.

I thought that perhaps using manifest files to tell my
DLL exactly which versions of Intel run-time libraries to
use might be a way around this problem. My actual DLL is very
large and uses many run-time libraries, so I started off with
something small just to see if I could get it to work.
(By the way, I don't use Visual Studio to compile things
- everything I show below is done in a DOS command line).

Here's my file subby.f90:

  Subroutine subby(s1, s2)
    Character*(*) s1, s2
    s2 = s1
  End Subroutine subby

And here's a main program main.f90 that calls subby():

  Program main
    External subby
    Character*10 s1
    Character*10 s2
    s1 = '0123456789'
    Call subby(s1, s2)
    Print *, 's2 = "', s2 // '"'
  End Program main

I compile subby.f90:
  C:\> ifort /MD /Z7 -Od -c subby.f90
Then make it into a DLL named small.dll, exporting the symbol SUBBY :
  C:\> link /debug /dll /export:SUBBY /out:small.dll subby.obj
Then I compile and link the main program to it:
  C:\> ifort /MD /Z7 -Od main.f90 small.lib
And run it:
  C:\> main
   s2 = "0123456789"
Very good - results as expected. (By the way - I'm compiling with
a 32-bit version of Intel Fortran).

I list the dependents of small.dll using this command:
  C:\> link /dump /dependents small.dll
and the output mentions libifcoremd.dll as well as some Microsoft run-time
libraries.

Now comes the part that I'm having trouble with. I want to create a manifest
file to use to tell small.dll exactly what version of libifcoremd.dll it
needs.

Finding information about exactly what a manifest file should look
like is tricky (web searches find lots of discussion and documentation
of manifest files, but much of it is circular and it contains little
in the way of actual examples that fit what I want to do. perhaps that's
because it's just not possible). But, if it is to work, I *think* I need
a manifest file that looks something like this file, which I named
small.dll.manifest:

  <?xml version='1.0' encoding='UTF-8' standalone='yes'?>
  <assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
    <dependency>
      <!--I need Intel compiler runtimes to run-->
      <dependentAssembly>
        <assemblyIdentity name='libifcoremd.dll' version='18.0.1.0' processorArchitecture='ia32' />
      </dependentAssembly>
    </dependency>
  </assembly>

If I look at the properties of file libifcoremd.dll using Windows File
Explorer, and ask it to display details, I can see that the "File version"
is "18.0.1.0" and the "Product version" is "18.0 - 001". If I write a
program to call the routine for_ifcore_version(), it returns a version
string which includes "V18.0-001". The most sensible one of these looks
like "18.0.1.0" so that's what I used in my manifest file, shown above
(though I have tried other variants too).

By the way, the xmlns tag used in the manifest file comes about because
if I use the /manifest switch to the link command when I create small.dll,
it makes me a small manifest file containing that - so I used it as a starting
point for my own manifest.

I used Microsoft's Manifest Tool, mt, to put my manifest into the DLL:

  mt -manifest small.dll.manifest -outputresource:small.dll;#2

Now, if I re-link the main program to this revised version of small.dll,
it links without error, but at run-time I get an error popup window saying
"The application was unable to start correctly (0xc0150002)". Clearly
my injection of the manifest file has broken the executable.

So - my question is - does anyone know how to use manifest files in
the way I'm trying to do? Are manifest files the right way to go at all,
or am I barking up the wrong tree - is there a better way to get the
correct Intel compiler run-time libraries picked up by my DLL?

Mick Pont

[Steve_Lionel]

I definitely think you have the right approach, though I don't see where you said you put the DLLs in the appropriate folder under Windows\SxS.

But in this case why not link your DLL to the static libraries? I think it's an appropriate use case, unless you use OpenMP or coarrays. Otherwise you have to worry about not only the Fortran DLLs but also the MSVC DLLs.

I recall that I looked into creating manifests manually some years ago, but never got far with it,

[jimdempseyatthecove]

Alternatively you might be able to dynamically load the correct DLL and locate the entry points:

https://stackoverflow.com/questions/8696653/dynamically-load-a-function-from-a-dll

Jim Dempsey

[Steve_Lionel]

That won't work for the Fortran run-time library (libifcore, etc.)

[Mick_Pont]

Thanks Steve and Jim for your replies.

We did try doing as Steve suggested, creating a DLL linking in compiler run-time static libraries, but ran into a missing symbol problem that we didn't resolve - I think it was a wide-character string length function that's supposed to be in a Microsoft run-time library somewhere, but we couldn't locate it.

But, my colleague Mat persevered with the manifest solution, and eventually got it to work (in fact, once you know how to do it it's not that hard - finding out what to do is difficult though!)

In case it's helpful to anyone else in future, I'm adding Mat's comments below, showing how to generate a sample "small.dll" library.

------------------------------------------------------------------------------------------------------------------------

Here's how I've been able to get this to work.
I've used 64-bit ifort 19.0.1.
The page

  https://github.com/numpy/numpy/wiki/windows-dll-notes

was very helpful.

Here's an 'application' manifest (small.mnf) for small.dll, pretty similar to what Mick originally used:

<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
   <assemblyIdentity type="win32" name="small" version="1.0.0.0"/>
   <dependency>
     <dependentAssembly>
       <assemblyIdentity type="win32" name="libifcoremd" version="19.0.0.0"/>
     </dependentAssembly>
   </dependency>
</assembly>

This has to be loaded into the associated DLL in section #2, as he first tried. It can't live in a separate text file or anything like that.

The version attribute used for any dependencies is only relevant insofar as the associated 'assembly' manifest files (below) that you want small.dll's resolution to consume have to use matching values.

Stick the 'application' manifest into the DLL:

%> mt -manifest small.mnf -outputresource:small.dll;#2

(You don't need small.mnf any more now.)

For the 'assembly' manifests for each dependency, in the general case one could again insert data into the dependent DLLs (in section #1 this time) but the alternative we used is to use a separate manifest text file. The Windows name-resolution rules dictate that 'libifcoremd.manifest' in the same directory as small.dll will work in this instance:

<assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
   <assemblyIdentity type="win32" name="libifcoremd" version="19.0.0.0"/>
   <file name="libifcoremd.dll"/>
</assembly>

Note that the version attribute matches what we put in small.dll. You also need to add a file element explicitly associating this 'assembly' manifest with the DLL (libifcoremd.dll) that you want to explicitly be loaded. Put that dependent DLL in the same directory as small.dll too, e.g.

%> copy "c:\Program Files
(x86)\IntelSWTools\compilers_and_libraries_2019.1.144\windows\redist\intel64\compiler\libifcoremd.dll"
.

Then main should execute OK and always use the 'bundled' ifcore DLL.