Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
Announcements
This community is designed for sharing of public information. Please do not share Intel or third-party confidential information here.
27159 Discussions

.net Bad Image Format exception with FORTRAN 2021.3

MatthewZ
Novice
2,842 Views

I have a FORTRAN dll project that was started in Parallel Studio XE 2015 and I finished migrating it to Intel FORTRAN 2021.3. The project file itself was entirely recreated in 2021.3 as part of the migration.

In the 2015 version, I always built it as a 64-bit dll, and call functions in this dll from C#.net. It was working well prior to migrating 2021.3. The C#.net project that consumes the dll is and has been compiled to target 64-bit as well.

When using the dll with built with 2021.3, I am now getting the following exception:

System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B) when the C# code calls a function from the FORTRAN dll.

I researched this issue and it commonly comes from trying to execute 64-bit code in a 32-bit environment. This may be more on how my C# project is invoked by another process, however, during my investigation, it appears that there is a header section written to unmanaged binary files that indicate what environment, i.e., 32-bit/64-bit, the binary is intended to target. See this link.

In the migration process, I encountered other difficulties that were considered bugs that were solved in FORTRAN 2021.3, I am wondering if there was some sort of change that was made to the way that the Intel FORTRAN compiler writes the header information to the binaries that perhaps was considered a bug and was thus incorrect with FORTRAN 2015 and this has been corrected with the 2021.3 version? My curiosity is more looking for an explanation for this behavior in 2021.3 and the lack of the same behavior in 2015.

Thanks.

0 Kudos
1 Solution
jimdempseyatthecove
Black Belt
2,500 Views

>>if I execute my C# program within its own folder and then send data to it, it works properly. However, if the api executes the c# program, it is only then that we see the problem

This is indicative of directory search problem where the wrong version of a same named file is selected.

IOW "execute my C# program within its own folder" generally has a different current directory (and drive) than that of "the api executes".

 

Jim Dempsey

View solution in original post

17 Replies
FortranFan
Honored Contributor II
2,808 Views

@MatthewZ ,

Readers and/or Intel support staff may be able to help if you can provide a small reproducer toward the problem you face.  Your post had me worried because of some difficulties a team I work with has encountered with 2021.3 update and Fortran Expression Evaluator (FEE) as part of Intel Fortran integration with Visual Studio.  However a quick check turns out ok - see below.  Hence you will find it useful to illustrate here the problem you face.

So, here's the quick check.  Say there is a silly little Fortran procedure as follows, a la "Hello World", in a file named Fdll.f90:

module Fdll_m
   
   use, intrinsic :: iso_c_binding, only : c_size_t, c_char, c_null_char, c_loc, c_f_pointer
   
contains

   subroutine Greet( msg, lenmsg ) bind(C, name="Greet")
   
      ! Argument list
      character(kind=c_char,len=1), intent(in), target :: msg(*)
      integer(c_size_t), intent(in), value             :: lenmsg

      if ( lenmsg > 0 ) then
         block
            character(kind=c_char,len=lenmsg), pointer :: s
            call c_f_pointer( cptr=c_loc(msg), fptr=s )
            s = c_char_"Hello World!" // c_null_char
         end block
      end if
      
      return
      
   end subroutine
    
end module Fdll_m

and the above Fortran code is not sullied by all the compiler-specific directives and therefore a Microsoft Windows DLL definition file - Fdll.def - is provided as follows: 

LIBRARY Fdll
EXPORTS
     Greet                  @1 

And say there is a C# consumer in FortranInterop.cs file like so:

using System;
using System.Text;
using System.Runtime.InteropServices;

namespace FortranInterop
{

   static class Fdll
   {

      [DllImport("Fdll.dll", CallingConvention = CallingConvention.Cdecl)]
      internal static extern void Greet(StringBuilder msg, int lenmsg);

   }

   static class TestFortran
   {
      static void Main()
      {
         StringBuilder msg = new StringBuilder(capacity: 15);

         try
         {
            Fdll.Greet(msg, msg.Capacity);
         }
         catch (Exception ex)
         {
            Console.WriteLine(ex.Message);
         }
         finally
         {
            Console.WriteLine("Fortran DLL greets " + msg.ToString());
         }

      }
   }
}

 

And that 64-bit i.e., x64 targets of Fdll.dll and FortranInterop.exe are built for above as follows:

C:\temp>ifort /c /standard-semantics fdll.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.3.0 Build 20210609_000000
Copyright (C) 1985-2021 Intel Corporation.  All rights reserved.


C:\temp>link fdll.obj /dll /def:fdll.def /out:fdll.dll
Microsoft (R) Incremental Linker Version 14.29.30038.1
Copyright (C) Microsoft Corporation.  All rights reserved.

   Creating library fdll.lib and object fdll.exp

C:\temp>csc -platform:x64 FortranInterop.cs
Microsoft (R) Visual C# Compiler version 3.10.0-4.21318.11 (7ceb6331)
Copyright (C) Microsoft Corporation. All rights reserved.


C:\temp>FortranInterop.exe
Fortran DLL greets Hello World!

C:\temp>

Then the console app output is as I expect.

MatthewZ
Novice
2,779 Views

@FortranFan 

I am sorry my post worried you, however, it sounds like you have not encountered the same problem.

I do think the problem is on our end, and posting code to recreate the exact conditions, even simplistic code, would not be the best path.

The question in my original post was simple, regarding any changes. Perhaps I should have stated that I was not looking for a solution to the problem as I was only looking for any possible differences in the header data included in dlls between the 2015 version and the 2021.3 version. However, I answered my own question using dumpbin, and I see no differences in the relevant flags.

I suspect that the problem is in the api that we are using to invoke our C#.net executable because, to make the long story as short as possible, if I execute my C# program within its own folder and then send data to it, it works properly. However, if the api executes the c# program, it is only then that we see the problem, and the exception stack trace generated looks like it is going through Win32 apis and the badimageformatexception is known to happen when trying to execute 64-bit code in a 32-bit environment.

 

I do note a difference difference in your code and my code. At the time, the guidance on creating interop dlls in FORTRAN was to use compiler directives

 

So my code has compiler directives like this -

 

	!DEC$ ATTRIBUTES DLLEXPORT :: FEAServerCleanup

 

 

I note your FORTRAN sample has this -

 

   use, intrinsic :: iso_c_binding, only : c_size_t, c_char, c_null_char, c_loc, c_f_pointer

 

There is nothing like that in my code - note - however that the dll was working properly when compiled with the 2015 compiler.

 

My c# code is, other than obvious differences for function names, the same as yours.

My question regarding this is the compiler directive approach still valid? If not, then I have more work to do to complete the migration of this project from the 2015 compiler to the 2021.3 compiler. I have reviewed the new documentation on creating dlls and I have not seen any references to the compiler directive approach.

jimdempseyatthecove
Black Belt
2,501 Views

>>if I execute my C# program within its own folder and then send data to it, it works properly. However, if the api executes the c# program, it is only then that we see the problem

This is indicative of directory search problem where the wrong version of a same named file is selected.

IOW "execute my C# program within its own folder" generally has a different current directory (and drive) than that of "the api executes".

 

Jim Dempsey

MatthewZ
Novice
2,476 Views

I double-checked this and proved to myself that the api dll is enforcing its own runtime environment that over-rides the system path. So this is the problem.

FortranFan
Honored Contributor II
2,343 Views
@MatthewZ wrote 07-20-2021 01:25 PM
I double-checked this and proved to myself that the api dll is enforcing its own runtime environment that over-rides the system path. So this is the problem.

@MatthewZ ,

In this comment with the sentence, "Some additional things to check are the runtime environments for this program on the Windows 10 OS and the Windows PATH, etc. and whether any x86/Win32 DLLs - either the Fortran DLLs built with this project or their other DLL dependencies - would somehow get loaded instead of x64 targets," I was trying to suggest the same to you.

It looks like you've managed to figure out and solve the problem and I presume your end conclusion is the latest Intel oneAPI 2021.3 version and the changes therein have nothing to do with the issue you ran into - can you please confirm?

So that reminds me: given the layered workflow with this application you presented, have you or anyone working on the managed code (C#) side looked at the Windows APIs for unmanaged DLLs such as LoadLibrary/LoadLibraryEx and GetModuleFileName and like; with these one can get direct control of which exact version of unmanaged DLLs are loaded from where i.e., if one so chooses and if nothing else, these APIs can also be used to report/log someplace which of the unmanaged DLLs are loaded during the managed code execution.  A team I have worked with in the past who employs managed code apps but with a few unmanaged DLLs for compute-heavy tasks has found such control and logging rather helpful.

MatthewZ
Novice
2,298 Views

@FortranFan wrote:
@MatthewZ wrote 07-20-2021 01:25 PM
I double-checked this and proved to myself that the api dll is enforcing its own runtime environment that over-rides the system path. So this is the problem.

@MatthewZ ,

It looks like you've managed to figure out and solve the problem and I presume your end conclusion is the latest Intel oneAPI 2021.3 version and the changes therein have nothing to do with the issue you ran into - can you please confirm?

Yes, that appears to be the case.

Thanks for your other tips.

IanH
Black Belt
2,768 Views

Given you re-created the project file from scratch (why?), I suspect you are simply compiling/creating for the wrong platform (i.e. 32 bit versus 64 bit). 

Have you checked that platform settings are consistent?  Are you using Visual Studio?  What does the Build > Configuration Manager dialog box show?

Consider attaching the C# and Fortran project files.

(To be precise, the compiler isn't the thing that writes the DLL "binary" file.  That sort of final output is produced by the linker, which is provided by Microsoft (perhaps through an Intel wrapper).  The compiler generates object code files (amongst others) that are specific for a particular platform, the linker then stitches those object code files together to make DLLs and EXEs.)

The in-source stuff like compiler directives and use statements are not relevant, given information you've provided so far.

MatthewZ
Novice
2,697 Views

@IanH wrote:

Given you re-created the project file from scratch (why?), I suspect you are simply compiling/creating for the wrong platform (i.e. 32 bit versus 64 bit). 

Have you checked that platform settings are consistent?  Are you using Visual Studio?  What does the Build > Configuration Manager dialog box show?

Consider attaching the C# and Fortran project files.

(To be precise, the compiler isn't the thing that writes the DLL "binary" file.  That sort of final output is produced by the linker, which is provided by Microsoft (perhaps through an Intel wrapper).  The compiler generates object code files (amongst others) that are specific for a particular platform, the linker then stitches those object code files together to make DLLs and EXEs.)

The in-source stuff like compiler directives and use statements are not relevant, given information you've provided so far.


Thanks for your reply. Why did I recreate the project? Because that was what was suggested by Steve Lionel because of the number of years that had passed since our team last updated and doing so would, according to Steve, eliminate possible problems with compiler settings mismatches - of which I experienced settings that are now using the "-" character instead of the "_" character such as /extend-source:132. IMO, renames like that serve no useful purpose except to cause headaches to developers.

And, BTW - the 2021 compiler settings within VS still use the "_" character and thus, by default, generate a "this setting has been deprecated warning use the version of this setting that uses the '-' characer" which cannot be set in the VS project properties dialog. Opps!

And no, I am not compiling for the wrong platform. The configuration manager shows x64 for the project as expected. Although I have only been writing FORTRAN professionally for about 10-years, I have over 25-years of professional experience with Visual Studio starting with  Visual Studio C++/MFC 1.52. Most of that experience was with unmanaged code - so I am familiar with what linkers do.

When we first started the conversion process, I discovered what Steve Lionel determined to be a bug. So my statement about the compiler was intended as a general question about whether there is another problem in the 2021.3 compiler/linker/HPC Toolkit package in general.

I discovered late yesterday afternoon, that the same dll built x64 debug runs properly, however, the release build does not. I am about to go through the compiler/linker settings - to attempt to determine which of the settings is causing the problem.

Thanks for letting me know that the compiler directives appear to have no relevance. I still do not see what good posting the code would do as it appears now that it is compiler/linker settings for the release build that appears to be causing the problem.

You can be sure I'll post again when I finish going through the compiler/linker settings and determine which, if any, are causing the issue.

Thanks again.

IanH
Black Belt
2,724 Views

I think the forum might be eating things again, as I see replies on email that are not here.  Anyway...

I discovered late yesterday afternoon, that the same dll built x64 debug runs properly, however, the release build does not. I am about to go through the compiler/linker settings - to attempt to determine which of the settings is causing the problem.

Thanks for letting me know that the compiler directives appear to have no relevance. I still do not see what good posting the code would do as it appears now that it is compiler/linker settings for the release build that appears to be causing the problem.

Not so much chasing the code - consider attaching the project files (e.g. the .vfproj for the Fortran project, the C# equivalents, perhaps the sln file too).  That lets us understand what compiler and linker settings you are using, for the different configurations and platforms.

 

MatthewZ
Novice
2,694 Views

My bet is that someone was offended by some of my comments in the post and the mods deleted it. My apologies to anyone offended.

I am attaching a zip of the pertinent project/solution files and a text file of the exception stack trace from the c# exe. In the zip file I have included the following files:

  1. FEAServer_original.vfproj - in this project file, x64 debug will run, but x64 release crashes.
  2. FEAServer_debug_not_working.vfproj - settings were changed until x64 debug crashed as described below.
  3. FEAServer_release_working.vfproj x64 release runs after removing /Qparallel as described below.
  4. FEAServerVisualization.UI.WPF.csproj c# exe that consumes the fortran dll.
  5. FEAServer.Common.Interop.csproj - c# dll that acts as a wrapper using dllimport for the fortran dll.
  6. FEAServerVisualization.WPF.sln solution file for FEAServerVisualization.UI.WPF.csproj
  7. FEAServer-DLL_2015_compiler.vfproj the fortran dll project file from the 2015 fortran compiler - all build configurations work properly.

There is a complication in that there is another C# assembly that invokes C# exe - FEAServerVisualization.UI.WPF.csproj I do not have access to that assembly - either code or project file. I am reasonably sure that the assembly is built "Any CPU". However, I have my doubts that those files would show anything useful. The operating environment is Windows 10 x64. Note that if the c# exe is started by double-clicking it in the folder where it is installed instead of the intermediate assembly starting it, the crash does not happen. Also, the fortran dll built with the 2015 compiler ran properly when the c# exe was started by the intermediate assembly. Given what I describe below, I do not think that the problem is in the intermediate assembly.

A bit more on the workflow. When data is sent to the C# exe for analysis, it invokes a call to the fortran dll. This is the first call that is made to the fortran dll and is when the crash happens.

The workflow is as follows:

  1. A design is created in a design app.
  2. The design app then sends the design data to the c# exe via an intermediate assembly that invokes the c# exe if it is not running. The crash does not occur if the c# exe has been started by double-clicking it in the folder where it is installed.
  3. When the c# exe is up and running, it loads the design data
  4. c# exe then makes a Cleanup call to the fortran dll - when the design is first loaded, there is nothing to cleanup so that call does nothing at this point. (i.e., checks for allocated memory, if there is none does nothing, closes file units that are not open, etc.) - if those operations were a problem, I would see a different exception from the fortran dll and not the invalid format exception, and the problems would have occurred long ago.
  5. Even though the Cleanup call does nothing in the fortran dll, the crash happens.

With the 2015 compiler and /Qparallel set for the x64 release build, the crash did not happen; however, it does happen with the 2021.3 compiler.

There are no projects that are being built 32-bit or "Any CPU" for the 64-bit builds.

As I mentioned in the deleted post, I found that the debug build of the fortran dll would run without crashing while the release build would crash.

As such, I started with the debug x64 project and gradually matched the settings to the release x64 build. It really did not take all that long as I first turned off debugging information for the compiler and the linker, then I added optimization /O2 (no crash) and then added /Qparallel which then caused the debug build to crash. Turning off /Qparallel in the release x64 build makes the crash go away. However, doing so appears to impact the performance of the application. I also tried /O3 with /Qparallel for the release x64 build, and the crash still happens with those settings.

I don't know that there is enough here for anyone to solve this without having that intermediate assembly.  Since the fortran dll x64 release build from the 2015 compiler worked, I have to assume that the issue is not in the intermediate assembly; however, I assume, incorrectly or not, that there have been changes between the 2015 compiler and the 2021.3 compiler that may be somehow causing this issue.

A potential work-around for us is that the design app directly invokes the c# executable without relying on the intermediate assembly, and I intend on discussing this with project management.

Please let me know if anything else would be useful, and I will do my best to provide it. My apologies again for lacking the project/code files for the intermediate assembly, and my apologies for the rather long and wordy post.

andrew_4619
Honored Contributor I
2,663 Views

Might be wholly irrelevant I haven't studied all the posts. I note you had /qparallel which requires OMP. In recent years OMP must use dlls as static build is not now supported.  Do any of the older non-fortran object use static OMP?

Steve_Lionel
Black Belt Retired Employee
2,598 Views

If an older object was compiled with OMP enabled and static linking, it would have a linker OBJCOMMENT directive asking for the OMP static library - which no longer exists, so the link would fail.

FortranFan
Honored Contributor II
2,591 Views

@MatthewZ ,

I suggest you look very closely at each of the layers in this program workflow for target platform compatibility (colloquially the bitness of linker outputs), particularly where the obfuscated .NET assembly(ies) being employed in this workflow come into play, and check whether x64/Any CPU/x86 differences are in effect and which can then lead to an attempt to load a DLL in an incorrect format.

Some additional things to check are the runtime environments for this program on the Windows 10 OS and the Windows PATH, etc. and whether any x86/Win32 DLLs - either the Fortran DLLs built with this project or their other DLL dependencies - would somehow get loaded instead of x64 targets.

I did a simple test with a Fortran DLL built using /Qparallel option and confirmed the expected additional dependency to the Intel OpenMP run-time library (e.g., libiomp5md.dll) but then there were no issues with consuming the x64 Fortran DLL built using oneAPI 2021.3 with Microsoft .NET x64 apps.

MatthewZ
Novice
2,550 Views

@FortranFan ,

Thanks for your reply.

One of the first things I checked was whether obfuscation, itself, could have been the cause of this, and I compiled the entire solution for the c# fortran dll consumer with obfuscation completely turned off. I realize what you are suggesting is to check each individual layer as well as assemblies for the c# exe, however, I have encountered obfuscation problems before and thought it might be a good thing to check.

The issue also happened with all obfuscation turned off, and for the x64 configuration of the c# exe solution, all assemblies are built x64. I was told that the intermediate assembly that launches the c# exe is compiled AnyCPU, and being the systems where I have so far seen this are all 64-bit Windows 10, I would assume that the intermediate assembly is running 64-bit due to the fact that it is compiled AnyCPU. The question is asked and answered as such in this thread.

As to the libiomp5md.dll being the cause of this, yes, I could understand that if the 32-bit version were attempting to load, it would cause the same exception. However, I had installed the 2021.3 Intel® Fortran Compiler Classic Runtime for Windows*  on this same PC along with requisite MSVC runtime packages. If libiomp5md.dll were missing, I would have expected that the exception would be a dll not found exception, not a bad image format exception. This particular PC is a test PC that has no development tools installed on it specifically because a PC configured as such will reveal any problems like a missing runtime dll. However, if a missing libiomp5md.dll were the cause of this, the application would not have run when executing it through Windows Explorer from its directory.

Thanks again.

MatthewZ
Novice
2,543 Views

The bitness of the c# assemblies does not make much sense as the root of this issue, at least as I see it, given that simply turning off /Qparallel in the X64 release build causes the bad image format exception to go away even when the c# exe that consumes the fortran dll is executed by the intermediate assembly. Not only that, if the bitness of the c# assemblies was the cause of the problem, why does the system work when the fortran dll is built x64 release with the 2015 compiler?

Steve_Lionel
Black Belt Retired Employee
2,532 Views

Do you have any large static arrays in the DLL? I am wondering if you are running into static address space limits - these can cause bizarre symptoms,

MatthewZ
Novice
2,511 Views

Hi Steve,

Thanks for your reply. There are no large static arrays in the code. Most arrays are allocatable.

Reply