Compiling with DLL switches

[ThermoX]

Greetings,

I have a combination of C/C++ and Fortran libraries in a Nuclear Simulator exe.  The previous Visual Studio configuration had all sorts of "Ignore Specific Default Libraries" to prevent conflicting libraries between Single-Threaded, Multi-Threaded Release, Multi-Threaded DLL, etc, etc, etc.  This has been a CONSTANT battle for me, and the battle apparently ain't over.

So in an attempt to finally clean this whole mess up, I've recompiled the numerous libraries we have, using 2 versions.  The Multi-Threaded DLL (/MD) for Release and the Multi-Threaded Debug (/MDd) for Debug.  I also added the _AFXDLL Preprocessor definition to ALL libraries, since some of the libraries use MFC. HOWEVER, all the libraries are still compiled to create Static Libraries, not DLL's.

For the Fortran libraries, I changed the "Libraries->Runtime Library" settings to use the "Multithread DLL" and "Debug Multithread DLL", respectively. So that everything lines up.

The exe has 2 configurations (the usual Release and Debug) and I switch between the 2 sets of generated libraries, accordingly.

I did all this and everything finally compiled cleanly, with no warnings and no "Ignore Specific Default Libraries" entries.  I was a happy man... Until I loaded the Simulator and the Fortran code immediately crashed.  The code crashes when it attempts to READ a binary file, returning a huge integer value, that might be an overflow (I don't remember the value, exactly).  When I compile the old way, that returned integer is just 16.  So it almost seems like things are compiling in 64 bits, while as they should be compiling in 32.  I'm honestly clueless.

So I'm wondering what sort of differences could there be between compiling the Fortran code using the static "Multithread" vs  dynamic "Multithread DLL".  Are all the default compile settings identical between those 2?  I.e. could it be that when selecting the static libraries, some compile settings, such as default integer sizes, are different than when using the DLL? This seems improbable, but I can't think of anything else.

Any help would be greatly appreciated.

Platform: Windows 7, Pro
Intel Parallel Studio XE 2013 -> using the ia32\ifort.exe
Visual Studio Premium 2012

 

[mecej4]

Here is some example code to illustrate the issues with environmental variables and multiple runtimes. Let us use object files instead of building DLLs and import libraries, for simplicity. I DID NOT set the environment variable F_UFMTENDIAN in the (Intel configured) command window itself.

Consider the C function

extern int _putenv(const char *);

void SETBIN(){
_putenv("F_UFMTENDIAN=big:1,26,65,89");
return;
}

and the Fortran test program

program tstenv
use ifport
character(len=200) envstr,envvar
envvar='F_UFMTENDIAN'
call getenv(envvar,envstr)
write(*,10)'Before',trim(envstr)
call setbin()
call getenv(envvar,envstr)
write(*,10)'After',trim(envstr)
10 format(1x,A,' calling setbin(), F_UFMTENDIAN = |',A,'|')
end program

I compiled the C code using Intel C and the Fortran using IFort, both with the /MD option:

icl /c /MD setbin.c
ifort /MD tst.f90 setbin.obj

When I ran the resulting EXE, I found that the environment variable remained blank after setbin() was called. I then changed /MD to /MT in the second line, and this time I found that the environment variable was changed from the empty string to the desired value as a result of the setbin() call.

To find the particulars of which runtime was responsible for what happened, I could have generated linker map files and peered into them. However, I feel that the example is sufficient to show why using environment variables is a rather fragile way of handling your needs. Alternative methods may be more robust, such as making your program read a short configuration file at the outset to see if big-endian-little-endian conversion is to be performed, etc. Or, you could test for the existence of a file with a name such as 'USE_BIG_ENDIAN_INPUT'. Such a file could even be of zero length.

[ThermoX]

@mecej4

It seems that you have effectively replicated my problem, under the same precise circumstances, with yet a very simple example!

Now, there are a number of ways I can fix that and I've done so already. But what bothers me is WHY using a consistent set of compile switches results in the unexpected behavior; while as using inconsistent switches actually works as desired, in this case?  I'm also wondering if there will be other hidden problems, when consistently using the /MD switch? In this case, am I better off compiling the Fortran code in /MT and "ignore" the MT library in my final exe?  This would suck as the whole point of this exercise was to get rid on the inconsistencies in the libraries.

P.S. My preferred way would be to explicitly define the BIG_ENDIAN setting directly in the READ statement, since all the files are already open through a common custom subroutine, which ultimately calls READ.  But this results in one small problem, which would require me to generate other binary files. I've done so already with 1 file. But I'd need to do it with more than one.  Therefore, this approach is kinda demanding and the solution was to set the environment variable in Fortran code, instead of C.  That seems to work fine... But I'm still worried about unexpected behavior.

Again... your time is appreciated.

[ThermoX]

Hello again,

So today, I create the same files @mecej4 created, since it simplified the whole test drastically.  I got the same results as he did, which are the same results I'm getting in my real EXE.  I then also generated map files for both configurations (see attached files).

Now, I don't often generate map files and I might be missing some subtle detail... But I cannot find anything wrong with the "map_fmd_cmd.txt" version, which is "FortranMD - CMD". Everything seems to be properly linked using the DLL versions.  On the other hand, the "map_fmt_cmd.txt" version clearly has both the MSVCRT and LIBCMT versions in it.  Yet, that's the one that works.

The only somewhat unknown in the "map_fmd_cmd.txt" version are the "libirc" entries.  Based on Intel's documentation, it's a "message catalogs and related text message files" library. And the "libirc" seems to be the proper DLL version of those message catalogs.

So I'm pretty much stalled here.  It's as far as I can personally go.  Does anybody else have an idea why the Environment Variable set in the C file, doesn't get propagated to the Fortran code, when compiling both files with the /MD option?

At least, mecej4's test has ruled out a bunch of other unknowns I had previously mentioned.  So that's been of great help.  But I'm still missing that one last explanation...

[ThermoX]

Ah... and in case you're wondering... Yes, I did do the opposite test... i.e. compiling the C code with the /MT version, then experimenting with the /MD vs /MT versions on the Fortran code.  In this case, I got consistent results.  I.e. when the switches "lined up", the Environment Variable was properly seen by the Fortran code. When the Fortran code was compiled with the /MD version... It didn't see the Environment Variable.

So maybe the Fortran /MD option is the real unknown... As it never seems to see the Environment Variable, no matter how the C code was generated.

[Steven_L_Intel1]

I think I may know why, but I want to check with our developers first.

[ThermoX]

Great... Thanks.

[Steven_L_Intel1 · Accepted solution]

I have bad news and good news. Which do you want first?  Anyone?  Anyone?

Ok, bad news first. mecej4's test program definitely shows a problem with getenv. Because of the Fortran DLL libraries needing to work with multiple versions of the MSVC libraries, they do runtime lookup of some MSVC routines to make sure they get the right ones. Most of the time - but not getenv.  We'll fix that.

If mecej4 had used the Fortran intrinsic GET_ENVIRONMENT_VARIABLE instead, then he would have seen that the value did make it to Fortran, even when compiled /MD.

A bit more bad news - in the Fortran version ThermoX is using, the OPEN code didn't do the "runtime lookup" and instead was using its own static library version of getenv, and this explains the original problem. (It does puzzle me, though, that this is a problem at all - shouldn't putenv/getenv in the C library just be using the WIndows API routines?)

Now for the good news - we fixed OPEN in Parallel Studio XE 2016 (compiler 16.) and if the program is rebuilt with the newer version (or even just install the redistributables from the 2016 version), it should work. Here's my test case:

extern int _putenv(const char *);

void setbin(){
_putenv("F_UFMTENDIAN=big");
return;
}

program U601024
use, intrinsic :: iso_c_binding
interface
  subroutine setbin () bind(C)
  end subroutine setbin
end interface
character(80) :: ctype, envvar
call setbin
call get_environment_variable ("F_UFMTENDIAN", envvar)
print *, "F_UFMTENDIAN=",trim(envvar)
open (unit=1,status='scratch',form='unformatted')
inquire (unit=1,convert=ctype)
print *, "CONVERT=",trim(ctype)
close (1)
end

C:\Projects\U601024>cl /MD /c cfunc.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23506 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

cfunc.c

C:\Projects\U601024>ifort /MD U601024.f90 cfunc.obj
Intel(R) Visual Fortran Intel(R) 64 Compiler for applications running on Intel(R
) 64, Version 16.0.1.146 Build 20151021
Copyright (C) 1985-2015 Intel Corporation.  All rights reserved.

Microsoft (R) Incremental Linker Version 14.00.23506.0
Copyright (C) Microsoft Corporation.  All rights reserved.

-out:U601024.exe
-subsystem:console
U601024.obj
cfunc.obj

C:\Projects\U601024>U601024.exe
 F_UFMTENDIAN=big
 CONVERT=BIG_ENDIAN

[ThermoX]

Hi Steve,

That's actually pretty decent news.... At least, the problem was finally identified.  It makes me feel more confident about all the changes I made.

Now... I don't suppose we could get a nice discount to upgrade to Parallel Studio XE 2016, as a result of this (I'm assuming that version integrates with VS 2015) ;)  It's something I'd really like to do in the near future cause VS 2015 is so damn slick.

I flagged you as the Best Reply. But I wish I could flag mecej4 as well, since his stripped down version of my problem really ruled out a few "unknowns" and made it easy to replicate the problem.  Well mecej4... You're a Star in my heart, at least :)

[ThermoX]

(Although what you're saying, Steve, is that the problem of GETENV was actually a different problem than the READ.... even if on the surface, it looked like the same problem)

[Steven_L_Intel1]

Well, not quite. They are both the same problem in the version you have. But mecej4 is using the current version and there only GETENV has the issue.

We do offer upgrade discounts. Log in to the Intel Registration Center to see what your options are. Yes, Intel Parallel Studio XE 2016 works with VS2015.

mecej4 is indeed a star - he has earned his "Intel Black Belt" many times over.

[mecej4]

Thanks to both of you for the kind words. You would be mildly surprised if you could see me, while writing this note, actually wearing the physical black belt that Intel sent as a gift. The belt is made of fabric, black (of course!), 1.5 in X 10 ft, and I can wrap it around my waist many times over for back support.

[ThermoX]

Is there a paper version...  For those less worthy people?

BTW Thanks to IanH as well...

This is starting to feel like an Award show...

[Steven_L_Intel1]

The GETENV problem with the DLL libraries is expected to be fixed in Update 2 to Parallel Studio 2017.