Compiling with DLL switches

[ThermoX]

Greetings,

I have a combination of C/C++ and Fortran libraries in a Nuclear Simulator exe.  The previous Visual Studio configuration had all sorts of "Ignore Specific Default Libraries" to prevent conflicting libraries between Single-Threaded, Multi-Threaded Release, Multi-Threaded DLL, etc, etc, etc.  This has been a CONSTANT battle for me, and the battle apparently ain't over.

So in an attempt to finally clean this whole mess up, I've recompiled the numerous libraries we have, using 2 versions.  The Multi-Threaded DLL (/MD) for Release and the Multi-Threaded Debug (/MDd) for Debug.  I also added the _AFXDLL Preprocessor definition to ALL libraries, since some of the libraries use MFC. HOWEVER, all the libraries are still compiled to create Static Libraries, not DLL's.

For the Fortran libraries, I changed the "Libraries->Runtime Library" settings to use the "Multithread DLL" and "Debug Multithread DLL", respectively. So that everything lines up.

The exe has 2 configurations (the usual Release and Debug) and I switch between the 2 sets of generated libraries, accordingly.

I did all this and everything finally compiled cleanly, with no warnings and no "Ignore Specific Default Libraries" entries.  I was a happy man... Until I loaded the Simulator and the Fortran code immediately crashed.  The code crashes when it attempts to READ a binary file, returning a huge integer value, that might be an overflow (I don't remember the value, exactly).  When I compile the old way, that returned integer is just 16.  So it almost seems like things are compiling in 64 bits, while as they should be compiling in 32.  I'm honestly clueless.

So I'm wondering what sort of differences could there be between compiling the Fortran code using the static "Multithread" vs  dynamic "Multithread DLL".  Are all the default compile settings identical between those 2?  I.e. could it be that when selecting the static libraries, some compile settings, such as default integer sizes, are different than when using the DLL? This seems improbable, but I can't think of anything else.

Any help would be greatly appreciated.

Platform: Windows 7, Pro
Intel Parallel Studio XE 2013 -> using the ia32\ifort.exe
Visual Studio Premium 2012

 

[IanH]

I can't think of any particular reason why static versus DLL runtimes would result in that sort of difference.  Did you recreate configurations for the projects from scratch?  If the latter, is there the possibility that you've dropped an otherwise necessary option along the way?

When spelunking in project properties I sometimes find it handy to inspect/difference the XML in the vfproj files themselves.

[ThermoX]

Hello IanH,

Actually, as part of this whole process, I've explored Project-Specific Property Sheets.  I was doing some of that before, but I really implemented most of my changes through a Property Sheet, which is included in all my base libraries.  Therefore, when I want to play around with some settings and make them the same across all my libraries, I simply edit the XML entry directly in my Property Sheet, and all my libraries benefit from it.  So most of the settings, for all my Library Projects, have been set to "Inherit from parent or project defaults".

It's actually a VERY cool way of doing things when there are many libraries to maintain and you want to make sure you apply the exact same settings across all libraries (when applicable).  So I guess the answer is Yes. I do look at the XML files quite a bit. Sometimes, rather than plowing through the different configurations within Studio, just looking at the XML file is a lot simpler.

With this setup and because of my issues, I'm able to change just the Fortran entry in my XML Property Sheet (for my different configurations) back to using static libraries, and re-introduce the "Ignore Specific default Libraries" for LIBCMT.lib & LIBCMTD.lib on the exe project with little effort.  Things are working fine this way and it's better than the numerous Ignores I had before... but still not where I'd like to be.

Thanks for the time.

[IanH]

I've used property sheets for MSBuild based projects (e.g. C++), but their relevance to the Intel Fortran integration into Visual Studio, which isn't MSBuild based, is news to me.
 

[ThermoX]

Wellll..... That might be because I left out a few "little" details of my setup... hum, hum...

I didn't want to put in too much details in my description since I didn't want the post to be a book, with people just skipping it as a result of it.  So I started with the basic question about possible differences between static-dynamic.

So here I go with the extended details... Which may or may not interest you...

The reality is that the whole setup is quite a bit more involved. As mentioned above, this is for a Nuclear Simulator.  There are thousands of Simulation Variables, which you can just "tap in" in any Fortran/C file you add to the Simulator.  For instance, the pressure value for a Pressure Gauge named PG100 may be stored into a variable called PG100PR.  Any Fortran code can just use PG100PR into its calculations.

Because of this, the variables are defined in a database. When we compile a file, we have a pre-processing tool that parses the file and extract variable names. Then the tool goes into the DB to extract the details of the variables (is it a logical*1, integer*4, etc...) The tool finally generates an Include file, where it dumps all that information, so that the Fortran compiler can actually compile, with all the proper variable definitions.

All of this to say that none of the projects we have actually use the standard Fortran project. They are actually configured to be C/C++ projects, with a bunch of Fortran Files, getting processed through custom build tools.  So yes, there are tons of places where things could go wrong :)

I'm pretty confident about the compilation of the Fortran code into obj file. When I look at the parameters that ifort.exe actually "sees" at the end of the custom build process, it's got all the right switches.  I've set that up a while ago and all of that works just fine when using static libraries.

What I'm less confident about, is the actual creation of the library, from all those obj files. Since this is ultimately done using C/C++'s lib.exe, not whatever tool Fortran uses to create libraries. I don't know too much about the internal structures of libraries. But I can't imagine that the results would be much different.  This is probably pretty standard stuff, at that point.

So there you have it... Why I can use Property Sheet on all my projects. Because none of them uses actual Fortran Projects (or the old VisualStudioProject XML structures, as Fortran apparently still did in 2013).  They all use MSBuild stuff.

Still awake?

[Steven_L_Intel1]

When you say "binary file", how are you opening it and reading it? What RECORDTYPE and ACCESS are you specifying?  Can you put together a reproducer that just tries reading the file and validating the values?

[ThermoX]

hmm... So I decided to take all the Fortran files in that one problematic project and create a "real" Fortran Project out of them. That one project actually does not need the pre-processing tool.  I built the library using the dynamic version. And linked my .exe with that library, for that project (all other Fortran projects were still using the dynamic versions in a C/C++ project).

Same problem.

So I changed the settings in the real Fortran Project to used static libraries, with all my other Fortran projects still configured as above.

Same problem.

So clearly, the problem didn't lie with that one project where the crash was happening, nor did it lie in my C/C++ version of the project.

This led me to my main exe project, which links all those libraries, and compiles a few additional files. That project also has a combination of C & Fortran files.  Ultimately, the problems comes when I set the Fortran settings for THAT main project to dynamic. When I do this, then my program crashes in that other library.  So all those other libraries seem to do what they're supposed to do. It's the settings in my main that seems to change the final behavior. That's probably because that's where the final linkage to all the Fortran standard libraries ultimately happen.

At least, this gives me something to focus my attention on.

[Steven_L_Intel1]

It could be a memory stomper that is dependent on the order in which things get loaded in memory.

[ThermoX]

Hi Steve,

I was probably writing the above comment while you were writing your question. That's why I didn't address your questions in it.

But still. My last answer, along with the fact that the exact same code has been working fine for years, leads me to believe that the problem doesn't really lie in the READ statement per say. That whole code deals with the Reactor Core Neutronics and is pretty horrible.  It was provided by a specialized vendor and few people actually dig into it, in my position.  I probably know more about it than the average guy but it's still pretty difficult to follow with variables of the like A(200000) and portions of A being mapped to sub arrays. It ain't pretty.

I suspect the reason it crashes there is simply because of its location in the whole calculation. It's where things really start to pick up. All code before that is mostly basic initialization stuff (i.e. variables are initialized to 0 and things like that). The code DOES partially read a text file during that initialization process and it does that fine.

Putting something together would be pretty hard, actually. Not to mention the security aspect of it.

Following my comment above, I did a dumpbin of my exe with the dll version vs static version for the Main Project ONLY.  All other Library Projects have been compiled with the /MD switch (for the C/C++ Projects) and the equivalent Fortran switches for the Fortran Libraries.

I dumped the dependents only.

I did a diff on the dump and is attached below.

One interesting thing... The order of the libraries does change, with the Fortran stuff showing up earlier in the dynamic version.  Not sure if that order shown on the dump is representative of the memory loading order you mention above.

Now... Based on what you know... are those differences normal? I'm not sure why the "imagehlp.dll" gets loaded in one case and not the other. Don't know what's that got to do with Fortran.  I read that the additional libifportMD.dll is just some portability thing.  But I'm not sure why the math library libmmd.dll shows up in the dynamic version and not on the static one.  That, to me, *could* be my problem.

Thanks again for the help.

 

[Steven_L_Intel1]

imagehlp.dll is used by the traceback feature. 

libifportmd.dll is used if you USE IFPORT or reference "portability library" routines.

libmmd.dll is the math library. If you link statically, it also is linked statically.

The differences are indeed normal. You're going to have to dig deeper and there's not much I can do to help you without a test case.

[ThermoX]

I understand and appreciate the time you took already. I feel I'm still a little ahead of where I originally was.

Thanks again.

[ThermoX]

Alright,

I'm getting somewhere. But it's going to be a little difficult to explain all of this.  So here goes nothing...

Steve, I DID end up going back to that READ statement after all.  Which sucks cause I HATE Fortran I/O stuff and its billion cryptic parameters.

So I created a new empty Fortran program and basically added just the lines that open up the binary files, and attempts to read the first series of bytes, as the original program.  Here's what the code looks like (which should answer all your original questions, Steve):

program Main

	INTEGER IOS
	INTEGER ITAP
	CHARACTER*20  LABEL
	CHARACTER*120 MSG

	ITAP = 1

	OPEN(ITAP,FILE='TheBinaryFile.res',FORM='UNFORMATTED',STATUS='OLD',IOSTAT=IOS, ACTION='READ')
	REWIND ITAP

!       Note that NREC is implicit, as is in the original code.
	READ(ITAP,ERR=1002,END=1001,IOMSG=MSG) NREC,LABEL

	! some dummy code to see where the next debug step goes to
	ITAP = 1.0

1001 CONTINUE

	! some dummy code to see where the next debug step goes to
	ITAP = 1.0

1002 CONTINUE

end program Main

I've also included the beginning of the binary data as a picture.

Ok. So here are some results.

• My original Library Project had the Optimization->I/O Buffering set to Yes.  This is because that project does a lot of I/O during initialization.  I noticed, however, than when I configure my Main Project (the exe one) to use static (with all other projects as dynamic), the READ code works fine, as before.  However, when I switch the Main to dynamic, the READ statement just fails completely with a very meaningful error message :'error during read, unit 1, file TheBinaryFile.res'.  When I remove that setting in my Library Project, the READ statement no longer fails, regardless of the static vs dynamic on my Main.  In the case of the short program above, the READ statement always fails, no matter the static vs dynamic setting.
• So I went back to my original Library Project and removed the Optimization->I/O Buffering setting. The READ statement succeeds, as described above. However, the NREC read value switches between 14 and 234881024, depending on my static vs dynamic setting on my Main project.  It's always 234881024 under the short version of the program above, no matter the static vs dynamic setting.
• So I looked at the memory for NREC, when trying this on my real program and I switched the static vs dynamic settings to see the internal representation of the variable... And... you've guessed it by now... It's a little-endian-big-endian problem.
• So I went back to my short program above, which always gave me 234881024 and changed the Compability->Unformatted File Conversion to Big Endian, the program now always read 14, as desired.
• I then decided to bring back the Optimization->I/O Buffering settings on the short program, which always failed to read anything... and now that the Big Endian is set, it works just fine.

So that's the main issue... the eternal  little-endian-big-endian problem.  Now, I need to figure out WHY switching the Main Project to between static vs dynamic messes things up.  Could it be that the problem comes from the fact that the Main project combines both C and Fortran files?

Any inputs will still be greatly appreciated...

[ThermoX]

I think I'm closing this post.

I couldn't exactly figure out why the code was switching between little-endian vs big-endian, every time I was switching my Main Project to use static vs dynamic.

However, I ended up adding CONVERT="BIG_ENDIAN" to the OPEN statement. Seems to work now, but the files in that project are all open through a custom subroutine, which ultimately calls OPEN. I added the new specifier for all "SEQUENTIAL" requests. I'm making the assumption that all files open in sequential mode will be binary files and that they should all use a big-endian format.

Time... and tests will tell...

[ThermoX]

Don't think anybody is reading this post, at this point. But I'll add up another finding, in case others might come across the same problem.

It turns out that after digging some more into the code, I found a place in a C function called when the program gets started, with the following line: _putenv( "F_UFMTENDIAN=big:1,26,65,89" );

As I've learned that this, effectively sets the listed Fortran Units (1,26,65,89) to use Big-Endian, through the environment variable F_UFMTENDIAN.  I then read the environment variable just before opening up the above-mentioned file.  When i switch the Main Project to use static, that environment variable is properly set.  However, when I switch the Main Project to use dynamic (like all other libraries in the solution), then this variable is blank. Which explains why those files were reading in little-endian format, as opposed to the big-endian.

I'm not sure WHY the Fortran code isn't seeing the variable, under those circumstances. But I suspect it's truly related to the DLL portion of it, where maybe the environment variables set programmatically don't always end up in the same memory space.

Frustrating but instructive experience...

[IanH]

I could imagine that symptom if you had multiple C runtimes active within the one process.

[Steven_L_Intel1]

Or multiple Fortran runtimes. The executable must be linked to the Fortran DLL libraries as well as any DLL built that has Fortran code.

[ThermoX]

Hmmmm.... I don't think I'm familiar with the use of runtimes in both of your comments.  To me, runtime has always been the exe running "in runtime". It's a single exe, with multiple threads. But there's just one instance of the exe running.  I'm assuming that's not what you're talking about, when you talk about multiple runtimes... i.e. multiple instances of the exe.  So I'm actually not sure what you mean, exactly. How would a given exe create multiple runtimes?  Looks like I'm gonna learn something new...

As for the linked Fortran code.  All custom Fortran code that's part of the exe is contained in individual Projects within the same Solution.  They're all compiled the same way using the Property Sheet, as mentioned above.  Some Projects contain a combination of C, C++ and Fortran code.  But all of them are compiled with the DLL versions.  I did not have to "exclude" any libraries in my final built.

Thanks for your additional inputs!  I figured out quite a bit... but there's still that one last unsolved mystery.  It'd be great if I knew the cause of it as well.

[IanH]

"Runtime" refers to the runtime library components of the C or Fortran compiler that was used to generate the executable modules that comprise the process - the specific variant of which (statically linked runtime, dynamically linked runtime) you are affecting with /MD, /MT, etc. 

Do any of your component projects (C++ or Fortran) generate DLL's (or do you use non-system DLL's compiled from Fortran or C++ code)?  When you talk about "libraries" in the various posts above, it is not always clear whether you are referring to the compiler runtime libraries or your own.

If you code results in the generation of an EXE and one or more DLL's, that subsequently get used within the one process, then the EXE and DLL's generally must all be using the "shared" DLL versions of the language runtimes (/MD).  Otherwise the EXE and DLL's will have isolated instances of the runtime, they will have isolated views of the state of the program as known by the runtime, and strange "split personality" type issues, such as what you describe in #14, may result.  The linker cannot warn you about this situation, because it is invoked separately for each executable module (.EXE or .DLL).  The linker can warn you when the mismatch in runtimes is visible within an executable module.

[mecej4]

ThermoX: Do you really want to use BigEndian in your Fortran file OPEN statements? Support for big-endian unformatted Fortran files in C/C++ is probably non-existent, so you are likely to run into various compatibility problems in mixed-language projects.

If you are working with a set of old big-endian files, which are not going to be replaced/refreshed, why not do a one-time conversion of the data files to little-endian, and use the converted files in your application? There are a number of file conversion programs available for translating from big-endian to little-endian and vice versa.

[ThermoX]

@IanH

I gotta say... I'm a little embarrassed I didn't realize what you guys meant by different Runtime Libraries... After all, this entire post was about it!  I guess, for some reason, I was separating the very "Runtime Library" setting I've been talking about all along vs the actual "Runtime" behavior, when the program is running. Did I already mention I was a little embarrassed?  Maybe it was too late when I read the posts? If anybody asks, that's my story and I'm sticking to it...

There are a few things that I'm uncertain about, which may result in that "split personality".

I know for sure that every single C, C++ and Fortran file has been compiled with the /MD option (as a reminder, I have 2 configurations; Release and Debug).  However, none of my own libraries result in the creation of dynamic libraries.  They are ALL compiled to generate static libraries.  So that's the first thing I'm unsure about.  The effect of compiling all the files with /MD option, while generating static libraries.  Could the split personality come from using my own libraries, which are ultimately static libraries, with the standard Fortran ones, which are linked with the DLL versions ??  Even though all the files within my own libraries use the /MD option?

The other thing I'm uncertain about is the use of the _AFXDLL Preprocessor Definition for ALL my C & C++ files, regardless if the resulting library needs the MFC stuff.  Sometimes, a given project is made up of C++ files using MFC, and regular C files.  Depending on the compile order, this was generating library conflicts as well.  If the C file was built first, then the non-MFC library version was initially used, which later conflicted with the MFC library version.  For those C files, I had to "include" a special MS header I once found on the web, which would force the C generated code to use the MFC library version.  But none of the Fortran files have been compiled with that _AFXDLL Preprocessor Definition switch (or equivalent).  However, from what I understand, the _AFXDLL only expands on the DLL version of the standard library.  I wouldn't think this would effectively create different runtimes.

This is all part of the ongoing battle I've had with these settings, mentioned in my very first post.

p.s. I've removed my "best reply" since the current discussion is the real root of my problems.
 

[ThermoX]

@mecej4:

This is the one thing I have no control over. There are more than 1 file and these are large binary Reactor Neutronics files generated by another program, running under a Unix emulator.  The files are changed on a somewhat regular basis.

In my EXE, all the reading and writing is done in this Fortran "black box" code.  We have the code, but like I said above, few people dig into it.  I wouldn't think the ability to specify big-endian as a READ setting in Fortran will go away any time soon, but I might be wrong.  The only "C" part is the setting of that environment variable, which was an add-on for that EXE.  I have since moved it elsewhere, in the Fortran code instead, where I think it's more appropriate.

But thanks for the suggestion anyway.