Linking object files crashes in Visual Studio, command line ok.

[Timothy_W_]

Hi

I am running Visual Studio 2015 and Intel Parallel Studio XE 2016 Composer Edition.

I have many Fortran projects in visual studio.  Many of them build DLLs.  However, because of circular dependencies with other DLLs, the DLL projects are dependent on the object files only.  Other Fortran projects compile the object files.

One Fortran DLL project has 636 object files.  If I just build only this project, Visual Studio eventually crashes after 5 - 10 minutes. If I just run the linker on the command line, the DLL is created no problems.

Why?  I have not tried it yet but if I create a C++ DLL project and give it just the object files as dependencies, then that would run just like the command line.  Is there differences between a Visual Studio C++ project and a Visual Studio Fortran project if I am only building a DLL from an object file?  When I run the linker on the command line, I am running the Microsoft compiler.

Thanks

Tim 

[TimP]

Do you have IPO turned on when you build in Visual Studio?  Linking a large number of IPO objects may exhaust virtual memory, particularly under 32-bit Windows.  I have seen projects where I had to turn off both IPO and IP to make command line and GUI mode consistent.  There is some documentation about how the /Ob options are supposed to work but I couldn't get 2016 to work as described with those options.

I have been wishing for a way to duplicate options between building in GUI mode and command line.  For starters, you should compare what is shown in the build log with what you use on the command line.  Unfortunately, there seem to be hidden differences.

[Steven_L_Intel1]

Here's what I do - I build in VS and then open buildlog.htm. This will contain the series of ifort commands used to compile each object, as well as the linker options. I copy the command to the command line with default set to the project folder, then I can compile exactly the way VS does. I will usually remove some options that have no effect on compiling, such as /Qoptions,link.

[Timothy_W_]

Thanks for the responses.

I am not doing any optimization.  IPO etc all turned off.

I was able to mimic the linker command on the command line and in Visual Studio.  The command line linker used very little memory 10 mb or so and ran through the approximately 600 object files quickly.  Each object file is small, most only define one subroutine.

The command I used was:

link /OUT:"C:\rewrite\Debug\sc.dll" /VERBOSE /INCREMENTAL:NO /NOLOGO /LIBPATH:"C:\rewrite\Debug" /NODEFAULTLIB:msvcrt.lib /NODEFAULTLIB:libcmt.lib /MANIFEST:NO /SUBSYSTEM:WINDOWS /MACHINE:IX86 /DLL /qnoipo lapack.lib blas.lib myfile1.lib myfile2.lib ...

Under Visual Studio Fortran project, the IDE would pause for minutes processing something.  The memory usage for devenv was very high, it would reach 3 GB.  The IDE would finish generating the DLL then crash.  I noticed it completed by looking at buildlog.htm.

The command line in the Linker command line properties is identical to the command I ran on the command line.  I confirmed it by looking at buildlog.htm

I created a new project under the same solution.  This time I made it a C++ DLL project and gave it the same *.obj files (generated from Fortran) I gave it the additional directories to find all the dependent *.lib files including ifconsol.lib from C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2016\windows\compiler\lib\ia32_win.  I made the link commands very close to what was running above.  The dll was generated quickly, Visual studio memory use does not get much beyond 500 MB.  This time visual studio can use msbuild instead of devenv.  Visual Studio did not crash.

Is there any drawbacks for this method?

It would be very nice if Intel Fortran's integration with Visual Studio used msbuild, I will have more on this in another forum post soon.

Thanks!

Tim

[Steven_L_Intel1]

One issue is that Visual Studio lies to you about the link command - it actually invokes xilink. Does using that at the command line make a difference?

That devenv is taking memory tells me that it isn't the actual tools but something else. Is this a project you can share with us (either by attaching a ZIP or through Intel Premier Support)?

[Timothy_W_]

xilink on the command line works the same as link.  It is fast and does not use much memory.  It completes, no problems.  I used the same command line as link just replaced the executable name.

Unfortunately, I can't give you the project.

What is the difference between xilink and link?  I am improving the build process.  I think previously link was used in the scripts to build the DLLs not xilink.  I am moving the projects to Visual Studio.

Thanks

Tim

[Steven_L_Intel1]

xilink looks to see if you're linking any files compiled with /Qipo. If it finds any, it runs them through the optimizer and then calls link. If you don't use /Qipo then you don't need xilink.

As I said above, it is not the actual compile/link that is the issue but something in the VS environment itself.  It would be interesting to see what the minimal project you can create is that shows the problem (even if it's an incomplete program.)

[Timothy_W_]

Thanks Steve.  I might try to create a dummy project to recreate this problem if this continues to be in my way.

Regards

Tim