Can we improve link times with IPO in VS2010?

[Mark_Lewy]

One of our VS2010 solutions contains two static libraries that are built with /Qipo in release configuations.  These are then linked into a console application project and a DLL project, also with /Qipo set.  The link phase for the console application and the DLL takes well over 10 minutes each; I would like to improve this.  Disabling IPO in the console application and DLL projects significantly reduces link times, but also significantly impairs the performance of the application, so while useful for experimenting, this is not desirable for production use.

The documentation suggests that the /Qipo and /Qipo-jobs options (amongst others) may help with build performance, but they do not have IDE equivalents and specifying them as additional options in the Command Line property page for the static library or console application projects seems to have no effect.

Any suggestions for improving our link times would be welcome.

We are using Intel Fortran Composer XE 2013 Update 1 for Windows.

[dwwade]

Are you using /fast or /O3 and /ipo? I've been investigating a similar but not the same issue that causes me considerable grief with /ipo as well. I have linking times that in VS just sit with "linking" for as much as 2 hours or in some cases even longer for the final product to finish linking. I have three libraries, two of which I compiled and the third compiled by a vendor with which I have no clue what options they use for options. I put everything in the command-line using ifort and /link and found that by merely changing from /fast to /QxHOST /O2 /Qipo rather than using /O3 the linking time shortened to just a couple of minutes. I don't know why /O3 causes /Qipo to increase the time to link, but I tried it with /Qipo taken out and /O3 and it also didn't take nearly as long. I have the equivalent of /fast turned on in all of the libraries and they took forever to compile, but it doesn't seem to effect the link time of the final product with /O3 turned off. So you know, one library is about 5k lines, another is 75k lines (which depends on the first) and the final one is approximately 265k lines of code according to the vendor. My code depends on these libraries is only 3k lines of code, so I wouldn't think that it would be the cause of so much grief for /O3. Maybe Steve or IanH will point out what is causing this.

[Mark_Lewy]

Thanks dwwade. Yes, we use /O3 for some of our sources. Here's some more details... In the two static libraries, most of the source files are compiled with default options for release builds with these exceptions: iw2d_sim: /debug:full /Qparallel /Qopt-matmul- /Qopenmp /Qdiag-disable:7025 /Qauto /libs:dll /threads /Qipo There are a couple of files in iw2d_sim that are compiled with /O3. sim: /debug:full /Qparallel /Qopt-matmul- /Qauto /libs:dll /threads /Qipo There are four files in sim that are compiled with /O3 /Qopenmp The files that are compiled with /O3 are those identified as containing hotspots by VTune. Iw2d_sim consists of 38580 lines of non-comments. Sim consists of 57169 lines of non-comments and depends on iw2d_sim. Including sim and iw2d_sim, the console and DLL projects depend on the same 8 projects each, but only the console app project, DLL project, sim and iw2d_sim are compiled with /Qipo. Our build process builds the DLL and console application on the Win32 platform and the console application for the x64 platform, so that's at least 30 minutes link time.

[dwwade]

I'm assuming you are on newer hardware better than me, and thus that much faster than me. I'm using a Core 2 Duo E6850 using Windows XP with a pathetically slow 256 GB hard drive. If you use verbose or whatever linking options that show some progress, does yours sits at the "linking..." line without moving on for most of the 10 minutes? I saw in a previous post that Steve mentioned that it was related with xilink having to dig into all the object files to look for something specific that ifort creates for the ipo setting, but I don't know if that is the same thing. I'm still a bit puzzled by exactly where which optimizations are being performed. We didn't purchase Studio XE, despite the fact that the it says that we have it in the Start menu, so I am manually searching for optimization requirements. All of my libraries are compiled in as static, which I'm sure hurts my link time.

[Steven_L_Intel1]

Please keep in mind that the "linking" step for /Qipo also includes optimization and code generation, and for large programs this can indeed take a long time. Adding the /Qipo-jobs, etc. options for a static library project will have no effect, since the "link" step does not occur.

[Mark_Lewy]

Thanks Steve. I appreciate the work that IPO is doing, however a question remains. Should I expect the Qipo-jobs etc. options to have any effect on the link time, if, say, I put the sources that were in the static libraries and the main program into a single console application project? My experience of doing this with the source for sim and iw2d_sim would suggest not; there is a single ipo_*.obj file and a single fortcom process running at a time while linking occurs on my machine with Additional Options set to "/Qipo-separate /Qipo-jobs:8"in the Fortran->Command Line property page. For information, my machine contains 2 Xeon E5530 processors (so 8 available cores).

[dwwade]

Steve, one side question related to this then. If we do a /Qipo and it compiles in what we need for it to analyze in later steps such that we can make a static library, should the linker take any time at all since everything is done in the final executable or does the xilib program that gets called when we make a library still do a lot of the nitty gritty work?

[Steven_L_Intel1]

I did a bit of experimenting and wasn't able to see that /Qipo-jobs had an effect when used in a VS project. Perhaps I was missing something, or the program I built wasn't big enough to show it - I will ask the developers to see how this can be done. When you build a static library with /Qipo, the compiler creates "pseudo-objects" in the library that contain compiler intermediate code. None of the optimization/code generation work is done by the librarian. It's only when an executable is linked that the "nitty gritty work" is done.