Slow compiles with 2013 compiler

[Jacob_Williams]

For my project, I’ve noticed very long compile times for the 2013 version of the compiler (compared to the 2011 version).  Files that used to take seconds are now taking minutes to compile.  Has anyone else noticed this?  It is consistent for updates 0,1,2 (and also on Linux).  I’ve got a fairly complicated project (a few hundred modules, split into several libraries and one main program), and am using a lot of the newer object oriented stuff.  I notice that the low level files (old style f77 code, the library modules, etc.) compile as fast as before.  However, once it gets to the main program where there are a lot of dependencies on the low-level modules, it slows to a crawl.  I suspected that the compiler is performing an extraordinary amount of code optimization, but the problem persists with debug builds where all the optimization is turned off.  Any ideas?

[Andrew_Smith]

I have a large Fortran project where module processing has been a major issue since XE2013. I saw many Internal Compiler Errors and some slow compiles. Often a slow compile lead to an ICE. I have got around it all by having a private statement in most of my modules, then only having whats really neccessary as public. This took me weeks to do.

Another issue I got was that sometines when calling a procedure in module 2 from module 1 and passing some derived type object, the compiler complained that the type defintion of the dummy argument of the module 2 procedure was unresolved, even though module 2 compiled without incident. I got around that by putting the use statement that defines the derived type in the module 2 header rather than in the procedure defintion.

I dont know if any fixes happened in update 2 because my project is already stable before then.

[Steven_L_Intel1]

We know of a general problem with modules and compile-time performance that started in the 13.0 compiler. We've been doing a lot of work which seems to have resolved that, but the changes are deemed too significant to go into an update so they'll be in a release later this year. Some changes did go into update 2.

[Steven_L_Intel1]

You may want to try two temporary switches we added that help some users with module issues. In Visual Studio type these in under Fortran > Command Line > Additional Options

First try /switch:cq237837

You can also add /switch:cq235485 but this one is more to fix an issue with accessibility. Both of these require Update 2.

[Jacob_Williams]

I tried the switches, and they didn't work for me.  I also just downloaded 2013 Update 3, tried it with and without the switches, and it still takes forever to compile.  Also, after about 1 hour of compiling, I start to get these messages:

2>Fatal compilation error: Out of memory asking for 2097160.
2>ifort: error #10298: problem during post processing of parallel object

I have 8 GB or RAM, so that should be enough.  With the 2011 version, the whole project takes < 10 minutes to compile with no problems.  I guess I'll have to stay with the 2011 version until this issue is fixed.

[Steven_L_Intel1]

Jacob, would you be willing to provide us your sources through Intel Premier Support so that we can make sure we fix the problem?

[jimdempseyatthecove]

Jacob,

Experiment with /Qipo-jobs??? and set to 1

Not sure of correct syntax to enter number

/Qipo-jobs1
/Qipo-jobs:1

I know you would prefer parallel IPO, but try a build with 1 job and see what happens.

Jim Dempsey

[Jacob_Williams]

Steve: the problem is that I don't think I can send you the code (some is third-party that I don't have a license to distribute, some is ITAR, etc.). 

Jim: I'll try what you suggest (although this is a debug build with IPO disabled, so it shouldn't be doing any in the first place).  I've already tried various build settings, and haven't found any combination that doesn't experience this problem.

[Steven_L_Intel1]

Jacob, please attach a ZIP of the buildlog.htm for the failed build. This is obviously not from the module processing. Are there lots of or large libraries that are part of this project? Even if you are not doing an IPO build, the xilink "pre-linker" is invoked and I have seen issues where this step is very slow. A workaround is to rename xilink.exe in the Fortran BIN folder and then to copy in a copy of link.exe (from the VC\BIN folder) and rename it xilink.exe.  See if that helps. It would mean you couldn't use IPO.

[Jacob_Williams]

Will send the files as a private message.

[Steven_L_Intel1]

Now that I look at Jacob's logs, I am pretty sure that this IS a module issue and not related to xilink. The compiler is running out of memory. Without access to the sources so I could reproduce it, all I can suggest is to see if our next major version, due out later this year, resolves the issue.

[Olsen__Oystein]

I have hit long compile times again. The attached archive should demonstrate this on Windows and Linux, although I have not tried on Windows for a while. To compile on Linux:

- Extract archive

$ cd slow_compile/hs-build

$ cmake -DCMAKE_BUILD_TYPE=DEBUG ../hs

$ make

The cmake and make command on Windows should be

$ cmake -G"NMake Makefiles" -DCMAKE_BUILD_TYPE=DEBUG  ../hs

$ nmake

I did have memory problems too. It would not compile with 16GB of RAM. I solved that by not using linked lists with unlimited polymorphic components.

[Steven_L_Intel1]

oysteinolsen, which file(s) in particular are a problem for you?  That's a large and complex build.

[Jacob_Williams]

Note: I haven't tried to compile oysteinolsen's example, but I noticed that it is similar to my project in that: 1) it is a complex program with a lot of modules, and 2) it uses a lot of the newer F2003 features (polymorphic classes with procedures, generic operators, linked lists, etc.)  Note that mine has significantly more modules (~240 vs. ~40 in this one).  I would bet money that it is the new language features that are causing the problem.

[Olsen__Oystein]

Steve,

the noticeable slow-down starts with hsspacecraft.f90 at 78%. From there, it gets progressively worse. The last file was still compiling after 12 hours. This is a relatively small part of my project and I have not been able to find what causes the slow-down. I can strip away parts and I only get a gradual decrease in compile times. I do suspect that it is caused by using unlimited polymorphic arrays in hsode.f90. I'll make another attempt at finding the exact cause.

With 12.1 it took about twenty minutes to compile the entire project, although I had to introduce a few work-around due to compiler bugs.

[Steven_L_Intel1]

In general, use of things such as polymorphic variables has no noticeable affect on compile time. Where we have seen problems is in following module USE chains, renames, public/private, etc. The more nested the modules get, the worse things can get. We've done a lot of work in the past few months to speed up this part of the compilation.

I will try your project when I can.

[Jacob_Williams]

Sorry to report that I'm still seeing the unusually-long compilation times for my project with the latest version of the compiler (v 14.0.0.103).  Steve, were you ever able to try out oysteinolsen's project?  Interestingly, I'm also now seeing what appear to be compiler errors.  It's telling me "this derived type name has not been declared" for some variables that should be available from a "use" statement for a module that already successfully compiled.  Adding an "only" for the variable fixes it.  I can try to create a simple example...but it seems like my problems only show up for complex projects, so I may not be successful.

[Steven_L_Intel1]

The error you're seeing now is one we know about and will be fixed in the October update. Jacob, would you please provide the source for the file that compiles slowly and that of any modules/include files it uses (directly and indirectly)?  Thanks.

[Jacob_Williams]

I'll try to come up with an example set to send you (I can't send it all).  It may take a while.

[Jacob_Williams]

I've manged to create a simplified example project with a subset of code to demonstrate the issue.  I'd like to submit it via Premier Support, but it won't let me. I don't have any products listed under "Product Name", so I can't proceed beyond the "Submit Issue" page.  How do I get this to work?  I do have a current license for Intel Fortran for Windows.

FYI: the problem is due to a large type that I have defined (it contains lots of other types, some of which contain polymorphic or allocatable variables).  If the type contains procedures, any modules that call these procedures takes a long time to compile.  Also, any modules where this type is allocated or deallocated takes a long time to compile.  The slow-down can be dramatic.  For the case where a procedure in the type is called, the v12 compiler takes 1 sec to compile the sample file, and the v14 compiler takes 23 sec.

[TimP]

I wouldn't be surprised, in case you are concerned with win32 (you didn't say), if recent compiler work were oriented toward Intel64.

If using win32, 8GB RAM won't be of any help with a standard compiler installation.

I note that support for some 32-bit Intel software tools (presumably not compilers) is about to be "deprecated."  It's difficult even to install 32-bit prerequisites on the linux versions best supported by Intel.