/Qipo with 2500 files -- is that a problem?

[hendrix__marcel]

I try to compile the ngspice SPICE program with the Intel compiler on Windows 7 64bit, using Visual Studio 2013 Community. The machine has 32GB RAM. NGSPICE is a program with over 2500 files (2400 .obj in the final result). It is not possible to turn on /Qipo as it results in lots of [~44] object files that can't be found (the compiler seems to be doing some 4 passes before giving up). With /Qip the code is some 20% slower than with the old Microsoft compiler, which makes using the Intel compiler somewhat pointless :-) Id don't see a pattern in the files that can't be found, although sometimes they come from the same director(y)(ies). I have seen one occasion where the compiler complained it could not write a file. As /Qipo does not work, the advisor and instrumentation tools also fail. I have not checked carefully, but I think the profiler never works, even when /Qipo is set to /Qip. It is certainly possible that I am doing something wrong. Is there a limit on the number of files that can be processed, or on the number of files that can be passed to the linker? -marcel --------------- error message ------------- ... 1>C:\ngspice-ngspice\src\frontend\com_let.c(15) (col. 1): : error : 04010002_1670 1> 1>xilink: : error #10014: problem during multi-file optimization compilation (code 4) 1>xilink: : error #10014: problem during multi-file optimization compilation (code 4)

[TimP]

If you wish to use ICL IPO on a large project, you must read the ICL documentation index>compiler reference>optimization>IPO>IPO for large programs.

/Qip works on functions in a single file.  If a project is organized appropriately (not easy), it may be capable of optimizing better than /Qipo, but it still consumes lots of memory and may not run if compiling for 32-bit target.

You didn't give enough particulars on your use of Advisor, but I believe the current one expects compile options like /Qparallel-source-info:2 -debug:inline-debug-info -O.  There is a separate forum for Advisor. 

My experience with Advisor is that it misses legacy looping (more specifically loops without a pre-determined count), even possibly while() and do..while().  If you don't agree with Intel's terminology excluding those from "code modernization," I sympathize.

I guess Visual Studio 2012 qualifies as an "old Microsoft compiler." That one appears to have more instances than newer ones where it can out-perform ICL in the absence of  __restrict and #pragma insertion.  __restrict enables different optimizations with CL and ICL.

[hendrix__marcel]

Hi Tim, Thanks! > If you wish to use ICL IPO on a large project, > you must read the ICL documentation > index>compiler > reference>optimization>IPO>IPO > for large programs That looks useful. Can you provide a more precise reference? I could not find it in to distribution I downloaded and installed last week (last versions of everything). I tried /Qipo40 but that long shot did not work. I don't think it is a memory issue because I have 32GB and not more than about 10GB seems to be used. I work on 64bit exclusively. Advisor will not help me much as NGSPICE is extremely fond of linked lists and there is hardly any straight do loop in the source. I think getting rid of the linked lists would be a good thing for speed, but it is a huge job because of the inherently sparse matrices involved. -marcel

[Kittur_G_Intel · Accepted solution]

Marcel, some key points to note in addition to what Tim pointed out. 1) Make sure the app works without -ipo. That said, try ipo only on performance critical files/libs instead of the entire large project which will give you the ROI and reduce build time drastically. SInce IPO is indeed very memory bound process it's very important to remember that memory is a bottle neck. Use /QipoN switch to generate true object files for link time compilations but also specify /Qipo-jobs<n>  where n is equal to the number of processors so that that many jobs can be executed simultaneously during the link phase and thus speeding up build time especially on multi-systems with sufficient memory. The user reference manual at: https://software.intel.com/en-us/node/522666 

_Kittur

[hendrix__marcel]

Thanks Kittur! Unfortunately, even with your help, the manual, and several lengthy tries, I couldn't get it to work satisfactorily. In the end I turned ipo on only for the 'critical files' (as mentioned by Vtune). Then it works, however, the result is still about 10% worse than the MS compiler produces. (Maybe a project-wide /Qip overrules file-individual /Qipo ?) -marcel

[jimdempseyatthecove]

While VTune may identify the critical files, you may have to run Find in Files to locate the files which call the critical files. These would be the files that would benefit from multi-file IPO with these critical files. Your end solution would then have three levels, with the middle level being multi-file IPO'd with the lowest level, but not between middle levels nor with upper most level.

Jim Dempsey

[hendrix__marcel]

Hi Jim, But .. isn't that what profile-guided optimization should be doing (I don't see the combination mentioned in the manual)? That triggered me in to trying profile-guided optimization with VC++, and to my utter delight that shaved off 20% of the run-time and 80% of the exe size (it optimized 1.7% of the files for speed and the rest for size). I have switched to Visual Studio 2013 Community because it is needed for the Intel tools, and saw that the profiling option is not disabled like it was in my older VS2013 Express. A very welcome side-effect, although I am sure the Intel C compiler could pile something on top of that -- if I knew how to ask for it. -marcel

[jimdempseyatthecove]

>>if I knew  how to ask for it.

I run into that a lot. A nice feature may be there, but the documentation does not lend itself to discovery of the feature without a full re-read of the complete documentation. Too often I have to resort to an external search tool to find topics in the documentation.

Jim Dempsey

[Kittur_G_Intel]

Marcel, Jim's comment is right on.  BTW, you can find more details at PGO   If using PGO helped with MS then your code is a good candidate for optimizing with pgo. Obviously you'll have to use that feature with Intel compiler as well 'cause you want the compiler to optimize more frequently executed paths in your code. If you still think there's a performance issue then you'll have to attach a reproducer so we can file the issue with the developers, appreciate much.

_Kittur

[hendrix__marcel]

Hi Kittur, Thank you for your thoughtful advice. I tried to use PGO, but when I rebuild, I get the below error. error MSB6006: "icl.exe" exited with code -1073741515. C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120 \Platforms\x64\PlatformToolsets\Intel C++ Compiler XE 15.0\Toolset.targets 355 > If you still think there's a performance issue then > you'll have to attach a reproducer so we can file the > issue with the developers, appreciate much. Because of the nature of the problem, I can not define a minimal reproducer for the ipo problem. -marcel

[Kittur_G_Intel]

Understood, Marcel.  BTW, try with the latest release as it could be related to the issue at:  https://software.intel.com/en-us/forums/topic/557303. 

_Kittur

[jimdempseyatthecove]

I forgot to mention that the multi-tier approach permits more parallelization in the build process.

Jim Dempsey

[Kittur_G_Intel]

@Marcel: I assume the earlier responses helped resolve your issue? Let us know if you need any further clarification, appreciate much.

_Kittur

[hendrix__marcel]

Hi Kittur, Thanks for asking. My current feeling is that "/Qipo with 2500 files" is really problematic for the Intel compiler. I posted the error numbers of several crash dumps, but nobody was able to tell me what they meant. My time was not wasted as I learned a few new tricks to try with the MS compiler (which has no problems that I can see with /LTCG and profiling a 2500 file project). I have decided to wait a bit and see if the issues in the Intel software get resolved -- I am betting a further 20% or more speed increase should be possible. Until that time, I focus on high-level optimizations in the SPICE code. There are more than enough to keep me busy for a very long time. -marcel

[Kittur_G_Intel]

Thanks Marcell for the feedback. Again, profiling the application with Vtune and applying IPO to critical files/hot spots (instead of whole program optimization) should help. Additionally it should also reduce the build time as well. I assume you filed issues in Premier Support and posted error numbers there? IPO should handle that many and more files per-se but it's hard to decipher what's going on unless the issue is triaged accordingly. Anyways, just letting you know that the latest version 16.0 is out and an update release will happen soon which you can try out to see if those errors you mention gets resolved. Again, thanks for your input which I'll pass on to the product team.

Regards,
Kittur