Multi-processor Compilation option /MP is ignored

[jirina]

I am using IVF Compiler XE 13.1.0.149 on Windows 7 Professional 64-bit and I would like to use the /MP compiler option. It worked in previous versions of IVF Composer, but the latest version seems to ignore this parameter. I found here that is it not recommended to use /MP; is this recommendation still valid even in the latest version?

[jirina]

I have just tested various configurations of installed versions of IVF Composer XE 2013 - Update 1 only, both Update 1 and 2, Update 2 only, and the problem in Update 2 persists - the /MP option is ignored and just one core is used for compiling. When compilers are switched to those from Update 1, everything works just fine and all four cores on my i5 processor are utilized.

[Steven_L_Intel1]

This was unintentional. A change made elsewhere broke this in Update 2. I will report this to the developers and will ask that it be fixed for update 3. Issue ID is DPD200241461.

[Steven_L_Intel1]

The fix was made for update 3.

[Andrew_Smith]

It seams to be broken in Intel(R) Visual Fortran Compiler XE 14.0.1.139 [Intel(R) 64] under VS2010

[Steven_L_Intel1]

I can't reproduce a problem here. I've attached a ZIP with some sources - please create a project from them, set the multiprocessor option under Fortran > General, do a build, and attach a ZIP of or insert the build log.

[mecej4]

A guess about the reported problem: /MP has to precede other options to take effect. If VS2010 is set up with other options in addition to /MP, it may not be putting the /MP first. The build log should contain information from which one may ascertain if this is happening.

[Steven_L_Intel1]

Rather than add /MP under Command Line, use the "Multi-processor Compilation" property under Fortran > General.

[Andrew_Smith]

Attached build log as instructed. Here is also contents of output window. The compile was too quick to see if sources compiled in parallel.

1>------ Build started: Project: Console4_0, Configuration: Debug Win32 ------

1>Compiling with Intel(R) Visual Fortran Compiler XE 14.0.1.139 [IA-32]...

1>Source3.f90

1>Source1.f90

1>Source2.f90

1>Console4.f90

1>Linking...

1>Embedding manifest...

1>

1>Build log written to "file://D:\temp\Console4_0\Debug\BuildLog.htm"

1>Console4_0 - 0 error(s), 0 warning(s)

========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

[Steven_L_Intel1]

It did the MP compilation according to your build log. To see this, look for more than one source file being compiled in a single ifort command.

[Andrew_Smith]

In my project I have more than 600 source files. There are several unrelated groups of files due to extensive use of polymorphism. I can confirm that a change in one group of files does not cause a compile of the others so theoretically it should be possible to compile several files at once for a significant proportion of the time on my 6 core machine. However the build log for a re-build shows only the first and last compiler invocations had more than one source file, actually just two files for each of these.  The task manager shows 17% or lower CPU use throughout the build. Is /MP broken or just not intelligent enough?

[jimdempseyatthecove]

Andrew, I am going to throw out an unfounded observation, you can try a build to see what happens.

The IPO portion of the build now occurs at Link time, not Build time. IOW during Link, the linker calls back the compiler to re-do the .obj file. This somewhat iterates until satisfied. What may be happening is the goodly portion of "compile tilm" is actually instigated after the first Link phase following the first pass at compilation. This first pass may be multi-threaded build, but (assumption) the secondary IPO passes are not.

You can try setting IPO to single file, then run the build and observe the CPU time. If this improves utilization, then for the performance critical sections of code, you can enable multi-file IPO just for those files.

The above is a wild hunch. It may be worth a try.

Jim Dempsey

[Andrew_Smith]

Sorry Jim but in my last post I was referring to a compile of the DEBUG configuration where I have disabled optimization. The link stage is a tiny proportion of the total time. My release config is currently already set to single file optimization.

This is the ifort command:

ifort /nologo /debug:full /MP /Od /QxHost /I"..\..\3rdParty\IMSL\emt64" /I"..\..\3rdParty\MKL\include\intel64\lp64" /DIntelQuad /Dx64 /Ddebugv /Ddebug /Qopenmp /standard-semantics /Qopenmp-report2 /debug-parameters:all /warn:declarations /warn:unused /fpe:0 /fp:strict /fp:except /Qfp-speculation=off /module:"x64\Debug\\" /object:"x64\Debug\\" /traceback /check:pointer /check:bounds /check:uninit /check:format /check:output_conversion /libs:dll /threads /Qmkl:parallel /c /Qvc10 /Qlocation,link,"c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64" [filename here]

[Steven_L_Intel1]

The grouping of files is not as smart as theoretically possible. It does work well for many of our customers. If you'd be willing to provide us a ZIP of the sources so we can see if there are reasonable improvements, we'd be glad to take a look. You can use Intel Premier Support (ask that the issue be assigned to me) or send me a private message if you like.
 

[FortranFan]

In my projects, I don't see any lines in the build log that have more than one source with the ifort command.  As Steve suggested, I used the "Multi-processor Compilation" property under Fortran > General.

Note my projects that have more than one Fortran source file tend to be static library projects where the Librarian instead of the Linker comes into play.  I assume, therefore, the comments by Jim about the back-and-forth between the Linker and Compiler are not applicable in my case?  Anyways, turning IPO on or off seems to have no effect.

Re: the comments by Andrew

Is /MP broken or just not intelligent enough?

and Steve

The grouping of files is not as smart as theoretically possible. It does work well for many of our customers.

I think the issue lies with source files that make use of many of "modern Fortran" constructs including heavy use of modules (USE associations)with derived types with abstract declarations and concrete type extensions, etc.  That might be one common ground between Andrew's project and mine.  I don't know if Intel developers' test suite includes such cases.

Unfortunately I don't have the liberty to share my source files.  But I would think Intel staff can easily reproduce the problem if they focused on any realistic application of polymorphism that included multiple, independent source files which depend on some common module files e.g., one module file that defines an abstract type, another that extends the abstract into a base concrete type, and many other module files that are extended instances of the concrete type and which are, in effect, independent of each other.  The option /MP should lead to parallel compilation of many of these files, but I'm not seeing it.