IVF uses Multicore during compilation?

[onkelhotte]

Hi there,
Im using the 11.0. Version of IVF with VS2008.

Now Im "chosen" to continue a project from a colleague who retires end of 2009.

This project is a mixed language project, CVF6 and C++. This project is very large, compilation takes over half an hour when you rebuild all on his machine (sources itself are over 200MB!!!).

Ive noticed that, when I compile a project, the fortcom process does only use one core (it never exceeds 50% cpu).

Does IVF 11.1 use all cores while compiling or is it considered in a future version of IVF?

Thanks in advance,
Markus

[Steven_L_Intel1]

The compiler itself is single-threaded. We have work ongoing to multithread at least some parts of the compiler, but that's on the back burner for now. What you might look at is splitting the project into multiple projects (as static libraries) and use Visual Studio's ability to build projects in parallel. You'd have to ensure that each group of files didn't depend on modules from another group.

I'd also highly recommend splitting the very large source file. This will significantly reduce the memory and CPU usage of the compiler.

[onkelhotte]

Quoting - Steve Lionel (Intel)

The compiler itself is single-threaded. We have work ongoing to multithread at least some parts of the compiler, but that's on the back burner for now. What you might look at is splitting the project into multiple projects (as static libraries) and use Visual Studio's ability to build projects in parallel. You'd have to ensure that each group of files didn't depend on modules from another group.

I'd also highly recommend splitting the very large source file. This will significantly reduce the memory and CPU usage of the compiler.

Thanks for the fast answer!

There are hundreds of source files, they are split up already. Maybe I try to seperate the solution into some projects, but after I migrated the solution to IVF.

Markus

[TimP]

Have you considered the Multiple Process feature of VS?
http://msdn.microsoft.com/en-us/library/bb385193.aspx
Of course, I'm not so familiar with that, as gnu "make -j 4" does a good job with Makefile builds.

[Steven_L_Intel1]

Quoting - tim18

Have you considered the Multiple Process feature of VS?
http://msdn.microsoft.com/en-us/library/bb385193.aspx
Of course, I'm not so familiar with that, as gnu "make -j 4" does a good job with Makefile builds.

That doesn't work well with Fortran as it doesn't take module dependencies into account. If you don't have modules, you can try that.

[jimdempseyatthecove]

Marcus,

The VS parallel build procedure works well for me...

At least after I figured out how to do the project dependencies with Fortran. Not getting the dependencies list right together with project organization results in circular dependencies. The major bug-a-boo is the Fortran .MOD files. Since these are (dependency) equivalent to both input and output files the dependency checker needs a little help from you. My suggestions are to visualize the dependencies as a tree. AChristmas tree, viewed top down. You have a key project, then branch out in parallel projects, then branch again, etc.. thena collation/assembly project for final app.

In the top projectplace the module files that contain the PARAMETERS use elsewhere.
In the next project layer (you can have multiple projects at this layer), place the module files that use the parameter module(s).
In the next project layer (you can have multiple projects at this layer), place the library projects used by the app.
In the next project layer (you can have multiple projects at this layer), place the larger functional groups/sub-programs.
In the final project layer (typically one project) assemble all the pieces

If you don't get it right, the symptoms are after build, a subsequent build with no intervening change will build something. If you get it right, the dependency check will indicate no build required of component(s).

In VS there is a property specifying the max number of "processors" to use for parallel build. Sometimes you have to diminish this from total number of available processors.

The solutions I have experience with contain 13 projects, ~750 files (mayby 50 modules).

Jim Dempsey

[onkelhotte]

Quoting - jimdempseyatthecove

Marcus,

...

In VS there is a property specifying the max number of "processors" to use for parallel build. Sometimes you have to diminish this from total number of available processors.

The solutions I have experience with contain 13 projects, ~750 files (mayby 50 modules).

Jim Dempsey

Hi Jim,
thanks for your answer.

Do you mean the setting Tools -> Options -> Project & Solutions -> Build and Run -> Maximum number of parallel project builds?

I use VS2008.

Markus

[jimdempseyatthecove · Accepted solution]

Quoting - onkelhotte

Hi Jim,
thanks for your answer.

Do you mean the setting Tools -> Options -> Project & Solutions -> Build and Run -> Maximum number of parallel project builds?

I use VS2008.

Markus

Markus,

Yes, that is what I mean.

Note, on this forum are postings relating to problems with parallel project builds. Most of the suggestions are if you have problems resort to serial build (Max #=1). Considering your build times, it would be appropriate to spend some time in working out the parallel builds. As stated in one of my earlier posts is the Fortran modules produce both a .MOD file and .OBJ file. The dependencies on the .OBJ file are quite usual with respect to Visual Studio. The .MOD files are more like pre-compiled headers. Getting your project dependencies right is a bit tricky at first. It is easier to do this when you separate the data types within the modules from the code within the modules. At times this may require using two modules where you coding preference would have been to use one. I tend to use

!MOD_FOO.F90
module MOD_FOO
... data declarations
end module MOD_FOO
-------------------------------
! MOD_FOO_code.F90
module MOD_FOO_code
use MOD_FOO
... code
end module MOD_FOO_code

By having the two files you can at times break the circular dependencies.

Jim Dempsey