Intel® C++ Compiler
Community support and assistance for creating C++ code that runs on platforms based on Intel® processors.
7944 Discussions

icc speed issue due to intense access to /var directory

mzivi
Beginner
328 Views
I'm having some speed issue while compiling a project with icc (unfortunately I can't provide the source code). I have compiled my project in two different machine (both running Red Hat 4.1.2-44):
machine1: Intel dual-core Xeon 3.00GHz 4MB cache (model 5160 "Woodcrest")
machine2: Intel quad-core Xeon 3.00GHz 2x6MB cache (model E5450 "Harpertown")
The compilation is about 5 times slower on machine2. I pinned down the problem to the fact that icc writes a lot of stuff on the /var directory, which as it happen, in machine2 is on a disk much slower than machine1. monitoring the system resource usage during the compilation I observed that:
machine1:
* around 2 seconds for compilation (>90% usage of core 0) with no io operation;
* around 1 second during which the cpu is iddle and there's intense io traffic on the /var disk, around 6MB are witten as a sequence of 1000 transactions each one of around 5KB.
On machine2 the same happen but the second phase (cpu iddle, io traffic) takes 5 seconds instead of one because the disk is so slow.

I strace-d the compilation, and there's no explicit system call involving the /var path (apart from a couple of unix sockets in which very little data goes through, passwords and stuff like that), so I take it as that some of the system functions is actually doing stuff on /var internally - maybe it has to do with file locking or something like that? I also run the same compilation with the option -MM so that no compilation is done but all the dependencies are checked. Same situation as above without the 2seconds of cpu mangling.
I did the same analysis using gcc instead (gcc version 4.3.2). Compilation time is reduced from 2 seconds to a around 1.5 on both machines, but the time spent with the cpu iddled and intense io traffic on /var is totally removed, making the whole process much faster especially on machine2.
My questions are: why does icc require so much writing on /var (although not directly but via some system call) when gcc doesn't seem to be doing that at all? are there any chances this could be improved? What is being written on /var? is there a way I could make icc using a disk on ram instead of /var to speed up the thing?

Help is much appreciated, thanks.
0 Kudos
4 Replies
TimP
Honored Contributor III
328 Views
Please check your environment variable settings TMP TEMP TMPDIR. In the compiler .htm documentation, the section "Temporary Files..." explains how the compiler should be choosing a directory for temporary files. My guess is that you are directing them inadvertently to /var or /var/tmp, but you could use this facility to direct it to your fast storage. Incidentally, compile failures result in failure to clean up this space, so it is good to use a directory where you have a scheduled periodic cleanup.
0 Kudos
mzivi
Beginner
328 Views
Quoting - tim18
Please check your environment variable settings TMP TEMP TMPDIR. In the compiler .htm documentation, the section "Temporary Files..." explains how the compiler should be choosing a directory for temporary files. My guess is that you are directing them inadvertently to /var or /var/tmp, but you could use this facility to direct it to your fast storage. Incidentally, compile failures result in failure to clean up this space, so it is good to use a directory where you have a scheduled periodic cleanup.

None of these variable is set in my environment. I checked that setting a TMP variable pointing to a path not in /var nothing changes.

0 Kudos
TimP
Honored Contributor III
328 Views
Did you check whether /tmp may be redirected to /var/ ?
Does the compile option '-watch all' give any information?
0 Kudos
mzivi
Beginner
328 Views
Quoting - tim18
Did you check whether /tmp may be redirected to /var/ ?
Does the compile option '-watch all' give any information?

Yes, /tmp is not redirected anywhere and in on a separate partition from /var. I don't know what the compiler option '-watch all' is. I couldn't find it in the documentation (I'm using icc 11.1) and is not recognized by the compiler as a valid option.

0 Kudos
Reply