Linking takes too long

[evankw21]

I've just converted from Absoft to Intel as my FORTRAN compiler. The only thing I don't like is that it takes several minutes to re-compile. Specifically, the compiling stage goes very quickly, but after the Output says "Linking...", it takes several minutes. I'm using the MKL library for FFTs and two small libraries that I wrote, one in FORTRAN, the other in C. Does anyone know why the linking takes so long and if there's a way to speed up the link process? The Absoft compiler took just a few seconds for the same task. Thanks.

[TimP]

If you are comparing link speed of one compiler without link time optimization with a compiler which (optionally) performs such optimization as a prelude to linking (Microsoft /GL or Intel IPO) you may expect differences. For example, in the Visual Studio properties for Intel compilers, there are 3 levels of interprocedural optimization: none, within a source file, or link time IPO.

[Steven_L_Intel1]

It's not normal for linking to take a long time, unless you have a very large application or are using the IPO feature, as Tim mentions. Does this happen for any application, including a simple "Hello World"?

[jimdempseyatthecove]

Other than for IPO as tim18 mentions, there was a thread mentioning an issue where the developer was on a client in a client/server situation and where tools and libraries were located (and/or paths pointed to) the server. I cannot recall if the problem had to do with client authentication or some other Managed Network issue. Maybe this will jog someones memory and they can point out the forum post.

Jim Dempsey

[evankw21]

Thanks to all for the suggestions.

I found the IPO option, and it is set to "No".

I'll set up the simplest of programs and see if that takes a long time to link.

I am running the compiler in a networked situation where perhaps some other program is objecting to what the linker has to do. I read about someone who had firewall issues that caused slowness in some stage of the process.

[Steven_L_Intel1]

What exactly is "a networked situation"? Is the compiler and Visual Studio installed on your PC's local hard drive? Are you using projects on a network drive? Is your home directory on a network drive?

[evankw21]

Quoting - Steve Lionel (Intel)

What exactly is "a networked situation"? Is the compiler and Visual Studio installed on your PC's local hard drive? Are you using projects on a network drive? Is your home directory on a network drive?

No, nothing like that. All the code and libraries are local on a PC running Windows XP. A colleague who has helped me with getting ifort running has the same issue.

We use the HDF4 libraries. I wonder if there's something about them that would take a long time to link to.

[evankw21]

Quoting - evankw21

I've just converted from Absoft to Intel as my FORTRAN compiler. The only thing I don't like is that it takes several minutes to re-compile. Specifically, the compiling stage goes very quickly, but after the Output says "Linking...", it takes several minutes. I'm using the MKL library for FFTs and two small libraries that I wrote, one in FORTRAN, the other in C. Does anyone know why the linking takes so long and if there's a way to speed up the link process? The Absoft compiler took just a few seconds for the same task. Thanks.

I made a "Hello World" project, and it linked in no time. I then added my HDF call and all the things required to do it, and it still links quickly. So it looks like it's not the HDF library that's doing it to me. The projects that take a long time to link are not trivial. One of them that I looked at has 42 subroutines. Would this be enough to make the linking take several minutes?

My next suspect would be the MKL library. I link to a library of mine that uses MKL_DFTI.

If anyone has any more thoughts, I'd appreciate it. Waiting several minutes to link is really making it difficult to debug programs.

[Steven_L_Intel1]

No, 42 subroutines is trivial. Turn on the Linker property page "Show progress messages: Show some progress messages". This will show you how it is finding various libraries and you may be able to identify the sluggard. My guess is that the linker is searching network shares.

[evankw21]

Quoting - Steve Lionel (Intel)

No, 42 subroutines is trivial. Turn on the Linker property page "Show progress messages: Show some progress messages". This will show you how it is finding various libraries and you may be able to identify the sluggard. My guess is that the linker is searching network shares.

Steve:

I tried what you asked. After saying, "Linking...', it took about 90 s before the rest of it came out, and when the rest of it came out, it only took a second or two. I'll try the very verbose option next.

Evan

1>------ Build started: Project: abxp, Configuration: Debug Win32 ------
1>Compiling with Intel Visual Fortran 11.1.035 [IA-32]...
1>abx_init.f90
1>Linking...
1>Searching libraries
1> Searching E:EvanifortlibHDF42r3_sourcehdf4libreleasedllhd423m.lib:
1> Searching E:EvanifortlibHDF42r3_sourcehdf4libreleasedllhm423m.lib

.....

1> Searching C:Program FilesIntelCompiler11.1�35mklia32libmkl_core.lib:
1> Searching C:Program FilesIntelCompiler11.1�35libia32libiomp5mt.lib:
1> Searching C:Program FilesMicrosoft SDKsWindowsv6.0Alibuuid.lib:
1>Finished searching libraries
1>LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library
1>Embedding manifest...
1>Performing Post-Build Event...
1>1 file(s) copied.
1>
1>Build log written to "file://E:EvanifortabxpDebugBuildLog.htm"
1>abxp - 0 error(s), 1 warning(s)
========== Build: 1 succeeded, 0 failed, 3 up-to-date, 0 skipped ==========

[evankw21]

Quoting - evankw21

Steve:

I tried what you asked. After saying, "Linking...', it took about 90 s before the rest of it came out, and when the rest of it came out, it only took a second or two. I'll try the very verbose option next.

Evan

1>------ Build started: Project: abxp, Configuration: Debug Win32 ------
1>Compiling with Intel Visual Fortran 11.1.035 [IA-32]...
1>abx_init.f90
1>Linking...
1>Searching libraries
1> Searching E:EvanifortlibHDF42r3_sourcehdf4libreleasedllhd423m.lib:
1> Searching E:EvanifortlibHDF42r3_sourcehdf4libreleasedllhm423m.lib

.....

1> Searching C:Program FilesIntelCompiler11.1�35mklia32libmkl_core.lib:
1> Searching C:Program FilesIntelCompiler11.1�35libia32libiomp5mt.lib:
1> Searching C:Program FilesMicrosoft SDKsWindowsv6.0Alibuuid.lib:
1>Finished searching libraries
1>LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other libs; use /NODEFAULTLIB:library
1>Embedding manifest...
1>Performing Post-Build Event...
1>1 file(s) copied.
1>
1>Build log written to "file://E:EvanifortabxpDebugBuildLog.htm"
1>abxp - 0 error(s), 1 warning(s)
========== Build: 1 succeeded, 0 failed, 3 up-to-date, 0 skipped ==========

So now I set the verbose option to full. This time I timed in on my watch, and it was a little over 120 s to complete. Still, almost all the time was spent after the "Linking ..." command. I looked at Task Manager, and it said that fortcom.exe was using a single CPU. After a couple minutes I noticed that a task called debenv.exe took over from fortcom.exe. Here was the start of the printout:

1>------ Build started: Project: abxp, Configuration: Debug Win32 ------
1>Compiling with Intel Visual Fortran 11.1.035 [IA-32]...
1>abx_init.f90
1>Linking...
1>Starting pass 1
1>Processed /DEFAULTLIB:ifconsol
1>Processed /DEFAULTLIB:libifcoremt
1>Processed /DEFAULTLIB:libifport
1>Processed /DEFAULTLIB:libmmt


Does that shed any light?

Evan

[evankw21]

Quoting - evankw21

So now I set the verbose option to full. This time I timed in on my watch, and it was a little over 120 s to complete. Still, almost all the time was spent after the "Linking ..." command. I looked at Task Manager, and it said that fortcom.exe was using a single CPU. After a couple minutes I noticed that a task called debenv.exe took over from fortcom.exe. Here was the start of the printout:

1>------ Build started: Project: abxp, Configuration: Debug Win32 ------
1>Compiling with Intel Visual Fortran 11.1.035 [IA-32]...
1>abx_init.f90
1>Linking...
1>Starting pass 1
1>Processed /DEFAULTLIB:ifconsol
1>Processed /DEFAULTLIB:libifcoremt
1>Processed /DEFAULTLIB:libifport
1>Processed /DEFAULTLIB:libmmt


Does that shed any light?

Evan

Steve:
Just to make sure, I compiled the Release configuration, and the same thing happened. "Linking..." got printed very quickly, then fortcom.exe popped up in the Task Manager and used a CPU for 80 s, then disappeared. devenv.exe, which had been present but using no CPU, then started using the entire CPU for another 40 s or so. Then the print statements came pouring out for about 15 s. Then it said it was creating the Manifest file, then it finished about 5 s later.
Evan

[Steven_L_Intel1]

Mysterious. Some additional things to try.

First, I'll ask that you download and install 11.1.048 and try the build again. If it still takes a long time, do this experiment.

Open the vcbin subfolder of your Visual Studio installatioin. Rename link.exe to linkx.exe. Find notepad.exe on your system (probably in Windows), copy it into vcbin and rename it to link.exe. Now try linking again. This will fail, but I'm interested in knowing how long it takes before you see notepad come up with an error message. Once this is done, delete the fake link.exe and rename the old one back.

[evankw21]

Quoting - Steve Lionel (Intel)

Mysterious. Some additional things to try.

First, I'll ask that you download and install 11.1.048 and try the build again. If it still takes a long time, do this experiment.

Open the vcbin subfolder of your Visual Studio installatioin. Rename link.exe to linkx.exe. Find notepad.exe on your system (probably in Windows), copy it into vcbin and rename it to link.exe. Now try linking again. This will fail, but I'm interested in knowing how long it takes before you see notepad come up with an error message. Once this is done, delete the fake link.exe and rename the old one back.

Steve:

Could you give me a link to the 11.1.048 download. I've been clicking around for 10 minutes, but can't find it.

Thanks.

Evan

[Steven_L_Intel1]

Log in to the Intel Registration Center - and the link should be there for you.

[evankw21]

Quoting - evankw21

Steve:

Could you give me a link to the 11.1.048 download. I've been clicking around for 10 minutes, but can't find it.

Thanks.

Evan

My sysadmin was able to get the newest version and I then had to re-enter stuff under Tools => Options => IVF => Libraries and Includes.

It still took a long time after "Linking ..."

I then renamed the link.exe and replaced it with a link to notepad.exe. It still said "Linking ..." and waited a long time before coming back with the expected error:

Error 1 error #10036: unable to run 'link' Link

So why could it be taking so long just get to the linker???

Evan

[DavidWhite]

Quoting - evankw21

My sysadmin was able to get the newest version and I then had to re-enter stuff under Tools => Options => IVF => Libraries and Includes.

It still took a long time after "Linking ..."

I then renamed the link.exe and replaced it with a link to notepad.exe. It still said "Linking ..." and waited a long time before coming back with the expected error:

Error 1 error #10036: unable to run 'link' Link

So why could it be taking so long just get to the linker???

Evan

Evan,

Just a long shot here -- I had compile time issues when I am connected to our network via VPN because Visual Studio by default puts files in My Documents, which in our scenario are on a network drive. Do you have any settings pointing to a network drive. Similarly, when searching for the linker, if it is searching through PATH and there are any network drives before the location of the linker, then you could be waiting for these slow connections. Try putting the linker and compiler folders towards the start of PATH and see if that makes any difference.

Regards,

David

[jimdempseyatthecove]

Evan,

David's VPN post was the one I referenced in my prior message.

Jim

[Steven_L_Intel1]

Can you try a command-line build and see if the problem is still there? I think you've identified that the delay is before the linker is actually invoked, but I can't think of what it might be, especially when some of your projects link quickly.

Oh, I thought of another experiment. Open the folder C:Program FilesIntelCompiler11.1�46binia32 and rename xilink.exe to xilink2.exe. Open the VCBIN folder as above and copy link.exe into the compiler's binia32 folder and rename the copy to xilink.exe. Test your build again.

[evankw21]

Quoting - Steve Lionel (Intel)

Can you try a command-line build and see if the problem is still there? I think you've identified that the delay is before the linker is actually invoked, but I can't think of what it might be, especially when some of your projects link quickly.

Oh, I thought of another experiment. Open the folder C:Program FilesIntelCompiler11.1�46binia32 and rename xilink.exe to xilink2.exe. Open the VCBIN folder as above and copy link.exe into the compiler's binia32 folder and rename the copy to xilink.exe. Test your build again.

Steve:

That makes the Linking very fast!! Do you know what the problem was, then, and why? Is there a more standard fix than renaming the files like that?

Evan

[evankw21]

Quoting - David White

Evan,

Just a long shot here -- I had compile time issues when I am connected to our network via VPN because Visual Studio by default puts files in My Documents, which in our scenario are on a network drive. Do you have any settings pointing to a network drive. Similarly, when searching for the linker, if it is searching through PATH and there are any network drives before the location of the linker, then you could be waiting for these slow connections. Try putting the linker and compiler folders towards the start of PATH and see if that makes any difference.

Regards,

David

David/Jim:

I don't think this is the issue because everything I have is local. Thanks anyway.

Evan