Intel® HPC Toolkit
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2153 Discussions

slow execution of ifort/icpc on MacOSX catalina

HKruse
Beginner
2,397 Views

Hi,

Compiling a single .f90 with ifort is slow on Catalina, and even for medium-sized projects this becomes tedious. Even CMake Fortran testing is slow because of this. To an extend where it becomes unusable.

Another post discussed possible license server issues, but this is OneAPI and there is no license server (so I assume).

However, from testing things out, the compilation itself is fast, it is the driver program that is slow. Running

 

 

ifort -v file.f90

 

 

 and then using the output commands to manually run fortcom (and eventual linker) shows that compiling is actually quick! The driver is the slow component. No idea what is does besides constructing a fortcom/ld command.

As far as colleagues reported, this isn't a Catalina specific problem and similar slow-ness was seen before. This isn't a Fortran specific problem, same behaviour is seen for C++ though I did investigate less.

Anyway, I don't necessarily expect a solution to this, OSX is typically low-priority, but at least wanted to document it with a post.

 

edit: this is HPCToolkit beta08; ifort (IFORT) 2021.1 Beta 20200602

Labels (1)
0 Kudos
6 Replies
PrasanthD_intel
Moderator
2,384 Views

Hi Holger,


Thanks for reporting this issue to us.

For better analysis and exploration we are forwarding this to Subject Matter Experts.


Regards

Prasanth


0 Kudos
Barbara_P_Intel
Moderator
2,378 Views

Thanks for reporting this. You are right. There is no license with oneAPI beta08.


I just compiled some of the Fortran sample codes using oneAPI beta08 on macOS. I don't experience any slowness.


I'll research this a bit to see if other users have reported this.



0 Kudos
Ron_Green
Moderator
2,354 Views

This is an interesting one. I see the same thing but I have some observations.

1) macBook Pro early 2015, 8GB ram


as Barbara mentioned, in oneAPI the beta ifort 2021.1 driver does no license check. I talked to the developer of the driver - on macOS we do query Xcode on each ifort invocation: doing this query of xcode properties to find paths to the command line tools, libraries, headers etc. This is unique to macOS compared to other OSes.


without doing anything special I timed compiling 100 sources sequentially.

Was finding compile times of 2.7sec per file - brutal, ugly.

On a lark I rebooted. I killed the Dropbox client but I do have a VPN client running to my company. Nice quiet system, no browsers, no MS Outlook or other "clutter". Compile times dropped to 0.9 seconds, so 3x faster. Question - can you try a reboot, only bring up terminal windows, kill off Dropbox or other background apps, and try again? time just one ifort compilation, forget the make especially if you are using -j to parallelize the build. Just time 1 file compilation. Run same timing 3 or more times to look for an average


I just do 'time ifort -c -O0 sub1.f90'


My old macBook is a dual core. And 8GB is lame. I'll go try some servers with quad core and none of the usual User apps and rubbish running.


It may be that the lookups we do for Xcode paths and location of tools is eating up more time in Catalina vs older macOS versions.


I tried dtruss and instruments but my system is not cooperating. I'll keep trying to get a system trace on the ifort invocation and it's child processes and threads.


ron


0 Kudos
HKruse
Beginner
2,281 Views

Thanks for trying it out!

Yes, turning off all syncing tools and especially(!) browsers reduces the `time ifort`  from about 3s to 1s on average for my test. Usable for tiny codes, but not even then unpleasant.

I have the same Macbook version, btw. 

 

It may be that the lookups we do for Xcode paths and location of tools is eating up more time in Catalina vs older macOS versions.

Looks that ways. Is there any chance of a work around? Specifying the Xcode paths somewhere to avoid searching for it..? 

0 Kudos
Ron_Green
Moderator
2,060 Views

the slowness is due to ifort driver calling 'xcodebuild' twice, once to determine the SDK version, the second to find the SDK path.  the exact calls are:

xcodebuild -sdk macosx -version | grep SDKVersion

xcodebuild -sdk macosx -version Path

 

time those on Catalina and Big Sur.  Not speedy are they?  We have a bug report into Apple on this since last year.   Interestingly if you have macOS from like 3 to 4 versions ago these commands fly in a fraction of a second.  OK nothing we can do about that that we haven't already done (bug report).  
But I have a hack workaround

First step, define 2 env vars for the results of those 2 calls:

INTEL_OSXSDK_VER=`xcodebuild -sdk macosx -version | grep SDKVersion`
INTEL_OSXSDK_PATH=`xcodebuild -sdk macosx -version Path`

 

Next, we write our own 'override' version of 'xcodebuild' as a script file with execute permissions

#!/bin/bash
case "$4" in
    "")
      echo $INTEL_OSXSDK_VER;;
     *)
      echo $INTEL_OSXSDK_PATH;;
esac

Now save the script above as ~/bin/xcodebuild and give it execute permission.

Finally, make your ~/bin path come BEFORE other system paths

 

export PATH=/Users/myUserName/bin:${PATH}

 and there you have it.  'ifort' driver will now fly like the wind.  For me, I put the 3 'export' statements in my ~/.bashrc and made sure my ~/.bash_profile sources ~/.bashrc

 

This is a workaround until xcodebuild is fixed to run fast again (if ever).  We're looking at perhaps having the ifort driver look for those 2 INTEL_OSSDK_* variables and if they exist, use those instead of calling xcodebuild. 

JSol
Beginner
1,816 Views

Thanks for this. It's an order of magnitude faster for me in some situations.

 

Has any decision been reached on having ifort look for the INTEL_OSXSDK_* variables? It would be nice to not need to override xcodebuild with the script that you provided.

0 Kudos
Reply