Build issue with 2021.3 on macOS: for_mac_ftn_alloc.dylib

[Matt_Thompson]

All,

I've encountered a weird bug/issue with Intel 2021.3 on macOS that I don't get with Intel 2021.2.

I recently installed 2021.3 as I saw it was out, but when I try to build a library I work on on my mac with Intel 2021.3, it dies with this error:

ld: file not found: @rpath/for_mac_ftn_alloc.dylib for architecture x86_64

when CMake (via GNU Make generator) tries to link the dylib:

[100%] Building Fortran object generic/CMakeFiles/MAPL.generic.dir/MaplGenericComponent.F90.o
[100%] Building Fortran object generic/CMakeFiles/MAPL.generic.dir/MaplGeneric.F90.o
[100%] Linking Fortran shared library ../lib/libMAPL.generic.dylib
ld: file not found: @rpath/for_mac_ftn_alloc.dylib for architecture x86_64
make[3]: *** [generic/CMakeFiles/MAPL.generic.dir/build.make:392: lib/libMAPL.generic.dylib] Error 1
make[2]: *** [CMakeFiles/Makefile2:1949: generic/CMakeFiles/MAPL.generic.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1956: generic/CMakeFiles/MAPL.generic.dir/rule] Error 2
make: *** [Makefile:875: MAPL.generic] Error 2

But, if load Intel 2021.2, it works:

[100%] Building Fortran object generic/CMakeFiles/MAPL.generic.dir/MaplGenericComponent.F90.o
[100%] Building Fortran object generic/CMakeFiles/MAPL.generic.dir/MaplGeneric.F90.o
[100%] Linking Fortran shared library ../lib/libMAPL.generic.dylib
[100%] Built target MAPL.generic

And up to that point, the build is nigh identical. If I turn on VERBOSE=1, all the same options being used, CMake throws no different warnings.!

I use modulefiles on my system and the only difference between my 2021.2 and 2021.3 modulefile is the location of the install, and lots of other libraries built just fine (things like HDF5, netCDF, etc.).

Has anyone ever seen this error on their machine? I tried searching Google for "for_mac_ftn_alloc.dylib", but for the first time in a long time for me, the search returns no results .

I will note that this library (MAPL) that I'm building has a *lot* of OO Fortran in it, so perhaps this is some regression/change in ifort and OO?

[Matt_Thompson]

Huh. Is this dylib not part of Intel maybe:

❯ find /opt/intel -iname '*for_mac_ftn_alloc*'
❯

It apparently isn't there. I even tried searching my whole machine and I can't find it (though I don't have permission to look in some folders as I don't have sudo powers).

[Ron_Green]

That is definitely NOT an Intel Compiler library.  We don't use that many underscores or that naming convention.  Is it possibly a gfortran lib?  Is CMake confused on the FC or F90 compiler?  Or something that your application builds?  

[Matt_Thompson]

Definitely not something we build! And for this code, we are using `clang` as the C compiler (of which we have like 3 file) and `ifort` as the Fortran compiler (`icc` on my mac has always had issues, so we just stick with `clang`.) I also never install gcc/gfortran anywhere near the "default" path as I want to control my compilers myself.

And CMake seems pretty happy:

-- The Fortran compiler identification is Intel 2021.3.0.20210609
-- The CXX compiler identification is AppleClang 12.0.5.12050022
-- The C compiler identification is AppleClang 12.0.5.12050022
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /opt/intel/oneapi/compiler/2021.3.0/mac/bin/intel64/ifort - skipped
-- Checking whether /opt/intel/oneapi/compiler/2021.3.0/mac/bin/intel64/ifort supports Fortran 90
-- Checking whether /opt/intel/oneapi/compiler/2021.3.0/mac/bin/intel64/ifort supports Fortran 90 - yes
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done

I searched around our Base Libraries and didn't find any module/library/code like that. I also cloned the LLVM code base and about the closes I see is a C++ header file has a macro called FTN_ALLOC, but the resulting code would have omp_alloc_ so...not that. 

I also looked at my 2021.2 and 2021.3 installs on my laptop and I am missing some files from the 2021.3 install (I don't install everything as I don't do crypto work, etc.), but they seem pretty innocuous:

❯ diff -rq 2021.2.0/ 2021.3.0/ | grep -i only
Only in 2021.2.0/mac/compiler/include/icc/cilk: ReadMe.html
Only in 2021.2.0/mac/compiler/include/icc/cilk: legal_footer.html
Only in 2021.2.0/mac: documentation
Only in 2021.2.0/mac: man

I'm not sure how I didn't get the docs and manpages, but I'm thinking there better not be any libraries in there!

This is a real mystery.

Maybe I'll see if I can uninstall/reinstall Intel 2021.3? Maybe some weird step didn't happen?

[mecej4]

You may try generating and examining a linker map file to see what pulled in the miscreant (library, object, etc.).

[Matt_Thompson]

Hmm. Okay...query: do you know how? I've grabbed the command CMake is trying to do:

/opt/intel/oneapi/compiler/2021.3.0/mac/bin/intel64/ifort -g -O0 -ftz -align all -fno-alias -debug -nolib-inline -fno-inline-functions -assume protect_parens,minus0 -prec-div -prec-sqrt
...
-ldl -lcurl -lz -lm -framework SystemConfiguration -framework CoreFoundation -framework Security

and tried adding "-Wl,-map", "-Wl,-map,output.map", "-Xlinker -M" and lots of other things and nothing seems to make the map file. It's like it dies before `ifort` can do that?

[mecej4]

Perhaps, on your system the linker ld uses -m to signify "emulate linker ...". I can't say what Macs require, but on Linux the option is -Map or --print-map, so a GNU compiler invocation to build an a.out from a Fortran file and a C file could be similar to (this one worked on Cygwin-64 on Windows 10):

gfortran -g fmain.f csub.c -Wl,-Map,fmain.map

[Matt_Thompson]

Yeah, unfortunately the macOS ld is BSD ld. Now the manpage does say:

-map map_file_path
Writes a map file to the specified path which details all symbols and their addresses in the output image.

But I've tried "-Wl,-map,/Users/mathomp4/foo" and "-Xlinker -map -Xlinker /Users/mathomp4/foo" with no luck. But I do know something is happening because if I try "-Wl,-map" I get:

ld: warning: argument missing after -map

So...yeah. It's like the linker never gets to the point to run it? 

[mecej4]

Options, in particular the -M in this case, can be case-sensitive.

You probably need to specify the map file name, see this BSD ld man-page.

[Matt_Thompson]

Sadly, on macOS, it is -map. If you try to use -Map:

ld: unknown option: -Map

This is pretty close to the macOS ld page: https://www.unix.com/man-page/osx/1/ld/ in that it has the same -map entry. 

[Matt_Thompson]

I know @Ron_Green says for_mac_ftn_alloc.dylib is not an Intel library but I'm afraid it might be. I tried today to install Intel 2022.0 on my laptop. Like 2021.3 and 2021.4, it also throws the error:

ld: file not found: @rpath/for_mac_ftn_alloc.dylib for architecture x86_64

So I decided to do a deep dive and if I look inside of /opt/intel/oneapi/compiler/2022.0.0/mac/compiler/lib :

❯ otool -L libifcore.dylib | grep for_mac
	@rpath/for_mac_ftn_alloc.dylib (compatibility version 1.0.0, current version 1.11.0, weak)
❯ otool -L libifcoremt.dylib | grep for_mac
	@rpath/for_mac_ftn_alloc.dylib (compatibility version 1.0.0, current version 1.11.0, weak)

 If I go deeper with otool:

❯ otool -l libifcore.dylib | less
libifcore.dylib:
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __TEXT
...
Load command 8
      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.11
      sdk 10.13
Load command 9
      cmd LC_SOURCE_VERSION
  cmdsize 16
  version 0.0
Load command 10
          cmd LC_LOAD_WEAK_DYLIB
      cmdsize 56
         name @rpath/for_mac_ftn_alloc.dylib (offset 24)
   time stamp 2 Wed Dec 31 19:00:02 1969
      current version 1.11.0
compatibility version 1.0.0
Load command 11
          cmd LC_LOAD_DYLIB
      cmdsize 48
         name @rpath/libimf.dylib (offset 24)
   time stamp 2 Wed Dec 31 19:00:02 1969
...

 So, it looks like the reference to for_mac_ftn_alloc.dylib is baked into libifcore.dylib and libifcoremt.dylib.

Doing some grepping it looks like it came in with 2021.3:

❯ otool -l /opt/intel/oneapi/compiler/2021.2.0/mac/compiler/lib/libifcore.dylib | grep for_mac
❯ otool -l /opt/intel/oneapi/compiler/2021.3.0/mac/compiler/lib/libifcore.dylib | grep for_mac
         name @rpath/for_mac_ftn_alloc.dylib (offset 24)
❯ otool -l /opt/intel/oneapi/compiler/2021.4.0/mac/compiler/lib/libifcore.dylib | grep for_mac
         name @rpath/for_mac_ftn_alloc.dylib (offset 24)
❯ otool -l /opt/intel/oneapi/compiler/2022.0.0/mac/compiler/lib/libifcore.dylib | grep for_mac
         name @rpath/for_mac_ftn_alloc.dylib (offset 24)

 So do you know why Intel is looking for this library...and what might provide it? 

Does your installer somehow "create" the dylib at install time, or is it just copied. If it is just copied from the installer to the disk, then we are missing a dylib in mac/compiler/lib.

I might try to do some wacky experiments with install_name_tool to see if I can "shove" the error away, but this seems like something endemic to 2021.3+

[Matt_Thompson]

An update. Working with a colleague here we made a "blank" for_mac_ftn_alloc.dylib library from this exciting Fortran code:

subroutine cws8vn9ud915dz907vl_()
end

We then copied that file into /opt/intel/oneapi/compiler/2022.0.0/mac/compiler/lib and now things work. I guess this works because the library is weak loaded, so it's not "required" at runtime? 

So there is at least a workaround. I've made a little GitHub repo here with the code:

https://github.com/mathomp4/IntelMacMissingLibraryGenerator