missing files and wrong directory structure for mkl and tbb

[inakleinbottle]

I've been putting together a new build pipeline for a library I've been working on, and I was hoping to use the python distributions of mkl to greatly simplify the build system - and avoid installing the whole oneAPI base package in the CI system.

Unfortunately, I've run into a couple of problems: 

First, on Windows the OpenMP link stub file is missing in the mkl-devel wheel, meaning that one cannot build and link a program that uses intel threading using this distribution. The file libiomp5md.lib (link file, not the statically linked version) needs to be present to link the application, and CMake fails to find MKL when this is not present. Ideally, one would distribute this as part of a "intel-openmp-devel" package, that is a dependency of "mkl-devel"

Second, to try and circumvent the problem above on Windows, I tried to use TBB as a threading backend for MKL. Unfortunately, there is another problem with the "tbb-devel" Python package. The CMake config file distributed with this package references the wrong directory structure compared to the contents of the wheel. The wheel contains the ".lib" files directly in the "lib" folder and ".dll" files in the "bin" folder of the wheel, but the CMake config file tries to find these in "Library/redist/intel64/vc14". Either the CMake config needs to be amended, or the directory structure in the wheel needs to be amended to reflect what the CMake file is looking for.

Everything works just fine on Linux and MacOs (which have sane linking models and directory structures), so this is a uniquely Windows problem.

[VarshaS_Intel]

Hi,

Thanks for posting in Intel Communities.

Regarding the missing file "libiomp5md.lib", the development team is looking into the issue. We will update you once there is any update.

For the second scenario, "MKLConfig.cmake doesn't support oneMKL Conda package structure." please find the  Intel MKL Release Notes under Known Issues and Limitations.

Thanks & Regards,

Varsha

[inakleinbottle]

Hi Varsha,

Thanks for your reply.

The second scenario refers to the TBBConfig.cmake file shipped in the tbb-devel wheel. The problem is just to do with the paths provided in TBBConfig.cmake, this is separate from MKL - although MKL does try to use this TBB if the option is set.

Sam.

[VarshaS_Intel]

Hi,

Thanks for the reply.

We are working on your issue. We will get back to you soon.

Thanks & Regards,

Varsha

[VarshaS_Intel]

Hi,

>>the problem is just to do with the paths provided in TBBConfig.cmake

Could you please let us know what kind of error you are facing while using TBBConfig.cmake file?

Could you please elaborate on your issue so that we could investigate more?

Thanks & Regards,

Varsha

[VarshaS_Intel]

Hi,

We have not heard back from you. Could you please let us know what kind of error you are facing while using TBBConfig.cmake file?

Could you please elaborate on your issue so that we could investigate more?

Thanks & Regards,

Varsha

[inakleinbottle]

Sorry for the delay, I've been very busy recently. The error I got was as follows:

2023-06-13T10:42:09.0132802Z   -- Running vcpkg install - done
2023-06-13T10:42:26.3738296Z   -- The C compiler identification is MSVC 19.29.30148.0
2023-06-13T10:42:27.3608636Z   -- The CXX compiler identification is MSVC 19.29.30148.0
2023-06-13T10:42:27.4359057Z   -- Detecting C compiler ABI info
2023-06-13T10:42:30.6693658Z   -- Detecting C compiler ABI info - done
2023-06-13T10:42:30.6721536Z   -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
2023-06-13T10:42:30.6722361Z   -- Detecting C compile features
2023-06-13T10:42:30.6728360Z   -- Detecting C compile features - done
2023-06-13T10:42:30.6810849Z   -- Detecting CXX compiler ABI info
2023-06-13T10:42:32.2376279Z   -- Detecting CXX compiler ABI info - done
2023-06-13T10:42:32.2397888Z   -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
2023-06-13T10:42:32.2402564Z   -- Detecting CXX compile features
2023-06-13T10:42:32.2411845Z   -- Detecting CXX compile features - done
2023-06-13T10:42:32.4310334Z   -- SKBuild environment: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-6okzel7z\cp310-win_amd64\build\venv\Scripts
2023-06-13T10:42:33.5548091Z   -- Found Python: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-6okzel7z\cp310-win_amd64\build\venv\Scripts\python.exe (found suitable version "3.10.11", minimum required is "3.8") found components: Interpreter Development.Module NumPy
2023-06-13T10:42:33.6711409Z   -- Adding MKL dir from pip-installed mkl: C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-3m0tkghu\overlay\Library
2023-06-13T10:42:33.8446741Z   -- Found Boost: C:/Users/runneradmin/AppData/Local/Temp/tmp8e08br21/build/vcpkg_installed/x64-windows/include (found suitable version "1.82.0", minimum required is "1.81") found components: url
2023-06-13T10:42:33.8734510Z   -- MKL_ARCH: None, set to ` intel64` by default
2023-06-13T10:42:33.8735203Z   -- MKL_LINK: static
2023-06-13T10:42:33.8735690Z   -- MKL_INTERFACE_FULL: intel_lp64
2023-06-13T10:42:33.8906899Z   -- MKL_THREADING: tbb_thread
2023-06-13T10:42:33.8907499Z   -- MKL_MPI: None, set to ` intelmpi` by default
2023-06-13T10:42:33.8907987Z   -- Found MKL: intelmpi
2023-06-13T10:42:33.8908905Z   -- Found MKL: C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-3m0tkghu/overlay/Library
2023-06-13T10:42:33.9470372Z   -- Found MKL: C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-3m0tkghu/overlay/Library/lib/mkl_intel_lp64.lib
2023-06-13T10:42:33.9944283Z   -- Found MKL: C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-3m0tkghu/overlay/Library/lib/mkl_tbb_thread.lib
2023-06-13T10:42:34.0409152Z   -- Found MKL: C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-3m0tkghu/overlay/Library/lib/mkl_core.lib
2023-06-13T10:42:34.0519003Z   -- Missed required oneTBB component: tbb
2023-06-13T10:42:34.0519691Z   --   one or both of:
2023-06-13T10:42:34.0520637Z      C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-3m0tkghu/overlay/Library/redist/intel64/vc14/tbb12.dll
2023-06-13T10:42:34.0521349Z       C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-3m0tkghu/overlay/Library/redist/intel64/vc14/tbb12_debug.dll
2023-06-13T10:42:34.0521858Z      files must exist.
2023-06-13T10:42:34.0522266Z   CMake Error at vcpkg/scripts/buildsystems/vcpkg.cmake:853 (_find_package):
2023-06-13T10:42:34.0522693Z     Found package configuration file:
2023-06-13T10:42:34.0522956Z 
2023-06-13T10:42:34.0523304Z       C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-3m0tkghu/overlay/Library/lib/cmake/tbb/TBBConfig.cmake
2023-06-13T10:42:34.0523667Z 
2023-06-13T10:42:34.0523934Z     but it set TBB_FOUND to FALSE so package "TBB" is considered to be NOT
2023-06-13T10:42:34.0524284Z     FOUND.
2023-06-13T10:42:34.0524671Z   Call Stack (most recent call first):
2023-06-13T10:42:34.0525228Z     C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-3m0tkghu/overlay/Library/lib/cmake/mkl/MKLConfig.cmake:661 (find_package)
2023-06-13T10:42:34.0525814Z     vcpkg/scripts/buildsystems/vcpkg.cmake:853 (_find_package)
2023-06-13T10:42:34.0526212Z     CMakeLists.txt:121 (find_package)
2023-06-13T10:42:34.0526493Z 
2023-06-13T10:42:34.0526591Z 
2023-06-13T10:42:34.0526987Z   -- Configuring incomplete, errors occurred!

[VarshaS_Intel]

Hi,

Regrets for the delay in response.

Could you please share with us the complete project file as well as the steps being used by you it helps us to investigate more on your issue.

Thanks & Regards,

Varsha

[inakleinbottle]

Here is a link to the repository as at the point at which the error occurred - I've since swapped to using sequential mkl to avoid needing TBB, but this obviously isn't ideal. https://github.com/datasig-ac-uk/RoughPy/tree/5b36fd9745cc04821431030ef4e42478b9455b4c

In particular, here is the CMakeLists.txt https://github.com/datasig-ac-uk/RoughPy/blob/5b36fd9745cc04821431030ef4e42478b9455b4c/CMakeLists.txt

I should probably explain the setup a little here too. MKL (and TBB) are pip installed with the correct version of Python, which then invokes cmake. I then use this version of Python to locate the location of the TBBConfig.cmake using the tbb-devel metadata. (Line 100-109 of CMakeLists.txt.) Both the MKLConfig.cmake and TBBConfig.cmake files are found by the scripts, but the configuration still fails because of the missing files reported in the logs.

I think the logs I shared are from a very slightly earlier version of CMakeLists.txt, but I think the only thing that has changed is that I explicitly invoke find_package(TBB CONFIG REQUIRED) and explicitly locate the config file using the script, neither of which made a difference. However, in case you need them, here is the logs from this commit.

2023-06-13T09:48:04.3005565Z   -- Running vcpkg install - done
2023-06-13T09:48:32.1559269Z   -- The C compiler identification is MSVC 19.29.30148.0
2023-06-13T09:48:33.3282202Z   -- The CXX compiler identification is MSVC 19.29.30148.0
2023-06-13T09:48:33.4257304Z   -- Detecting C compiler ABI info
2023-06-13T09:48:37.1109814Z   -- Detecting C compiler ABI info - done
2023-06-13T09:48:37.1138601Z   -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
2023-06-13T09:48:37.1144653Z   -- Detecting C compile features
2023-06-13T09:48:37.1153409Z   -- Detecting C compile features - done
2023-06-13T09:48:37.1255233Z   -- Detecting CXX compiler ABI info
2023-06-13T09:48:39.0038608Z   -- Detecting CXX compiler ABI info - done
2023-06-13T09:48:39.0072482Z   -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
2023-06-13T09:48:39.0073431Z   -- Detecting CXX compile features
2023-06-13T09:48:39.0082490Z   -- Detecting CXX compile features - done
2023-06-13T09:48:39.2834401Z   -- SKBuild environment: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-a54cwdaz\cp310-win_amd64\build\venv\Scripts
2023-06-13T09:48:40.6291735Z   -- Found Python: C:\Users\runneradmin\AppData\Local\Temp\cibw-run-a54cwdaz\cp310-win_amd64\build\venv\Scripts\python.exe (found suitable version "3.10.11", minimum required is "3.8") found components: Interpreter Development.Module NumPy
2023-06-13T09:48:40.7685167Z   -- Adding MKL dir from pip-installed mkl: C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-ch0rrr8z\overlay\Library
2023-06-13T09:48:40.9194020Z   -- Missed required oneTBB component: tbb
2023-06-13T09:48:40.9194948Z   --   one or both of:
2023-06-13T09:48:40.9195690Z      C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-ch0rrr8z/overlay/Library/redist/intel64/vc14/tbb12.dll
2023-06-13T09:48:40.9196582Z       C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-ch0rrr8z/overlay/Library/redist/intel64/vc14/tbb12_debug.dll
2023-06-13T09:48:40.9197282Z      files must exist.
2023-06-13T09:48:40.9197895Z   CMake Error at vcpkg/scripts/buildsystems/vcpkg.cmake:853 (_find_package):
2023-06-13T09:48:40.9198512Z     Found package configuration file:
2023-06-13T09:48:40.9198867Z 
2023-06-13T09:48:40.9199356Z       C:/Users/runneradmin/AppData/Local/Temp/pip-build-env-ch0rrr8z/overlay/Library/lib/cmake/tbb/TBBConfig.cmake
2023-06-13T09:48:40.9200139Z 
2023-06-13T09:48:40.9200508Z     but it set TBB_FOUND to FALSE so package "TBB" is considered to be NOT
2023-06-13T09:48:40.9201052Z     FOUND.
2023-06-13T09:48:40.9201706Z   Call Stack (most recent call first):
2023-06-13T09:48:40.9202251Z     CMakeLists.txt:109 (find_package)
2023-06-13T09:48:40.9202532Z 
2023-06-13T09:48:40.9202665Z 
2023-06-13T09:48:40.9203038Z   -- Configuring incomplete, errors occurred!
2023-06-13T09:48:43.9180591Z 
2023-06-13T09:48:43.9181498Z   *** CMake configuration failed
2023-06-13T09:48:43.9580828Z   error: subprocess-exited-with-error
2023-06-13T09:48:43.9581521Z   
2023-06-13T09:48:43.9582575Z   × Building wheel for roughpy (pyproject.toml) did not run successfully.
2023-06-13T09:48:43.9583186Z   │ exit code: 1
2023-06-13T09:48:43.9583685Z   ╰─> See above for output.
2023-06-13T09:48:43.9584071Z   
2023-06-13T09:48:43.9584631Z   note: This error originates from a subprocess, and is likely not a problem with pip.
2023-06-13T09:48:43.9590007Z   full command: 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-a54cwdaz\cp310-win_amd64\build\venv\Scripts\python.exe' 'C:\Users\runneradmin\AppData\Local\Temp\cibw-run-a54cwdaz\cp310-win_amd64\build\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' build_wheel 'C:\Users\RUNNER~1\AppData\Local\Temp\tmp63ulcpnz'
2023-06-13T09:48:43.9593411Z   cwd: D:\a\RoughPy\RoughPy
2023-06-13T09:48:43.9597148Z   Building wheel for roughpy (pyproject.toml): finished with status 'error'
2023-06-13T09:48:43.9608574Z   ERROR: Failed building wheel for roughpy
2023-06-13T09:48:43.9623831Z Failed to build roughpy
2023-06-13T09:48:43.9628257Z ERROR: Failed to build one or more wheels
2023-06-13T09:48:44.6916912Z ##[endgroup]
2023-06-13T09:48:44.6961895Z ##[error]Command ['python', '-m', 'pip', 'wheel', 'D:\\a\\RoughPy\\RoughPy', '--wheel-dir=C:\\Users\\runneradmin\\AppData\\Local\\Temp\\cibw-run-a54cwdaz\\cp310-win_amd64\\built_wheel', '--no-deps', '-v'] failed with code 1. None
2023-06-13T09:48:44.6975956Z                                                             ✕ 612.87s
2023-06-13T09:48:44.6977465Z 
2023-06-13T09:48:44.9960858Z creating virtual environment...
2023-06-13T09:48:44.9961778Z creating virtual environment...
2023-06-13T09:48:44.9962576Z determining package name from 'D:\\a\\_actions\\pypa\\cibuildwheel\\v2.13.0'...
2023-06-13T09:48:44.9963351Z installing cibuildwheel from spec 'D:\\a\\_actions\\pypa\\cibuildwheel\\v2.13.0'...
2023-06-13T09:48:45.3518310Z ##[error]Process completed with exit code 1.

In the commit that follows I tried one last thing of setting "TBB_FIND_RELEASE_ONLY ON " to no avail. Then I abandoned MKL threading and swapped to sequential, which bypassed the problem.

[VarshaS_Intel]

Hi,

Thanks for your patience.

The issue raised by you has been fixed in the TBB 2021.10 which is the latest version that got released.

As you are installing only a single component using Intel Conda , you can download the latest version from the following link: https://www.intel.com/content/www/us/en/docs/oneapi/installation-guide-macos/2023-0/conda.html

Could you please try and let us know if you are still facing any issues?

Thanks & Regards,

Varsha 

[VarshaS_Intel]

Hi,

The issue raised by you has been fixed in the TBB 2021.10 which is the latest version that got released.

Could you please let us know if you have any other queries?

Thanks & Regards,

Varsha