MKL in Python extension module cmake build

[inakleinbottle]

I'm maintaining a Python package that makes use of MKL features and I'm currently updating our CI/CD pipeline. I'm using CMake to build the component that makes use of MKL, and currently we're using a large amount of custom, platform-dependent cmake code to find the MKL libraries using environment variables set in the CI pipeline, which is usually installed using either the package manager (ManyLinux) or some other means. From a maintainability point of view, and from a distribution point of view, it would be much better to use the Python distribution of MKL and friends installed via pip directly to the build environment.

Unfortunately, the findBLAS/findLAPACK modules distributed with CMake cannot find the MKL development environment installed via pip. There is no setvars.sh or equivalent included with the MKL-devel package to set up the environment, and the directory structure does not quite match the typical oneAPI MKL tree from the package manager installation. It would be really great if the python distribution of MKL-devel included a CMake config file so that one can replace lots of platform dependent code with a simple find_package(MKL CONFIG). This would dramatically simplify the build system.

I'd appreciate guidance or solutions that can be offered.

[VidyalathaB_Intel]

Hi Sam,

Thanks for reaching out to us.

>>From a maintainability point of view, and from a distribution point of view, it would be much better to use the Python distribution of MKL and friends installed via pip directly to the build environment....It would be really great if the python distribution of MKL-devel included a CMake config file 

We understand your concern and we will look into this issue.

Meanwhile, you may consider installing the oneAPI Basetoolkit from which you can get the Intel oneMKL library (I guess you are aware of this already as you have mentioned about setvars.sh script file). I think it might help you with what you are looking for.

>>one can replace lots of platform-dependent code with a simple find_package(MKL CONFIG)...I'd appreciate guidance or solutions that can be offered

We would like to remind you that Intel® oneAPI Math Kernel Library (oneMKL) ships with a CMake config file for easy integration of oneMKL into user-applications that already use CMake.This is a Config-mode file that you can use in find_package() macros. You can make use of this feature if you install the Intel oneMKL library via downloading oneAPI Basetoolkit. (You can also check the sample CMakeLists.txt file which is provided with the installed MKL library.)

If you are interested, here is the link for downloading oneAPI Base Toolkit:

https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html

For more details regarding cmake config file refer to the below link: https://www.intel.com/content/www/us/en/develop/documentation/onemkl-linux-developer-guide/top/getting-started/setting-environment-variables/using-the-cmake-config-file.html

Regards,

Vidya.

[inakleinbottle]

Hi Vidya, thanks for responding. Unfortunately the problem I'm having is installing the base toolkit inside a CI/CD workflow on Github actions is proving to be very difficult because of the size of the installation. Installing via package manager is preferable because it is platform independent and means I don't have to update the download URLs every time oneAPI updates. However, the package manager installed files (pip, nuget, conda) on Windows (and probably Mac) do not follow the same directory structure as the oneAPI base installation and don't include the same scripts for setting up the environment or the CMake config required.

[inakleinbottle]

So I've managed to work the download into the workflow, but I'm now having a problem specifying the mkl.devel component to install (on Windows currently). When I use the downloaded toolkit with a command such as

.\installer.exe -s -a -s -eula accept --components intel.oneapi.win.mkl.devel --action install # + other options

It does not install only the MKL component, but all of the other stuff from the toolkit (inc. VTune etc, which is obviously not useful in a ephemeral virtual machine). I managed to install intel.oneapi.win.cpp-compiler component, but none of the others seem to work. The documentation for the installer doesn't seem to be particularly helpful.

One little wrinkle on this is that the modify action with the mkl component does seem to install only the mkl component on an already installed base cpp-compiler component. I'm so very confused by this.

[VidyalathaB_Intel]

Hi,

>>It does not install only the MKL component, but all of the other stuff from the toolkit

We regret the inconvenience caused.

Yes, indeed Intel oneAPI Base Toolkit does have a lot of components apart from MKL.

Here is the link to get the standalone components of oneAPI

https://www.intel.com/content/www/us/en/developer/articles/tool/oneapi-standalone-components.html

which helps you to get only MKL component.

You can find it under >> performance libraries > Intel oneAPI MKL

Hope the provided information helps.

Regards,

Vidya.

[VidyalathaB_Intel]

Hi Sam,

Reminder:

As we haven't heard back from you, could you please provide us an update regarding your issue? Please do let us know regarding the information that has been provided in the previous post.

Regards,

Vidya.

[VidyalathaB_Intel]

Hi Sam,

As we haven't heard back from you, we assume that your issue is resolved and closing this thread. Please post a new question if you need any additional assistance from Intel as this thread will no longer be monitored.

Regards,

Vidya.