Using MKL 2023.0.0.20221201 with Eigen

[ChristophJunghans]

Using Intel's `icpx` with recent MKL and Eigen 3.4.0 leads to a compiler error:

/opt/intel/oneapi/compiler/latest/linux/bin/icpx -isystem /usr/include/eigen3 -isystem /opt/intel/oneapi/mkl/2023.0.0/include -DMKL_ILP64 -MD -MT CMakeFiles/dummy.dir/main.cpp.o -MF CMakeFiles/dummy.dir/main.cpp.o.d -o CMakeFiles/dummy.dir/main.cpp.o -c /home/votca/mkl_test/main.cpp
In file included from /home/votca/mkl_test/main.cpp:2:
In file included from /usr/include/eigen3/Eigen/Eigen:1:
In file included from /usr/include/eigen3/Eigen/Dense:4:
In file included from /usr/include/eigen3/Eigen/QR:45:
/usr/include/eigen3/Eigen/src/QR/ColPivHouseholderQR_LAPACKE.h:85:1: error: cannot initialize a variable of type 'long long *' with an rvalue of type 'Scalar *' (aka 'int *')
EIGEN_LAPACKE_QR_COLPIV(double, double, d, ColMajor, LAPACK_COL_MAJOR)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is with:

• Operating System: Linux / Fedora 37
• Architecture : x86_64
• Eigen Version : 3.4.0
• Compiler Version: Intel(R) oneAPI DPC++/C++ Compiler 2023.0.0 (2023.0.0.20221201)

One can reproduce this in the following small example, having "main.cpp" as

#define EIGEN_USE_MKL_ALL
#include <Eigen/Eigen>

int main() {
return 0;
}

with a "CMakeLists.txt" like:

cmake_minimum_required(VERSION 3.14)
project(foo CXX)
find_package(Eigen3)
find_package(MKL)
add_executable(dummy main.cpp)
target_link_libraries(dummy Eigen3::Eigen MKL::MKL)

Steps to reproduce:

1. `docker run -it ghcr.io/votca/buildenv/fedora:intel`
2. create `CMakeLists.txt` and `main.cpp` from the code above
3. `CXX=icpx cmake -B builddir && cmake --build builddir --verbose`

Instead of the docker container one can also use stock Fedora 37 install `mkl` and `icpx`:

printf "[oneAPI]\nname=Intel oneAPI\nbaseurl=https://yum.repos.intel.com/oneapi\nenabled=1\ngpgcheck=1\nrepo_gpgcheck=1\ngpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB" > /etc/yum.repos.d/intel-oneapi.repo
dnf -y update
dnf -y install intel-oneapi-compiler-dpcpp-cpp intel-oneapi-compiler-dpcpp-cpp-and-cpp-classic intel-oneapi-mkl-devel

I think the problem is -DMKL_ILP64 i.e. the ILP64 interface. Using the LP64 interface manually works. GCC uses that interface by default, however for icpx ILP64 is the only option that CMake allows.

So I think this might be a bug in MKL's CMake config.

Also see: https://gitlab.com/libeigen/eigen/-/issues/2586

[ShanmukhS_Intel]

Hi Christophs,

Thanks for posting on Intel Communities.

Could you please let us know if the below command is run on linux environment with icpx compiler and mkl being setup throw the error mentioned("cannot initialize a variable of type 'long long *'") or do we need to have any other dependencies?

'CMakeLists.txt` and `main.cpp` from the code mentioned earlier and running the command `CXX=icpx cmake -B builddir && cmake --build builddir --verbose`

Best Regards,

Shanmukh.SS

[ChristophJunghans]

Yes, this is under Linux (Fedora 37 specifically), the only other dependency is Eigen (https://gitlab.com/libeigen/eigen).

I think my specific question is: Why can I not use MKL_INTERFACE=lp64 with icpx?

[ShanmukhS_Intel]

Hi Christoph,

Why can I not use MKL_INTERFACE=lp64 with icpx?

>> mkl_set_interface_layer API or environment variable MKL_INTERFACE_LAYER can be used to set the interface layer for Intel® oneAPI Math Kernel Library at run time.

Please refer to the below link for more details which contains possible values depending on the system architecture and setting the interface.

https://www.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/support-functions/single-dynamic-library-control/mkl-set-interface-layer.html

# MKL_INTERFACE (for MKL_ARCH=intel64 only)

# Values: lp64, ilp64

# GNU or INTEL interface will be selected based on Compiler and the default is ilp64

Best Regards,

Shanmukh.SS

[ChristophJunghans]

Yeah, I tried that before, but for icpx mkl's CMake config restricts the interface to ilp64, so switching to lp64 is not an option.

To reproduce

1.) create a CMakeLists.txt:

cmake_minimum_required(VERSION 3.14)
project(foo CXX)
find_package(MKL)

2.)  run:

CXX=icpx cmake -B builddir -DMKL_INTERFACE=lp64

3.) get an error:

-- The CXX compiler identification is IntelLLVM 2023.0.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/intel/oneapi/compiler/latest/linux/bin/icpx - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- MKL_ARCH: None, set to ` intel64` by default
-- MKL_ROOT /opt/intel/oneapi/mkl/2023.0.0
-- MKL_LINK: None, set to ` dynamic` by default
CMake Error at /usr/local/lib/cmake/mkl-2023.0.0/MKLConfig.cmake:103 (message):
  Invalid MKL_INTERFACE_FULL `intel_lp64`, options are: intel_ilp64
Call Stack (most recent call first):
  /usr/local/lib/cmake/mkl-2023.0.0/MKLConfig.cmake:168 (mkl_message)
  /usr/local/lib/cmake/mkl-2023.0.0/MKLConfig.cmake:296 (define_param)
  CMakeLists.txt:3 (find_package)


-- Configuring incomplete, errors occurred!

[ShanmukhS_Intel]

Hi Christoph,

Thanks for sharing the details. We are working on the same. We will get back to you soon with an update.

Best regards,

Shanmukh.SS

[RainerB]

Hi,

I have "solved" the problem by adding a "${IFACE_TYPE}_lp64" to line 288 of the file "MKLConfig.cmake" (which I found in the "$HOME/lib/cmake" folder for some unknown reason. Original file is in "$MKLROOT/lib/cmake/mkl/"):

set(MKL_INTERFACE_LIST intel_ilp64 ${IFACE_TYPE}_lp64)

Kind regards,

Rainer

[ShanmukhS_Intel]

Hi Christoph,

Glad to know that your issue is resolved with your workaround. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Best Regards,

Shanmukh.SS

[ChristophJunghans]

@ShanmukhS_Intel this is not really fixed unless you include @RainerB 's patch in the mkl distribution. Could you do that?

[ShanmukhS_Intel]

Hi Christoph,

Thank you for your feedback. We have provided your feedback to the relevant team. At this moment there is no visibility when it will be implemented and available for use. Please let me know if we can go ahead and close this case?

Best Regards,

Shanmukh.SS

[nwalet]

Doesn't look like this has ever been resolved--still fails for 2023.1.0.

Especially on a system that I not maintain the requires a lot of effort to by-pass this setting when using cmake.

[ChristophJunghans]

I think 2023.2.0 was the target.

[ChristophJunghans]

Still not fixed in 2023.2.0!

[ChristophJunghans]

I was mistaken, for icpx it says:

-- Invalid MKL_DPCPP_INTERFACE_FULL `intel_lp64`, options are: intel_ilp64

But it still seems to work.