LAPACKE_dgetrf fails because it uses 8-byte integers

[sudoLife]

Hi,

Here's a test program:

#include <iostream>
#include <vector>
#include <mkl.h>

int main(void)
{
    constexpr static int testMatrixRows = 4;
    
    std::vector<MKL_INT> ipiv(testMatrixRows, 0); // gives segfault because it tries to write 8-sized ints

    std::cout << sizeof(MKL_INT) << "\n"; // outputs 4, so 32-bit integers


    std::vector<double> testMatrix({
                                    2.4620, 0.3520, 0.0000, 0.0000,
                                    0.3520, 2.4000, 0.0000, 0.0000,
                                    0.0000, 0.0000, 25.3098, 0.0000,
                                    0.0000, 0.0000, 0.0000, 50.3098
                                    });

    // after this operation, ipiv should be [1, 2, 3, 4].

    LAPACKE_dgetrf(LAPACK_ROW_MAJOR, testMatrixRows, testMatrixRows, testMatrix.data(), testMatrixRows, ipiv.data());

    for (auto tr : ipiv)
    {
        std::cout << tr << "\n";
    }

    // Segmentation fault & similar errors on trying to deallocate vector memory, since it's corrupted.

    return 0;
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.15)

project(test_lapack LANGUAGES CXX)
add_executable(test_lapack testing_lapack.cpp)

find_package(MKL CONFIG REQUIRED)
target_link_libraries(test_lapack PRIVATE MKL::MKL ipps)

I compile the program using -DCMAKE_CXX_COMPILER=dpcpp.

The issue is that LAPACKE_dgetrf calls a function that writes to the ipiv array with the integer of size 8, therefore going out of bounds and creating a fatal error when the vector is used / deallocated.

Evidence of why that is:

1. We can double the size of the ipiv array and output the contents after calling LAPACKE_dgetrf:

1
0
2
0
3
0
4
0

This clearly shows that the contents were written with integer of size_t, little endian.

2. Output of valgrind shows the culprit:

==19345== Invalid write of size 8
==19345== at 0x2E1186E5: mkl_lapack_dgetf2 (in /opt/intel/oneapi/mkl/2022.1.0/lib/intel64/libmkl_core.so.2)

Also inspecting CMakeConfig shows that the binary is compiled using intel_ilp64 (64-bit integer). However, MKL_INT and lapack_int both show the size of 4. So I suppose that's the problem, that someone somewhere doesn't change this size to match the compilation.

UPDATE:

After some digging around, I found a few workarounds:

1. adding MKL_ILP64 compile definition (this should be done by  the MKLConfig.cmake, as it is perfectly aware of what needs to be done)

2. setting the type of ipiv to int64 and type-casting it when passing to the LAPACK functions.

Regarding the first option, it seems that there is some work in the config, as the MKL_C_COPT contains the required pre-processor directive. However, as we can see from this snippet:

target_compile_options(MKL::MKL INTERFACE
    $<$<STREQUAL:$<TARGET_PROPERTY:LINKER_LANGUAGE>,C>:${MKL_C_COPT}>
    $<$<STREQUAL:$<TARGET_PROPERTY:LINKER_LANGUAGE>,Fortran>:${MKL_F_COPT}>
    $<IF:$<BOOL:${ENABLE_OMP_OFFLOAD}>,${MKL_OFFLOAD_COPT},>)

This never reaches my code even if I set language of the project to be both C and CXX, as the linker language is determined elsewhere .

Conclusion: if your code is in C++ but you use C-style IntelMKL API, your LAPACK by default is messed up and requires a manual compilation flag which is not documented.

If you compile with g++, you can set lp64 instead of ilp64, but then the LAPACK_dgetrf will still result in SEGFAULT. It seems it just has that integer size 8 hard-coded somewhere.

[ShanmukhS_Intel]

Hi,

Thanks for posting on Intel Communities.

Thank you for sharing all the findings wrt your code.

Could you please let us know if you are still facing the issue when trying to run the code in a standalone manner (compiling individual files and executing them) without using any Make file or using MKL link line advisor.

Best Regards,

Shanmukh.SS

[ShanmukhS_Intel]

Hi,

We are working on your issue. We will get back to you soon with an update.

Best Regards,

Shanmukh.SS

[sudoLife]

Hi,

Thank you for your efforts.

I have tried linking using this linker line (32-bit integer):

 -L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_tbb_thread -lmkl_core -lpthread -lm -ldl

This works fine.

However, then I tried using 64-bit integers:

 -L${MKLROOT}/lib/intel64 -lmkl_intel_ilp64 -lmkl_tbb_thread -lmkl_core -lpthread -lm -ldl

Now it's again the invalid write of size 8.  sizeof(MKL_INT) and sizeof(lapack_int) are both 4.

Adding "-D MKL_ILP64" to the compilation command solves the problem. But shouldn't this be set automatically when linking against mkl_intel_ilp64? I suppose that's the issue here, that this definition is not set automatically.

[Ruqiu_C_Intel]

Hi,

Thank you for reaching to us.

For working MKL 64-bit integers,  -DMKL_ILP64 is necessary in your application. Please check oneMKL link line advisor here: https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-link-line-advisor.html#gs.5zh3nv

[sudoLife]

Hi @Ruqiu_C_Intel ,

Linking advisor should not be my problem when I am using CMakeConfig you are shipping with the library. It should set the option automatically, but doesn't.

I didn't even need 64-bit integers when I linked your library. The CMakeConfig automatically linked me with mkl_intel_ilp64, but conveniently forgot to add the -DMKL_ILP64 option. This needs fixing.

[Ruqiu_C_Intel]

Hello,

Yes, you right. Might need update the README in example to let it more clear.

The default value of MKL_INTERFACE  is ilp64 in README from MKL example fold:

+--------------------------+-------------------------+---------------------------------+--------------+--------------+

| Option          | Description       | Supported            | Default   | MKLConfig  |

|             |             | values             | value    | option    |

+==========================+=========================+=================================+==============+==============+

+--------------------------+-------------------------+---------------------------------+--------------+--------------+

| MKL_INTERFACE      | Define Integer size for | ilp64 lp64           | ilp64    | Yes     |

|             | C/Fortran API      |                 |       |       |

README mentions that All build options should be defined as `-D<option>=<value>` at the CMake configuration. So  -DMKL_ILP64 option should be added by user himself according to oneMKL link line advisor here: https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-link-line-advisor.html#gs.5zh3...

Regards,

Ruqiu

[Ruqiu_C_Intel · Accepted solution]

Hello,

Thank you for your attention.

The issue about adding "-DMKL_ILP64" option automatically when linking against mkl_intel_ilp64 with dpcpp compiler has been fixed, it's available in our last version(oneAPI 2023.1) launched last quarter.

[sudoLife]

Hi,

Great, thanks for the fix! As far as I understand, I don't have to explicitly provide that option now, right?

I am going to accept this as a solution but I cannot verify it works at the moment, because I am away from the relevant hardware.