Intel OneAPI MKL Scalapack call gives segmentation fault

[OUKevinM]

Hi, I am attempting to compile/run a test code using the oneAPI MKL Scalapack build. I am able to run a similar version of the code using native Scalapack (build from netlib using non mkl libraries) and MPICH. For the oneAPI variant, I changed the header files and some data types from int to MKL_INT in order to get it to compile. The BLACS interface works as it does properly assign the processor grid and allocate the local arrays. However, the code returns a segmentation fault during runtime when calling the pdgbtrf subroutine.

I have attached four files. The ScalapackExample_PDGBTRF.cpp  and OutputFromScalapackExample.txt is the code and correct output from the code when running native (non-mkl) ScaLapack. The other two files: ScaLapackSimple.cpp  and ErrorLog.txt is the code and its output for the oneAPI version.

For the oneAPI version, I used only oneAPI tools (libraries and compilers) and compiled using:

mpiicc ScaLapackSimple.cpp -o ScaLapackSimple -DMKL_ILP64 -qmkl=sequential -L${MKLROOT}/lib/intel64 -lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64 -lpthread -lm -ldl

The example code is hard coded to run on 3 processors so I run it using:

mpirun -np 3 ScaLapackSimple

Is there more changes I need to make to use the oneAPI Scalapack build compared to native Scalapack?

Additional information: I am running on Ubuntu 20.04.4 LTS to test this code.

Thank you in advanced!

[VarshaS_Intel · Accepted solution]

Hi,

Thanks for posting in Intel Communities.

Could you please let us know the version of MKL you are using and do you observe the same error when using Intel MKL with MPICH?

Could you please consider using the lp64 instead of ilp64 and try using the below command?

mpiicc ScaLapackSimple.cpp -o ScaLapackSimple -qmkl=sequential -L${MKLROOT}/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 -lpthread -lm -ldl -g

Thanks & Regards,

Varsha

[OUKevinM]

Hi, you are my hero! Using lp64 instead of ilp64 worked like a charm! If you don't mind one more question... what is the difference between the lp64 and ilp64? Do they both handle 64-bit integers?

For the virtual machine of interest, I have only installed oneAPI to test without any conflicts with other software. I use a separate virutal machine for the native ScaLapack build. Also, I am using version 2022.0.2 for oneAPI.

[VarshaS_Intel]

Hi,

>>what is the difference between the lp64 and ilp64?

The difference between them is integer type length.

The term ILP64 denotes that integer, long, and pointer data entities all occupy 8 bytes whereas in the conventional LP64 model integer entities occupy 4 bytes.

>>Do they both handle 64-bit integers?

No, the ILP64 interface uses the 64-bit integer type, while LP64 uses the 32-bit integer.

So if you have an application where you are using a large array index that has more than 2^31- 1 elements you could choose ILP64 libraries.

For more details, you can refer to the below link.

https://www.intel.com/content/www/us/en/develop/documentation/onemkl-linux-developer-guide/top/linking-your-application-with-onemkl/linking-in-detail/linking-with-interface-libraries/using-the-ilp64-interface-vs-lp64-interface.html

Additionally, you can always take the help of the link line advisor which helps you to find the right set of libraries.

https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-link-line-advisor.html#gs.xto2dm

Hope this answers your question.

>>Using lp64 instead of ilp64 worked like a charm

Glad to know that it works and thanks for accepting our solution.

If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks & Regards,

Varsha.