Handling Large Data Counts with Intel MPI and ScaLAPACK in Fortran

BrianZHANG — Tue, 16 Jul 2024 06:11:47 GMT

I'm utilizing Intel MPI and ScaLAPACK from oneMKL in my Fortran project. I've encountered a limitation with MPI_Send and MPI_Recv where data counts can't exceed upper limitation of Integer(4). Given that my application handles large datasets, this restriction is frequently exceeded.

According to the MPI Library Developer Guide, there's a version of iLP, but it also mentions that data type counts and other arguments exceeding upper limitation of Integer(4) are not supported.

Despite linking with the iLP ScaLAPACK library, I still experience failures in mr2d_malloc within MKL, likely tied to the same Integer(4) limitation. Furthermore, when using pdgemr2d in ScaLAPACK for distributing matrices stored in the root process, any data count exceeding the limit leads to errors.

Could you provide guidance or workarounds for handling data sizes beyond the Integer(4) limit in these contexts?

Re:Handling Large Data Counts with Intel MPI and ScaLAPACK in Fortran

Gennady_F_Intel — Mon, 29 Jul 2024 05:34:29 GMT

MKL ScaLAPACK address 64 bit integers vis linking against ILP64 libraries ( here is the link of MKL Linker Adviser to see the list of MKL libs : https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-link-line-advisor.html).

With regard to mr2d_malloc: MKL ScaLAPACK doesn’t have mr2d_malloc interfaces explicitly and therefore you are talking about netlib’s ScaLAPACK.

Therefore the issue is beyond of MKL but related to MPI.

Regard to MPI – the current ( latest ) version of MPI ( 2021.13) has to support 64 bit ( ILP64 mode).

Could You give the reproducer which we could help us to investigate this case?

topic Re:Handling Large Data Counts with Intel MPI and ScaLAPACK in Fortran in Intel® oneAPI Math Kernel Library

Handling Large Data Counts with Intel MPI and ScaLAPACK in Fortran

Re:Handling Large Data Counts with Intel MPI and ScaLAPACK in Fortran