Handling Large Data Counts with Intel MPI and ScaLAPACK in Fortran

[BrianZHANG]

I'm utilizing Intel MPI and ScaLAPACK from oneMKL in my Fortran project. I've encountered a limitation with MPI_Send and MPI_Recv where data counts can't exceed upper limitation of Integer(4). Given that my application handles large datasets, this restriction is frequently exceeded.

According to the MPI Library Developer Guide, there's a version of iLP, but it also mentions that data type counts and other arguments exceeding upper limitation of Integer(4) are not supported.

Despite linking with the iLP ScaLAPACK library, I still experience failures in mr2d_malloc within MKL, likely tied to the same Integer(4) limitation. Furthermore, when using pdgemr2d in ScaLAPACK for distributing matrices stored in the root process, any data count exceeding the limit leads to errors.

Could you provide guidance or workarounds for handling data sizes beyond the Integer(4) limit in these contexts?

[Gennady_F_Intel]

MKL ScaLAPACK address 64 bit integers vis linking against ILP64 libraries ( here is the link of MKL Linker Adviser to see the list of MKL libs : https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-link-line-advisor.html).

With regard to mr2d_malloc: MKL ScaLAPACK doesn’t have mr2d_malloc interfaces explicitly and therefore you are talking about netlib’s ScaLAPACK.

Therefore the issue is beyond of MKL but related to MPI.

Regard to MPI – the current ( latest ) version of MPI ( 2021.13) has to support 64 bit ( ILP64 mode).

Could You give the reproducer which we could help us to investigate this case?