topic Re:Slow parallel IO with Intel MPI in Intel® MPI Library

Slow parallel IO with Intel MPI

spg — Wed, 19 May 2021 16:16:58 GMT

I wrote a piece code to add take advantage of the parallel file system on HPC, using the MPI-IO.

The code performs well (parallel version of the I/O faster than the sequential version) for GNU compiler with MPT, PGI compiler with MPT.

With Intel compiler andintel compiler/MPI 2020, the parallel version of the IO is 50 times slower than the sequential counter part (gather and save by few processes).

Previously with intel compiler/MPI 2018, by setting some environmental variables (I_MPI_EXTRA_FILESYSTEM="1" and I_MPI_EXTRA_FILESYSTEM_FORCE="lustre" ), I was able to get Intel compiler and intel MPI to perform slightly better on parallel I/O. With intel 2020.2.254, It doeas not work.

Any idea of what to do to get the paralelel I/O to be faster than the serial I/O (collect and save by few processes)?

Re:Slow parallel IO with Intel MPI

ShivaniK_Intel — Thu, 20 May 2021 10:35:36 GMT

Hi,

Thanks for reaching out to us.

Could you please provide the reproducer code so that we can investigate more on your issue?

From Intel MPI Library 2019, Parallel file systems (GPFS, Lustre, Panfs) are supported natively, no need to explicitly set these environment variables(I_MPI_EXTRA_FILESYSTEM*, I_MPI_LUSTRE* variables).

For more details please refer to the below link.

https://software.intel.com/content/www/us/en/develop/articles/intel-mpi-library-release-notes-linux.html#inpage-nav-3-1

Thanks & Regards

Shivani

Re: Re:Slow parallel IO with Intel MPI

spg — Thu, 27 May 2021 16:28:05 GMT

Thanks Shivani for looking into this.

As I try to build a minimal code to reproduce the issue, I found one part of the problem. A data processing performed before sending data to disk that is automatically optimized gnu and pgi regardless of optimization on, is not optimized by INTEL unless optimization is turned on.

When I turned the optimization on for INTEL, it reduced the factor of 50 significantly to about 2. I will double check everything.

Thanks again.

Re:Slow parallel IO with Intel MPI

ShivaniK_Intel — Fri, 04 Jun 2021 11:44:08 GMT

Hi,

Could you please provide the reproducer code so that we can investigate more on your issue?

Thanks & Regards

Shivani

Re:Slow parallel IO with Intel MPI

ShivaniK_Intel — Fri, 11 Jun 2021 06:23:08 GMT

Hi,

As we didn't hear back from you, Is your issue resolved? If not, please provide the details that have been asked in my previous post.

Thanks & Regards,

Shivani

Re: Slow parallel IO with Intel MPI

spg — Fri, 11 Jun 2021 20:03:35 GMT

@ShivaniK_Intel , Thank you for your continuous interest in this issue. I got it down to similar time as other compiler.

Thank you again.

Re:Slow parallel IO with Intel MPI

ShivaniK_Intel — Tue, 15 Jun 2021 07:34:49 GMT

Hi,

We are glad that your issue has been resolved. We will no longer respond to this thread. If you require any additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.

Have a good day

Thanks & Regards

Shivani