topic Re:Specify a shared memory transport in Intel® MPI Library

Specify a shared memory transport

Jervie — Mon, 08 Nov 2021 08:09:25 GMT

Hello,

I would like to know which transport the I_MPI_SHM use when it is specified with auto by default when running a parallel application using oneAPI, and could I specify it with like cma used in UCX.

When I look into I_MPI_SHM using impi_info -v I_MPI_SHM, it said

---

I_MPI_SHM
MPI Datatype:
MPI_CHAR
Description:
Select a shared memory transport to be used.
Syntax
I_MPI_SHM=<transport>
Arguments
<transport> - Define a shared memory transport solution.
-----------------------------------------------------------------------
> disable | no | off | 0 - Do not use shared memory transport.
> auto - Select a shared memory transport solution automatically.
> bdw_sse - The shared memory transport solution tuned for Intel(R)
microarchitecture code name Broadwell. The SSE/SSE2/SSE3 instruction
set is used.
> bdw_avx2 - The shared memory transport solution tuned for Intel(R)
microarchitecture code name Broadwell. The AVX2 instruction set is used.
> skx_sse - The shared memory transport solution tuned for Intel(R)
Xeon(R) processors based on Intel(R) microarchitecture code name Skylake.
The SSE/SSE2/SSE3 instruction set is used.
> skx_avx2 - The shared memory transport solution tuned for Intel(R)
Xeon(R) processors based on Intel(R) microarchitecture code name Skylake.
The AVX2 instruction set is used.
> skx_avx512 - The shared memory transport solution tuned for Intel(R)
Xeon(R) processors based on Intel(R) microarchitecture code name Skylake.
The AVX512 instruction set is used.
> knl_ddr - The shared memory transport solution tuned for Intel(R)
microarchitecture code name Knights Landing.
> knl_mcdram - The shared memory transport solution tuned for Intel(R)
microarchitecture code name Knights Landing. Shared memory buffers
may be partially located in the MultiChannel DRAM (MCDRAM).

---

I wonder when I_MPI_SHM is specified with auto by default, does it mean I_MPI_SHM use one of the transport above? Like for Intel(R) Xeon(R) processors based on Intel(R) microarchitecture code name Skylake, it uses skx_sse, bdw_avx2 or skx_avx512? And it prefers skx_avx512 when the AVX512 instruction set is used.

And could I use cma used in UCX by setting export I_MPI_SHM=cma?

Thank you!

Re:Specify a shared memory transport

SantoshY_Intel — Tue, 09 Nov 2021 09:40:50 GMT

Hi,

Thanks for reaching out to us.

The value of I_MPI_SHM depends on the value of I_MPI_FABRICS as follows:

if I_MPI_FABRICS is ofi, I_MPI_SHM is disabled.
If I_MPI_FABRICS is shm:ofi, I_MPI_SHM defaults to "auto" or takes the specified value.

>>"I wonder when I_MPI_SHM is specified with auto by default, does it mean I_MPI_SHM use one of the transport above?"

Yes.

If I_MPI_SHM defaults to auto, it takes any specified values depending on their Intel microarchitecture code name. For more information please refer to the below link:

https://www.intel.com/content/www/us/en/develop/documentation/mpi-developer-reference-linux/top/environment-variable-reference/environment-variables-for-fabrics-control/shared-memory-control.html

>>"And could I use cma used in UCX by setting export I_MPI_SHM=cma?"

No, you could not set cma to I_MPI_SHM.

Only the specified values ( disable | no | off | 0 / bdw_sse / bdw_avx2 / skx_sse / skx_avx2 / skx_avx512 / knl_ddr / knl_mcdram / clx_sse /clx_avx2 / clx_avx512 /clx-ap / icx ) can be explicitly specified using I_MPI_SHM.

Thanks & Regards,

Santosh

Re:Specify a shared memory transport

SantoshY_Intel — Wed, 17 Nov 2021 10:54:26 GMT

Hi,

We haven't heard back from you. Is there anything else that we could help you with? If not, then could you please confirm whether to close this thread from our end?

Thanks & Regards,

Santosh

Re:Specify a shared memory transport

SantoshY_Intel — Wed, 24 Nov 2021 06:17:58 GMT

Hi,

We have not heard back from you. This thread will no longer be monitored by Intel. If you need further assistance, please post a new question.

Thanks & Regards,

Santosh