AVX-512 being used incorrectly in MPI broadcast?

[bg2b]

I've been trying to run an application under Intel MPI and was running into illegal instruction errors.  After some debugging, I found that it was happening within the MPI libraries, apparently trying to use AVX-512 instructions despite the CPUs on my machine only supporting AVX2.  After further testing, I found that I get the same problem if I just try to run the Bcast benchmark in the IMB-MPI1 program that's part of the Intel MPI installation, so I don't think it's anything to do with the specific program I'm trying to run.  Questions:

1. Is this possibly some sort of installation misconfiguration?
2. I found that I can avoid the issue by disabling shared memory (I_MPI_SHM=0), but I'd like to keep shared memory and just disable AVX-512; is that possible?

A little log file with info about my CPU, the command to run the broadcast benchmark, and a backtrace showing it apparently trying to use an AVX-512 version of memcpy is attached.

[ShivaniK_Intel · Accepted solution]

Hii,

Thanks for reaching out to us.

>> Is this possibly some sort of installation misconfiguration?

  We are not sure yet.

>>I found that I can avoid the issue by disabling shared memory (I_MPI_SHM=0), but I'd like to keep shared memory and just disable AVX-512; is that possible?

 Based on the Intel microarchitecture code name you can use the I_MPI_SHM option refer to the below link for details.

 https://software.intel.com/content/www/us/en/develop/documentation/mpi-developer-reference-linux/top/environment-variable-reference/

 environment-variables-for-fabrics-control/shared-memory-control.html

 For example:

If your Intel microarchitecture code name is Broadwell you can use the option I_MPI_SHM=bdw_avx2 for the avx2 instruction set to be used.

Based on the processor you can refer to the below link to find out the Intel microarchitecture code name.

 https://ark.intel.com/

After using the specific options please let us know if you further face any issues.

Thanks & Regards

Shivani

[ShivaniK_Intel]

Hii,

Information provided regarding the Model name suggests that the Model name: Gold 6148 CPU supports AVX-512. Is there any case like AVX-512 is disabled, please check BIOS, or else you are using hypervisor which may disable AVX-512. Once can you recheck these details and get back to us.

Thanks&Regards

Shivani

[bg2b]

Ah, it may be that the AVX-512 support is disabled either in the kernel (the OS is RH 6.5) or the BIOS.  That would explain why the CPU flags show up as only supporting up to AVX2 despite the CPU name indicating that it's nominally capable of more.  The workaround above with shared memory but specifying avx2 seems fine, so I probably won't dig more into this or ask IT to upgrade the OS or something.  I'm not sure how the MPI library is detecting AVX-512 support, but when I started this thread I was just looking just at the CPU flags and I couldn't imagine how the library thought AVX-512 would be OK.

[ShivaniK_Intel]

Hii,

As setting the I_MPI_SHM resolved your issue. Please let us know if we can close the thread.

Thanks&Regards

Shivani

[ShivaniK_Intel]

Hii,

Thanks for the confirmation. If you need any additional information please, submit a new question as this thread will no longer be monitored.

Thanks & Regards

Shivani