topic Re: MPI rank hangs on single node in Intel® MPI Library

MPI rank hangs on single node

Jamie9 — Sat, 26 Feb 2022 13:14:23 GMT

Hello,

I have noticed that every few runs my program hangs on the second MPI rank.

This Fortran program comes from Intel Community web site as an example of hybrid Open MP and MPI programming style.

Details:

env: Windows 10 Pro x64 21H1, Processor Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz, 4 cores HT:ON, ifort 2021.5.0, Intel MPI 2021.5

compilation command: mpif90 -Qopenmp hybrid-hello.f90

run command: mpiexec -np 2 hybrid-hello.exe

Please share your thoughts how to get rid of these hangs.

Thank you in advance,

Jamie

Re: MPI rank hangs on single node

HemanthCH_Intel — Mon, 28 Feb 2022 10:32:21 GMT

Hi,

Thank you for posting in Intel Communities.

We are able to reproduce your issue at our end by using the below command.

mpiexec -n 2 -genv OMP_NUM_THREADS=8 -genv I_MPI_PIN_DOMAIN=omp hybrid-hello.exe

We are working on your issue and will get back to you soon.

Thanks & Regards,

Hemanth.

Re: MPI rank hangs on single node

HemanthCH_Intel — Mon, 07 Mar 2022 13:23:46 GMT

Hi,

We are still investigating your issue and will get back to you soon.

Thanks & Regards,

Hemanth.

Re: MPI rank hangs on single node

HemanthCH_Intel — Wed, 16 Mar 2022 03:48:05 GMT

Hi,

We tried to check the correctness of your code using the below command:

mpif90 -check_mpi hybrid-hello.f90 -Qopenmp mpiexec -n 2 -genv OMP_NUM_THREADS=8 hybrid-hello.exe --itc-args --check-tracing ON --itc-args-end

Using the above command, we encountered a deadlock as attached in the below log file(deadlock.txt).

Could you please try with the attached code(hybrid_hello_latest.f90) where we added: "!$OMP BARRIER" in line no:155 as shown in the screenshot?

We have tried at our end and it works fine. We also checked the correctness of the program which gave the expected results(output.txt)

Please let us know if this resolves your issue. If not, please get back to us.

Thanks & Regards,

Hemanth.

Re: MPI rank hangs on single node

Jamie9 — Tue, 15 Mar 2022 21:33:30 GMT

Hello,

It works as expected after you added !$OMP BARRIER. Thank you very much.

Best Regards,

Jamie9

Re:MPI rank hangs on single node

HemanthCH_Intel — Thu, 17 Mar 2022 11:17:46 GMT

Hi,

Glad to know that your issue is resolved. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks & Regards,

Hemanth.