topic Thanks for the info. This in Intel® Distribution for Python*

MPI4PY installation

AKOLKAR__RAHUL — Sat, 17 Feb 2018 05:44:46 GMT

Hi All,

I'm HPC Admin. I have installed MPI4PY Library on clusters by .tar and Pip2.7(python2.7). After that, we are facing the issue like a 256cores job (n=4,ppn=64) is not running on nodes. It happened after installing MPI4PY(3.0). normal python code is running.

Users unable to run jobs on Cluster like VASP,MPI4PY, etc.

The error is Given below:

[cli_0]: aborting job:

Fatal error in MPI_Init:

Other MPI error

[mpiexec@tyrone-node16] HYD_pmcd_pmiserv_send_signal (./pm/pmiserv/pmiserv_cb.c:184): assert (!closed) failed

[mpiexec@tyrone-node16] ui_cmd_cb (./pm/pmiserv/pmiserv_pmci.c:74): unable to send SIGUSR1 downstream

[mpiexec@tyrone-node16] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status

[mpiexec@tyrone-node16] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:181): error waiting for event

[mpiexec@tyrone-node16] main (./ui/mpich/mpiexec.c:405): process manager error waiting for completions

Kindly Help me.

Thanks in Advance!

Rahul Akolkar

Hi,

Frank_S_Intel — Mon, 19 Feb 2018 16:20:12 GMT

Hi,

could you please provide the exact install commands you used. The output of "conda list" would also be very helpful to understand what's happening.

frank

Hi Frank,

AKOLKAR__RAHUL — Thu, 08 Mar 2018 05:48:55 GMT

Hi Frank,

Actually, the ib_sdb error was in the boot process, I resolved that issue. But still, I get this error message when I trying to run MPI sample Codes.

This is not the problem by the MPI4PY library.

MPI path

source /opt/mvapich2-1.8-r5423/intel/etc/mvapich_vars.csh

Hi,

Frank_S_Intel — Thu, 08 Mar 2018 09:19:20 GMT

Hi,

you seem to be using MVAPICH. You have to make sure your mpi4py is built against the MPI library you are using underneath.

Please let us know the exact commands you used to install mpi4py and MPI.

frank

Hi Frank,

AKOLKAR__RAHUL — Thu, 08 Mar 2018 09:52:21 GMT

Hi Frank,

I used the command to install for this node: #pip2.7 install mpi4py.

when I invoke mpi4py, it worked. But when users tried to submit VASP job then they got the above HYD_pmcd_pmiserv_send_signal (./pm/pmiserv/pmiserv_cb.c:184): assert (!closed) failed

We are using MVAPICH, open64 and gcc compilers

Thanks!

Rahul A.

Thanks for the info. This

Frank_S_Intel — Thu, 08 Mar 2018 11:25:10 GMT

Thanks for the info. This might be an issue with your MVAPICH installation or a incorrect build of mpi4py (e.g. it might use a different MPI at built time). Unfortunately I can't say for sure.

Intel's mpi4py package uses Intel(R) MPI library and comes pre-compiled. You might want to try conda-install'ing it (and also use Intel(R) Distribution for Python). Let us know if you encounter any issue with our mpi4py package.