https://community.intel.com/t5/Intel-MPI-Library/MPI-run-crashes-on-more-than-one-node/m-p/970159#M3283 <P>Hi everyone,</P> <P>I'm using MPICH2 v1.5 to run my WRF model on INTEL Xeon Processors. I can run on one node with as many cores as I want but if it exceeds the number of porcessors in a core it will crash with following error:</P> <P>*********************************************************************************************************************************************</P> <P>[proxy:0:0@hpc1934] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:883): assert (!closed) failed<BR />[proxy:0:0@hpc1934] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status<BR />[proxy:0:0@hpc1934] main (./pm/pmiserv/pmip.c:210): demux engine error waiting for event<BR />[proxy:0:7@hpc1945] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:883): assert (!closed) failed<BR />[proxy:0:7@hpc1945] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status<BR />[proxy:0:7@hpc1945] main (./pm/pmiserv/pmip.c:210): demux engine error waiting for event<BR />[proxy:0:5@hpc1940] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:883): assert (!closed) failed<BR />[proxy:0:5@hpc1940] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status<BR />[proxy:0:5@hpc1940] main (./pm/pmiserv/pmip.c:210): demux engine error waiting for event<BR />[mpiexec@hpc1934] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting<BR />[mpiexec@hpc1934] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion<BR />[mpiexec@hpc1934] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:216): launcher returned error waiting for completion<BR />[mpiexec@hpc1934] main (./ui/mpich/mpiexec.c:325): process manager error waiting for completion</P> <P>**********************************************************************************************************************************************</P> <P>and this is how I run the model:</P> <P>$ulimit -s unlimited</P> <P>$source ~/setup-intel.sh</P> <P>$mpiexec -np nproc ./wrf.exe &gt;&amp; benchmark#n.log</P> <P>I would appreciate any help in this regard.</P> <P></P> <P>Bests,</P> <P>Arash</P> Wed, 30 Oct 2013 21:25:37 GMT arash_m_ 2013-10-30T21:25:37Z https://community.intel.com/t5/Intel-MPI-Library/MPI-run-crashes-on-more-than-one-node/m-p/970159#M3283 <P>Hi everyone,</P> <P>I'm using MPICH2 v1.5 to run my WRF model on INTEL Xeon Processors. I can run on one node with as many cores as I want but if it exceeds the number of porcessors in a core it will crash with following error:</P> <P>*********************************************************************************************************************************************</P> <P>[proxy:0:0@hpc1934] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:883): assert (!closed) failed<BR />[proxy:0:0@hpc1934] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status<BR />[proxy:0:0@hpc1934] main (./pm/pmiserv/pmip.c:210): demux engine error waiting for event<BR />[proxy:0:7@hpc1945] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:883): assert (!closed) failed<BR />[proxy:0:7@hpc1945] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status<BR />[proxy:0:7@hpc1945] main (./pm/pmiserv/pmip.c:210): demux engine error waiting for event<BR />[proxy:0:5@hpc1940] HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:883): assert (!closed) failed<BR />[proxy:0:5@hpc1940] HYDT_dmxu_poll_wait_for_event (./tools/demux/demux_poll.c:77): callback returned error status<BR />[proxy:0:5@hpc1940] main (./pm/pmiserv/pmip.c:210): demux engine error waiting for event<BR />[mpiexec@hpc1934] HYDT_bscu_wait_for_completion (./tools/bootstrap/utils/bscu_wait.c:76): one of the processes terminated badly; aborting<BR />[mpiexec@hpc1934] HYDT_bsci_wait_for_completion (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for completion<BR />[mpiexec@hpc1934] HYD_pmci_wait_for_completion (./pm/pmiserv/pmiserv_pmci.c:216): launcher returned error waiting for completion<BR />[mpiexec@hpc1934] main (./ui/mpich/mpiexec.c:325): process manager error waiting for completion</P> <P>**********************************************************************************************************************************************</P> <P>and this is how I run the model:</P> <P>$ulimit -s unlimited</P> <P>$source ~/setup-intel.sh</P> <P>$mpiexec -np nproc ./wrf.exe &gt;&amp; benchmark#n.log</P> <P>I would appreciate any help in this regard.</P> <P></P> <P>Bests,</P> <P>Arash</P> Wed, 30 Oct 2013 21:25:37 GMT https://community.intel.com/t5/Intel-MPI-Library/MPI-run-crashes-on-more-than-one-node/m-p/970159#M3283 arash_m_ 2013-10-30T21:25:37Z