Problem with MPIRUN run on multi-nodes Under LINUX

[dingjun_chencmgl_ca]

HI, Dear Sir or Madam,

I am testing the Intel MPI benchmar under RedHat Linux with 2 computers: node1-rh(8 cores) and node2-rh(8 cores). After I enter the following command on the local node1-rh:

bash-4.1$ mpirun -hosts node2-rh, node1-rh -np 16 IMB-MPI1
dingjun@node1-rh's password:

*********

IMB-MPI1 did not run. This indicates the IMB-MPI1 did not run on the remote computer node2-rh.

Could you experience such a problem? please tell my your ideas on how to run a executable file on multi-nodes under Linux with the Intel MPI library. Thanks in advance.

Have a good day and I look forward to hearing from you.

[James_T_Intel]

You should not be seeing a password prompt.  Please set up passwordless SSH for the nodes and try again.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[dingjun_chencmgl_ca]

Thanks and right now it is OK. However, a new problem occurred again.  The details are described as follows:

bash-4.1$ ls
hostfile IMB-EXT IMB-IO IMB-MPI1 IMB-NBC IMB-RMA

bash-4.1$ cat hostfile
imex-snb2s-rh
imexsvr4-rh
imexsvr3-rh
(*Note: all above 3 computers are 2-socket 16-core SundyBridge DELL computers.)

The following 3 benchmarkings can be successful to run without any errors.

bash-4.1$ mpirun -genv I_MPI_FABRICS shm:tcp -n 32 -f hostfile IMB-MPI1

bash-4.1$ mpirun -genv I_MPI_FABRICS shm:tcp -n 32 -f hostfile IMB-NBC

bash-4.1$ mpirun -genv I_MPI_FABRICS shm:tcp -n 32 -f hostfile IMB-RMA

BUT, the following 2 benchmarking failed to run and errors occurred as follows:

bash-4.1$ mpirun -genv I_MPI_FABRICS shm:tcp -n 32 -f hostfile IMB-IO

[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-IO (No such file or directory)

bash-4.1$ mpirun -genv I_MPI_FABRICS shm:tcp -n 32 -f hostfile IMB-EXT

[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:0@imex-snb2s-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
[proxy:0:1@imexsvr4-rh] HYDU_create_process (../../utils/launch/launch.c:572): execvp error on file IMB-EXT (No such file or directory)
 

Please tell me the reasons why such errors occurred for the benchmarking IMB-IO and IMB-EXT.

Thanks again and I look forward to hearing from you as soon as possible.

[dingjun_chencmgl_ca]

I look forward to hearing from you. Thanks anyway.

[James_T_Intel]

This looks to me like a problem with the network share on your system.  Try logging into each directly and simply running IMB-IO and IMB-EXT.  I've seen similar behavior if there are NFS problems, sometimes executables aren't visible on all systems, or sometimes different versions are on different nodes.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[Garg__Mukul]

Hi,

I am facing issue while running below command

mpiexec -np 2 --hosts master slave stress --cpu 8 --timeout 20

getting below error

HYDU_create_process (./utils/launch/launch.c:69): execvp error on file slave (No such file or directory)
HYDU_create_process (./utils/launch/launch.c:69): execvp error on file slave (No such file or directory)

Could you please help me with this.....

Regards,

M