topic Re: running intel oneAPI MPI test.f90 example on multiple nodes in Intel® MPI Library

running intel oneAPI MPI test.f90 example on multiple nodes

phangelder — Mon, 13 Dec 2021 12:16:08 GMT

Installed oneAPI basic tool and HPC kits on my Window 10 Computer.

Complied and linked successfully under Visual Studio 2017.

Program run successfully with mpiexec test.exe on my computer.

Hello world: rank 0 of 4 running on myPC
Hello world: rank 1 of 4 running on myPC
Hello world: rank 2 of 4 running on myPC
Hello world: rank 3 of 4 running on myPC

When i tried to run on NODE1 over our network with

mpiexec -n 2 -ppn 4 -hosts NODE1 test.exe

I received the following message

[mpiexec@myPC] HYD_sock_connect (..\windows\src\hydra_sock.c:240): Retrying connection, retry_count=1, retries=0
[mpiexec@myPC] HYD_connect_to_service (bstrap\service\service_launch.c:85): assert (!closed) failed
[mpiexec@myPC] HYDI_bstrap_service_launch (bstrap\service\service_launch.c:419): unable to connect to hydra service (ithink:8680)
[mpiexec@myPC] remote_launch (bstrap\src\intel\i_hydra_bstrap.c:607): error launching bstrap proxy
[mpiexec@myPC] single_launch (bstrap\src\intel\i_hydra_bstrap.c:665): remote launch error
[mpiexec@myPC] launch_bstrap_proxies (bstrap\src\intel\i_hydra_bstrap.c:849): single launch error
[mpiexec@myPC] HYD_bstrap_setup (bstrap\src\intel\i_hydra_bstrap.c:1043): unable to launch bstrap proxy
[mpiexec@myPC] HYD_print_bstrap_setup_error_message (intel\i_mpiexec.c:1031): error setting up the bootstrap proxies
[mpiexec@myPC] Possible reasons:
[mpiexec@myPC] 1. Host is unavailable. Please check that all hosts are available.
[mpiexec@myPC] 2. Cannot launch hydra_bstrap_proxy.exe or it crashed on one of the hosts. Make sure hydra_bstrap_proxy.exe is available on all hosts and it has right permissions.
[mpiexec@myPC] 3. Firewall refused connection. Check that enough ports are allowed in the firewall and specify them with the I_MPI_PORT_RANGE variable.
[mpiexec@myPC] 4. service bootstrap cannot launch processes on remote host. You may try using -bootstrap option to select alternative launcher.

Appreciate to receive any advice and help from the community in advance

Re: running intel oneAPI MPI test.f90 example on multiple nodes

HemanthCH_Intel — Thu, 27 Jan 2022 11:36:44 GMT

Hi,

Thanks for reaching out to us.

Could you please let us know whether you are able to do connect remotely to NODE1 or not?

Could you please provide us your network/cluster and environment details to investigate at our end?

Could you please provide the oneAPI and MPI library Version which you are using?

Could you also provide the complete debug log using I_MPI_DEBUG=20?

Initially set I_MPI_DEBUG=20 later run the command.

Example:

set I_MPI_DEBUG=20

mpiexec -n 2 -ppn 4 -host NODE1 test.exe

Thanks & Regards,

Hemanth.

Re: running intel oneAPI MPI test.f90 example on multiple nodes

phangelder — Thu, 16 Dec 2021 04:37:35 GMT

Could you please let us know whether you are able to do connect remotely to NODE1 or not?

===
I can ping and get response a from NODE1
But I can't get response with
mpiexec -n 1 -ppn 1 -hosts NODE1 notepad.exe

===
Could you please provide us your network/cluster and environment details to investigate at our end?

====

I am running on Windows 10 (Home edition) connecting to NODE1 thru' Ethernet. Total number of nodes (PC) is about 20

===
Could you please provide the oneAPI and MPI library Version which you are using?

====

I am using the latest oneAPI (2021.5.0) and MPI library (2021.5.0)

====

Could you also provide the complete debug log using I_MPI_DEBUG=20?

Initially set I_MPI_DEBUG=20 later run the command.

Example:

set I_MPI_DEBUG=20

mpiexec -n 2 -ppn 4 -hosts NODE1 test.exe

====

Attached please find the log running the test.exe locally.

No debug log was produced when running NODE1 as hosts

====

Re: running intel oneAPI MPI test.f90 example on multiple nodes

HemanthCH_Intel — Wed, 22 Dec 2021 12:47:15 GMT

Hi,

Could you please confirm whether the firewall is disabled across all the nodes in the cluster?

Please refer to the hello world program attached below.

Login to node1 and try running a sample hello world program on node1. Use the below command to compile and run the program.

mpiicc hello_world.c

mpiexec -n 4 hello_world.exe

Please run the above commands on node1 and provide us the results or screenshot.

Thanks & Regards,

Hemanth.

Re: running intel oneAPI MPI test.f90 example on multiple nodes

phangelder — Fri, 24 Dec 2021 05:21:19 GMT

K:\>mpiexec -n 4 hello_world.exe
Hello world from processor myPC, rank 2 out of 4 processors
Hello world from processor myPC, rank 1 out of 4 processors
Hello world from processor myPC, rank 3 out of 4 processors
Hello world from processor myPC, rank 0 out of 4 processors

K:\>mpiexec -n 4 -host NOD1 hello_world.exe
[mpiexec@myPC] HYD_sock_connect (..\windows\src\hydra_sock.c:240): Retrying connection, retry_count=1, retries=0
[mpiexec@myPC] HYD_connect_to_service (bstrap\service\service_launch.c:85): assert (!closed) failed
[mpiexec@myPC] HYDI_bstrap_service_launch (bstrap\service\service_launch.c:419): unable to connect to hydra service (meetingroom:8680)
[mpiexec@myPC] remote_launch (bstrap\src\intel\i_hydra_bstrap.c:607): error launching bstrap proxy
[mpiexec@myPC] single_launch (bstrap\src\intel\i_hydra_bstrap.c:665): remote launch error
[mpiexec@myPC] launch_bstrap_proxies (bstrap\src\intel\i_hydra_bstrap.c:849): single launch error
[mpiexec@myPC] HYD_bstrap_setup (bstrap\src\intel\i_hydra_bstrap.c:1043): unable to launch bstrap proxy
[mpiexec@myPC] HYD_print_bstrap_setup_error_message (intel\i_mpiexec.c:1031): error setting up the bootstrap proxies
[mpiexec@myPC] Possible reasons:
[mpiexec@myPC] 1. Host is unavailable. Please check that all hosts are available.
[mpiexec@myPC] 2. Cannot launch hydra_bstrap_proxy.exe or it crashed on one of the hosts. Make sure hydra_bstrap_proxy.exe is available on all hosts and it has right permissions.
[mpiexec@myPC] 3. Firewall refused connection. Check that enough ports are allowed in the firewall and specify them with the I_MPI_PORT_RANGE variable.
[mpiexec@myPC] 4. service bootstrap cannot launch processes on remote host. You may try using -bootstrap option to select alternative launcher.

=========================================================================================

Yes, the firewall is disabled for myPC and NODE1

Can you please explain "Login to NODE1" and "Please run the above commands on node1"?

I think i am missing something here.

MyPC has been upgraded to Windows 10 Pro and NODE1 is running on Windows 10 Home .

Please advice

Re: running intel oneAPI MPI test.f90 example on multiple nodes

HemanthCH_Intel — Thu, 27 Jan 2022 11:45:32 GMT

Hi,

We want you to run the MPI program from NODE1. Open a terminal in the NODE1 machine and try running the sample hello world mpi program using the below command:

mpiicc hello_world.c

mpiexec -n 4 hello_world.exe

>>>"MyPC has been upgraded to Windows 10 Pro and NODE1 is running on Windows 10 Home ."

It is possible to connect machines running dissimilar versions of Windows.

Please let us know whether you are able to run successfully. If not, please share with us the complete debug log.

Thanks & Regards,

Hemanth.

Re: running intel oneAPI MPI test.f90 example on multiple nodes

phangelder — Sat, 25 Dec 2021 01:04:36 GMT

Yes, I can run it successfully on NODE1.

Do I need to install windows server ? If so, which version.

Merry X'mas

Re:running intel oneAPI MPI test.f90 example on multiple nodes

HemanthCH_Intel — Sun, 02 Jan 2022 12:12:09 GMT

Hi,

We are working on your issue and will get back to you soon.

Thanks & Regards,

Hemanth.

Re: running intel oneAPI MPI test.f90 example on multiple nodes

HemanthCH_Intel — Thu, 27 Jan 2022 11:46:46 GMT

Hi,

>>>"Do I need to install windows server ? If so, which version."

We recommend you go through the below link. As the link is not updated, you can ignore the products' versions.

https://www.intel.com/content/www/us/en/developer/articles/training/micro-cluster-setup-with-intel-mpi-a-stepwise-guide-to-setting-up-the-smallest-windows.html

Please make sure that you are following all the guidelines mentioned in the link.

Thanks & Regards,

Hemanth.

Re: running intel oneAPI MPI test.f90 example on multiple nodes

HemanthCH_Intel — Wed, 12 Jan 2022 13:01:13 GMT

Hi,

Could you please check whether the file "hydra_bstrap_proxy" is available on all nodes using the below instruction?

Instruction:

Navigate to "C:\Program Files (x86)\intel\oneAPI\mpi\2021.5.0\bin" to this path and make sure you have the "hydra_bstrap_proxy.exe" in all the nodes.

Could you please try with the "whoami" command in node1?

whoami Output: domain-name\user-name

Also, run the below command:

mpiexec -n 2 -ppn 2 -host "domain-name\username" sample.exe

If still the issue persists, then please provide the debug log by using the below commands:

set I_MPI_DEBUG=30 set FI_LOG_LEVEL=debug mpiexec -n 2 -ppn 2 -host "domain-name\username" sample.exe

Thanks & Regards,

Hemanth.

Re: running intel oneAPI MPI test.f90 example on multiple nodes

HemanthCH_Intel — Wed, 19 Jan 2022 13:22:35 GMT

Hi,

We have not heard back from you. Could you please confirm whether your is fixed or not? If you are still facing any issues, then please provide the debug log.

Thanks & Regards,

Hemanth.

Re:running intel oneAPI MPI test.f90 example on multiple nodes

HemanthCH_Intel — Thu, 27 Jan 2022 11:33:04 GMT

Hi,

We assume that your issue is resolved. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks & Regards,

Hemanth.