Intel® oneAPI HPC Toolkit
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
1959 Discussions

running intel oneAPI MPI test.f90 example on multiple nodes

phangelder
Beginner
1,187 Views

Installed oneAPI basic tool and HPC kits on my Window 10 Computer.

Complied and linked successfully under Visual Studio 2017.

Program run successfully with mpiexec test.exe on my computer.

Hello world: rank 0 of 4 running on myPC
Hello world: rank 1 of 4 running on myPC
Hello world: rank 2 of 4 running on myPC
Hello world: rank 3 of 4 running on myPC

When i tried to run on NODE1 over our network with

mpiexec -n 2 -ppn 4 -hosts NODE1 test.exe

I received the following message

[mpiexec@myPC] HYD_sock_connect (..\windows\src\hydra_sock.c:240): Retrying connection, retry_count=1, retries=0
[mpiexec@myPC] HYD_connect_to_service (bstrap\service\service_launch.c:85): assert (!closed) failed
[mpiexec@myPC] HYDI_bstrap_service_launch (bstrap\service\service_launch.c:419): unable to connect to hydra service (ithink:8680)
[mpiexec@myPC] remote_launch (bstrap\src\intel\i_hydra_bstrap.c:607): error launching bstrap proxy
[mpiexec@myPC] single_launch (bstrap\src\intel\i_hydra_bstrap.c:665): remote launch error
[mpiexec@myPC] launch_bstrap_proxies (bstrap\src\intel\i_hydra_bstrap.c:849): single launch error
[mpiexec@myPC] HYD_bstrap_setup (bstrap\src\intel\i_hydra_bstrap.c:1043): unable to launch bstrap proxy
[mpiexec@myPC] HYD_print_bstrap_setup_error_message (intel\i_mpiexec.c:1031): error setting up the bootstrap proxies
[mpiexec@myPC] Possible reasons:
[mpiexec@myPC] 1. Host is unavailable. Please check that all hosts are available.
[mpiexec@myPC] 2. Cannot launch hydra_bstrap_proxy.exe or it crashed on one of the hosts. Make sure hydra_bstrap_proxy.exe is available on all hosts and it has right permissions.
[mpiexec@myPC] 3. Firewall refused connection. Check that enough ports are allowed in the firewall and specify them with the I_MPI_PORT_RANGE variable.
[mpiexec@myPC] 4. service bootstrap cannot launch processes on remote host. You may try using -bootstrap option to select alternative launcher.

 

Appreciate to receive any advice and help from the community in advance

 

0 Kudos
11 Replies
HemanthCH_Intel
Moderator
1,155 Views

Hi,

 

Thanks for reaching out to us.

 

Could you please let us know whether you are able to do connect remotely to NODE1 or not?

Could you please provide us your network/cluster and environment details to investigate at our end?

Could you please provide the oneAPI and MPI library Version which you are using?

Could you also provide the complete debug log using I_MPI_DEBUG=20?

 

Initially set I_MPI_DEBUG=20 later run the command.

 

Example:

set I_MPI_DEBUG=20

mpiexec -n 2 -ppn 4 -host NODE1 test.exe

 

Thanks & Regards,

Hemanth.

 

phangelder
Beginner
1,129 Views

Could you please let us know whether you are able to do connect remotely to NODE1 or not?

===
I can ping and get response a from NODE1
But I can't get response with
mpiexec -n 1 -ppn 1 -hosts NODE1 notepad.exe

===
Could you please provide us your network/cluster and environment details to investigate at our end?

====

I am running on Windows 10 (Home edition) connecting to NODE1 thru' Ethernet. Total number of nodes (PC) is about 20

===
Could you please provide the oneAPI and MPI library Version which you are using?

====

I am using the latest oneAPI (2021.5.0) and MPI library (2021.5.0)

====

Could you also provide the complete debug log using I_MPI_DEBUG=20?

Initially set I_MPI_DEBUG=20 later run the command.

 

Example:

set I_MPI_DEBUG=20

mpiexec -n 2 -ppn 4 -hosts NODE1 test.exe

====

Attached please find the log running the test.exe locally.

No debug log was produced when running NODE1 as hosts

====

HemanthCH_Intel
Moderator
1,007 Views

Hi,

 

Could you please confirm whether the firewall is disabled across all the nodes in the cluster?

 

Please refer to the hello world program attached below.

Login to node1 and try running a sample hello world program on node1. Use the below command to compile and run the program.

 

mpiicc hello_world.c

mpiexec -n 4 hello_world.exe

 

Please run the above commands on node1 and provide us the results or screenshot.

 

Thanks & Regards,

Hemanth.

 

phangelder
Beginner
987 Views

K:\>mpiexec -n 4 hello_world.exe
Hello world from processor myPC, rank 2 out of 4 processors
Hello world from processor myPC, rank 1 out of 4 processors
Hello world from processor myPC, rank 3 out of 4 processors
Hello world from processor myPC, rank 0 out of 4 processors

K:\>mpiexec -n 4 -host NOD1 hello_world.exe
[mpiexec@myPC] HYD_sock_connect (..\windows\src\hydra_sock.c:240): Retrying connection, retry_count=1, retries=0
[mpiexec@myPC] HYD_connect_to_service (bstrap\service\service_launch.c:85): assert (!closed) failed
[mpiexec@myPC] HYDI_bstrap_service_launch (bstrap\service\service_launch.c:419): unable to connect to hydra service (meetingroom:8680)
[mpiexec@myPC] remote_launch (bstrap\src\intel\i_hydra_bstrap.c:607): error launching bstrap proxy
[mpiexec@myPC] single_launch (bstrap\src\intel\i_hydra_bstrap.c:665): remote launch error
[mpiexec@myPC] launch_bstrap_proxies (bstrap\src\intel\i_hydra_bstrap.c:849): single launch error
[mpiexec@myPC] HYD_bstrap_setup (bstrap\src\intel\i_hydra_bstrap.c:1043): unable to launch bstrap proxy
[mpiexec@myPC] HYD_print_bstrap_setup_error_message (intel\i_mpiexec.c:1031): error setting up the bootstrap proxies
[mpiexec@myPC] Possible reasons:
[mpiexec@myPC] 1. Host is unavailable. Please check that all hosts are available.
[mpiexec@myPC] 2. Cannot launch hydra_bstrap_proxy.exe or it crashed on one of the hosts. Make sure hydra_bstrap_proxy.exe is available on all hosts and it has right permissions.
[mpiexec@myPC] 3. Firewall refused connection. Check that enough ports are allowed in the firewall and specify them with the I_MPI_PORT_RANGE variable.
[mpiexec@myPC] 4. service bootstrap cannot launch processes on remote host. You may try using -bootstrap option to select alternative launcher.

=========================================================================================

Yes, the firewall is disabled for myPC and NODE1

Can you please explain "Login to NODE1" and "Please run the above commands on node1"?

I think i am missing something here.

MyPC has been upgraded to Windows 10 Pro and NODE1 is running on Windows 10 Home .

Please advice

HemanthCH_Intel
Moderator
980 Views

Hi,

 

We want you to run the MPI program from NODE1. Open a terminal in the NODE1 machine and try running the sample hello world mpi program using the below command:

mpiicc hello_world.c

mpiexec -n 4 hello_world.exe

 

>>>"MyPC has been upgraded to Windows 10 Pro and NODE1 is running on Windows 10 Home ."

It is possible to connect machines running dissimilar versions of Windows.

 

Please let us know whether you are able to run successfully. If not, please share with us the complete debug log.

 

Thanks & Regards,

Hemanth.

 

phangelder
Beginner
972 Views

Hi

Yes, I can run it successfully on NODE1.

Do I need to install windows server ? If so, which version.

 

Merry X'mas

HemanthCH_Intel
Moderator
929 Views

Hi,


We are working on your issue and will get back to you soon.


Thanks & Regards,

Hemanth.


HemanthCH_Intel
Moderator
881 Views

Hi,

 

>>>"Do I need to install windows server ? If so, which version."

We recommend you go through the below link. As the link is not updated, you can ignore the products' versions.

https://www.intel.com/content/www/us/en/developer/articles/training/micro-cluster-setup-with-intel-m...

 

Please make sure that you are following all the guidelines mentioned in the link.

 

Thanks & Regards,

Hemanth.

 

 

HemanthCH_Intel
Moderator
803 Views

Hi,

 

Could you please check whether the file "hydra_bstrap_proxy" is available on all nodes using the below instruction?

Instruction:

Navigate to "C:\Program Files (x86)\intel\oneAPI\mpi\2021.5.0\bin" to this path and make sure you have the "hydra_bstrap_proxy.exe" in all the nodes.

 

Could you please try with the "whoami" command in node1?

whoami

Output: domain-name\user-name

Also, run the below command:

mpiexec -n 2 -ppn 2 -host "domain-name\username" sample.exe

 

If still the issue persists, then please provide the debug log by using the below commands:

set I_MPI_DEBUG=30
set FI_LOG_LEVEL=debug
mpiexec -n 2 -ppn 2 -host "domain-name\username" sample.exe

 

Thanks & Regards,

Hemanth.

 

HemanthCH_Intel
Moderator
716 Views

Hi,

 

We have not heard back from you. Could you please confirm whether your is fixed or not? If you are still facing any issues, then please provide the debug log.

 

Thanks & Regards,

Hemanth.

 

HemanthCH_Intel
Moderator
658 Views

Hi,


We assume that your issue is resolved. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.


Thanks & Regards,

Hemanth.


Reply