Intel® Distribution for Python*
Support and discussions for achieving faster Python* applications and core computational packages.

spawn_multiple with mpi4py on Windows

maartenb
New Contributor I
910 Views

I'm trying to spawn multiple processes from python using mpi4py.

 

sub_comm = MPI.COMM_SELF.Spawn_multiple(["app.exe"]*nProcess,info=mpiInfo, args = args)

 

It fails because the root of my intel python distribution (C:\intelpython3\) gets prepended to the command.

 

[proxy:1:0@xxxx] HYD_spawn (..\windows\src\hydra_spawn.c:231): unable to run process c:\intelpython3/app.exe (error code 2)
[proxy:1:0@xxxx] launch_processes (proxy.c:603): error creating process (error code 2). The system cannot find the file specified.

 

I tried passing an info object with the "wdir" set to the location of "app.exe", but it didn't help. How can I prevent this?

Thanks,

Maarten

 

 

0 Kudos
1 Solution
maartenb
New Contributor I
864 Views

OK I solved it. Turning on debug information by setting environment variable I_MPI_HYDRA_DEBUG=on showed that there's another parameter in the MPI Info object called "path", which is set by default to the location of the python distribution. The wdir directory is assumed to be relative to this path. Setting it to the appropriate directory and passing it to Spawn_multiple() solved it.

mpiInfo = MPI.Info.Create()
mpiInfo.Set('path','path/to/set')
mpiInfo.Set('wdir','working/directory')
sub_comm = MPI.COMM_SELF.Spawn_multiple(["app.exe"]*nProcess,info=[mpiInfo]*nProcess, args = args)

 

View solution in original post

2 Replies
maartenb
New Contributor I
865 Views

OK I solved it. Turning on debug information by setting environment variable I_MPI_HYDRA_DEBUG=on showed that there's another parameter in the MPI Info object called "path", which is set by default to the location of the python distribution. The wdir directory is assumed to be relative to this path. Setting it to the appropriate directory and passing it to Spawn_multiple() solved it.

mpiInfo = MPI.Info.Create()
mpiInfo.Set('path','path/to/set')
mpiInfo.Set('wdir','working/directory')
sub_comm = MPI.COMM_SELF.Spawn_multiple(["app.exe"]*nProcess,info=[mpiInfo]*nProcess, args = args)

 

RaeesaM_Intel
Moderator
854 Views

Hi Maarten,

 

Glad to know that your issue is resolved and thank you for sharing the solution as it will be beneficial for the community.We will no longer monitor this thread and please feel free to raise a new thread if you have any further queries.

 

Regards,

Raeesa

 

 

Reply