MPI_Init_thread or MPI_Init failed in child process

[Yongjun_L_]

I have two programs, A and B. They all are developed with MPI. A will call B. 

If I directly start A and call B, every thing is OK.

If I start A with mpiexec, like mpiexec -localonly 2 A.exe, and call B. MPI_Init_thread or MPI_Init will fail in B. 

Below is the error message I got.

[01:2668]..ERROR:Error while connecting to host, No connection could be made because the target machine actively refused it. (10061)
[01:2668]..ERROR:Connect on sock (host=localhost, port=53649) failed, exhaused all end points
SMPDU_Sock_post_connect failed.
[1] PMI_ConnectToHost failed: unable to post a connect to localhost:53649, error: Undefined dynamic error code
uPMI_ConnectToHost returning PMI_FAIL
[1] PMI_Init failed.
Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(659):
MPID_Init(154).......: channel initialization failed
MPID_Init(448).......: PMI_Init returned -1

Can anyone tell me what is the problem? How to solve it?

Thanks

Yongjun

[Gergana_S_Intel]

Hi Yongjun,

You'll have to give us more details on how exactly you call B from A.  Is program A a script that simply calls B?  Or do you mean you're spawning a process from A that then executes B?

If you have 2 executables, you can use argument sets to run both of them simultaneously:

mpiexec -localonly -n 1 A.exe : -n 1 B.exe

Looking forward to hearing back.

Regards,
~Gergana

[Yongjun_L_]

Hi Gergana,

Than you for your reply.

A and B are two separate executable files.  B must be called from A.

A use API to start B and pass argument to B. There is no communication between A and B.

Yongjun 

[Yongjun_L_]

Hi Gergana,

I want to start multiple As on different computers and each A will start B with different argument. A doesn't need use mpiexec to start B. 

Yongjun

[Gergana_S_Intel]

Thanks for clarifying, Yongjun.  Are A and B both MPI applications?  Or is A non-MPI but B is an MPI application?

[Yongjun_L_]

A and B are both MPI applications. 

I start two A on two computers. The command is 

      mpiexec -np 2 -machinefile host.txt  A.exe

Each A will start B with different argument

      system( "B arg1" );

MPI_Init_thread will fail in B and cause B crash.

[Gergana_S_Intel]

That's not a supported model.  Most likely what's happening here is: prog A gets initialized correctly and has the global communicator (MPI_COMM_WORLD) properly defined.  Then prog B starts from within A but it's unable to get initialized.

Can you run B as a stand-alone?  I looked back in the thread but didn't see that scenario.  You only say that A starts outside of mpiexec.

Alternatively, have you tried doing system("mpiexec -localonly B arg").  I don't know if that'll work but might be worth a try.

I'll ask around and see if anyone has worked on a similar application before.

Thanks,
~Gergana

[Yongjun_L_]

Hi Gergana,

Of course I can run B as standalone.

I tried to use mpiexec -localonly 1 B arg to start B before. It doesn't work for me.

The only problem is that if I use mpiexec to start A, B will fail. 

Yongjun