Running mpi w/o mpiexec/mpirun

[jaewonj]

I can compile and run a mpi application using mpiexec without any difficulties.

What I want to do is to run a mpi application not in Terminal,I mean without using mpiexec or mpirun.

Is this doable? If it is, how can I setup mpd daemon and other necessary stuffs in source level?

Thanks. 

==========

hello.c

==========

#include 

#include 

static int run()

{
	int node;

	MPI_Init(xxx, xxx); /* TODO */

	MPI_Comm_rank(MPI_COMM_WORLD, &node);
	printf("Hello World from Node (%d)\n", node);
	MPI_Finalize();
	return 0;	
}

int main (int argc, char **argv)

{

run();

return 0;

}

[Gergana_S_Intel]

Hi jaewonj,

That's an interesting use case. Do you mind sharing with us why you're looking to start the daemons within the source code?

I, personally, have only seen the start of the MPD daemons and MPI programs via scripts (be it shell or python) but not within compiled code. Perhaps others on this forum have a different experience. I know that with the Intel MPI Library, we don't provide any sort of system-level calls you can use.

Regards,
~Gergana

[TimP]

Of course, it's a common mode of operation to run all of it from a script, either under a queuing system or directly from the user session.

[vastviewec]

Quoting - Gergana Slavova (Intel)

Hi jaewonj,

That's an interesting use case. Do you mind sharing with us why you're looking to start the daemons within the source code?

I, personally, have only seen the start of the MPD daemons and MPI programs via scripts (be it shell or python) but not within compiled code. Perhaps others on this forum have a different experience. I know that with the Intel MPI Library, we don't provide any sort of system-level calls you can use.

Regards,
~Gergana

Hello Gergana,

I can compile and run a program with INTEL MPI library and run with mpiexec without any difficulties. However, when I tried to run MPI programs via bash shell, I'm getting following error message

mpdroot: cannot connect to local mpd at: /tmp/mpd2.console_root
probable cause: no mpd daemon on this machine
possible cause: unix socket /tmp/mpd2.console_root has been removed
mpiexec_user-desktop (__init__ 1186): forked process failed; status=255

My test command in bash file is very simple

#!/bin/sh
mpdboot &
mpiexec -n 2 ./program.exe
mpdallexit

It seems to me that mpi daemon is not started correctly. Is there a speical way to call mpi in shell scripts?

Thanks very much!

JACKY

[Gergana_S_Intel]

Quoting - vastviewec

My test command in bash file is very simple

#!/bin/sh
mpdboot &
mpiexec -n 2 ./program.exe
mpdallexit

It seems to me that mpi daemon is not started correctly. Is there a speical way to call mpi in shell scripts?

Hi Jacky,

I would recommend that you amend your script as follows:

#!/bin/sh
mpdboot
mpiexec -n 2 ./program.exe
mpdallexit

Note the missing "&" after mpdboot. That's why the script was failing before: by putting mpdboot in the background, you're notwaiting forthe Intel MPI library to successfully start all MPDs, and *then* use those daemons to run your MPI program. In effect, it would be like running mpiexec with no mpdboot preceding it.

I hope this makes sense. Let me know how it goes and if you have further questions.

Regards,
~Gergana

[vastviewec]

Quoting - Gergana Slavova (Intel)

Hi Jacky,

I would recommend that you amend your script as follows:

#!/bin/sh
mpdboot
mpiexec -n 2 ./program.exe
mpdallexit

Note the missing "&" after mpdboot. That's why the script was failing before: by putting mpdboot in the background, you're notwaiting forthe Intel MPI library to successfully start all MPDs, and *then* use those daemons to run your MPI program. In effect, it would be like running mpiexec with no mpdboot preceding it.

I hope this makes sense. Let me know how it goes and if you have further questions.

Regards,
~Gergana

Thanks Gergana! The problem is solved by using your solustion!

Regards,

Jacky

[duzyatko2yahoo_com]

I think that it is a bit more convenient to run such programs via scripts.

___________
essay paper help

[jimmy82]

Previously in MPI implementation, there is a way to specify the environment variables and start each process individually without using MPIexec.

I wonder if anybody has been successful in doing it this way using Intel MPI? Certain applications that I require has to be invoked separately, so I will require such ways of starting each node.

Hope somebody can advise.

[Gergana_S_Intel]

Hi jimmy,

Can you perhaps give an example of an MPI implementation that doesn't use mpiexec. A lot of the ones I know either use that, or mpirun to control the MPI processes and communication.

In terms of environment variables for Intel MPI Library, those can be set whenever you want - either before you start your MPI processes, or on the command line.

Regards,
~Gergana

[jimmy82]

In the past, I have an application that actually calls mpi_init which blocks till all the applications on the other nodes are also initialised. This is achieved by setting the PMI environment variables. Is this method also applicable for Intel MPI?

The previous implementation that I talked about is using MPICH 1.

[lordme]

After loading an Intel MPI module file, compiler wrappers with prefix mpi are in your $PATH. The remaining name component of a compiler wrapper command specifies the underlying compiler to be used (mpigcc uses gcc, mpiifort uses ifort).
___________________

bands rings | swarovski crystal beads

[jaewonj]

Hello Gergana,

I haven't checked this thread for a while. Thanks for answering my question.

What I wanted to do was to run parallel libraries such as PETSc from within my application. As a quick and dirty workaround, I made a sperate executable and called it from my application using system command. I used binary file stream to communicate with that executable.

I think my question still stands. Do you think it is doable, I mean running MPI app without mpiexec?

Best Regards,
Jaewon

[TimP]

Quoting jaewonj

I think my question still stands. Do you think it is doable, I mean running MPI app without mpiexec?

This makes sense only in the context of future versions of MPI, where mpiexec may be implicit in an application which is linked against MPI. If you have available a system() function, and can set up the MPI environment, why not include mpiexec in the command you start under system()?

[jaewonj]

Thanks tim18. As you said it's possible to invoke mpiexec via system command but I don't like to use filestream.

Best regards,
Jaewon