topic Re: Running mpi w/o mpiexec/mpirun in Intel® MPI Library

Running mpi w/o mpiexec/mpirun

jaewonj — Tue, 10 Nov 2009 03:19:48 GMT

I can compile and run a mpi application using mpiexec without any difficulties.

What I want to do is to run a mpi application not in Terminal,I mean without using mpiexec or mpirun.

Is this doable? If it is, how can I setup mpd daemon and other necessary stuffs in source level?

Thanks.

==========

hello.c

==========

#include

#include

static int run()

{
	int node;

	MPI_Init(xxx, xxx); /* TODO */

	MPI_Comm_rank(MPI_COMM_WORLD, &node);
	printf("Hello World from Node (%d)\n", node);
	MPI_Finalize();
	return 0;	
}

int main (int argc, char **argv)

run();

return 0;

Re: Running mpi w/o mpiexec/mpirun

Gergana_S_Intel — Thu, 03 Dec 2009 23:00:15 GMT

Hi jaewonj,

That's an interesting use case. Do you mind sharing with us why you're looking to start the daemons within the source code?

I, personally, have only seen the start of the MPD daemons and MPI programs via scripts (be it shell or python) but not within compiled code. Perhaps others on this forum have a different experience. I know that with the Intel MPI Library, we don't provide any sort of system-level calls you can use.

Regards,
~Gergana

Re: Running mpi w/o mpiexec/mpirun

TimP — Fri, 04 Dec 2009 00:39:08 GMT

Of course, it's a common mode of operation to run all of it from a script, either under a queuing system or directly from the user session.

Re: Running mpi w/o mpiexec/mpirun

vastviewec — Wed, 16 Dec 2009 13:53:38 GMT

Quoting - Gergana Slavova (Intel)

Hi jaewonj,

That's an interesting use case. Do you mind sharing with us why you're looking to start the daemons within the source code?

I, personally, have only seen the start of the MPD daemons and MPI programs via scripts (be it shell or python) but not within compiled code. Perhaps others on this forum have a different experience. I know that with the Intel MPI Library, we don't provide any sort of system-level calls you can use.

Regards,
~Gergana

Hello Gergana,

I can compile and run a program with INTEL MPI library and run with mpiexec without any difficulties. However, when I tried to run MPI programs via bash shell, I'm getting following error message

mpdroot: cannot connect to local mpd at: /tmp/mpd2.console_root
probable cause: no mpd daemon on this machine
possible cause: unix socket /tmp/mpd2.console_root has been removed
mpiexec_user-desktop (__init__ 1186): forked process failed; status=255

My test command in bash file is very simple

#!/bin/sh
mpdboot &
mpiexec -n 2 ./program.exe
mpdallexit

It seems to me that mpi daemon is not started correctly. Is there a speical way to call mpi in shell scripts?

Thanks very much!

JACKY

Re: Running mpi w/o mpiexec/mpirun

Gergana_S_Intel — Wed, 16 Dec 2009 17:06:25 GMT

Quoting - vastviewec

My test command in bash file is very simple

#!/bin/sh
mpdboot &
mpiexec -n 2 ./program.exe
mpdallexit

It seems to me that mpi daemon is not started correctly. Is there a speical way to call mpi in shell scripts?

Hi Jacky,

I would recommend that you amend your script as follows:

#!/bin/sh
mpdboot
mpiexec -n 2 ./program.exe
mpdallexit

Note the missing "&" after mpdboot. That's why the script was failing before: by putting mpdboot in the background, you're notwaiting forthe Intel MPI library to successfully start all MPDs, and *then* use those daemons to run your MPI program. In effect, it would be like running mpiexec with no mpdboot preceding it.

I hope this makes sense. Let me know how it goes and if you have further questions.

Regards,
~Gergana

Re: Running mpi w/o mpiexec/mpirun

vastviewec — Thu, 17 Dec 2009 13:26:50 GMT

Quoting - Gergana Slavova (Intel)

Hi Jacky,

I would recommend that you amend your script as follows:

#!/bin/sh
mpdboot
mpiexec -n 2 ./program.exe
mpdallexit

Note the missing "&" after mpdboot. That's why the script was failing before: by putting mpdboot in the background, you're notwaiting forthe Intel MPI library to successfully start all MPDs, and *then* use those daemons to run your MPI program. In effect, it would be like running mpiexec with no mpdboot preceding it.

I hope this makes sense. Let me know how it goes and if you have further questions.

Regards,
~Gergana

Thanks Gergana! The problem is solved by using your solustion!

Regards,

Jacky

Re: Running mpi w/o mpiexec/mpirun

duzyatko2yahoo_com — Mon, 11 Jan 2010 14:39:00 GMT

I think that it is a bit more convenient to run such programs via scripts.

___________
essay paper help

Running mpi w/o mpiexec/mpirun

jimmy82 — Tue, 23 Mar 2010 16:41:27 GMT

Previously in MPI implementation, there is a way to specify the environment variables and start each process individually without using MPIexec.

I wonder if anybody has been successful in doing it this way using Intel MPI? Certain applications that I require has to be invoked separately, so I will require such ways of starting each node.

Hope somebody can advise.

Running mpi w/o mpiexec/mpirun

Gergana_S_Intel — Wed, 24 Mar 2010 00:14:48 GMT

Hi jimmy,

Can you perhaps give an example of an MPI implementation that doesn't use mpiexec. A lot of the ones I know either use that, or mpirun to control the MPI processes and communication.

In terms of environment variables for Intel MPI Library, those can be set whenever you want - either before you start your MPI processes, or on the command line.

Regards,
~Gergana

Running mpi w/o mpiexec/mpirun

jimmy82 — Thu, 25 Mar 2010 16:00:37 GMT

In the past, I have an application that actually calls mpi_init which blocks till all the applications on the other nodes are also initialised. This is achieved by setting the PMI environment variables. Is this method also applicable for Intel MPI?

The previous implementation that I talked about is using MPICH 1.

Running mpi w/o mpiexec/mpirun

lordme — Fri, 11 Jun 2010 16:05:09 GMT

After loading an Intel MPI module file, compiler wrappers with prefix mpi are in your $PATH. The remaining name component of a compiler wrapper command specifies the underlying compiler to be used (mpigcc uses gcc, mpiifort uses ifort).
___________________

bands rings | swarovski crystal beads

Re: Running mpi w/o mpiexec/mpirun

jaewonj — Fri, 13 Aug 2010 01:56:34 GMT

Hello Gergana,

I haven't checked this thread for a while. Thanks for answering my question.

What I wanted to do was to run parallel libraries such as PETSc from within my application. As a quick and dirty workaround, I made a sperate executable and called it from my application using system command. I used binary file stream to communicate with that executable.

I think my question still stands. Do you think it is doable, I mean running MPI app without mpiexec?

Best Regards,
Jaewon

Re: Running mpi w/o mpiexec/mpirun

TimP — Fri, 13 Aug 2010 02:40:17 GMT

Quoting jaewonj

I think my question still stands. Do you think it is doable, I mean running MPI app without mpiexec?

This makes sense only in the context of future versions of MPI, where mpiexec may be implicit in an application which is linked against MPI. If you have available a system() function, and can set up the MPI environment, why not include mpiexec in the command you start under system()?

Re: Running mpi w/o mpiexec/mpirun

jaewonj — Fri, 13 Aug 2010 17:49:00 GMT

Thanks tim18. As you said it's possible to invoke mpiexec via system command but I don't like to use filestream.

Best regards,
Jaewon