Intel® MPI Library
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2153 Discussions

Running mpi w/o mpiexec/mpirun

jaewonj
Novice
2,032 Views
I can compile and run a mpi application using mpiexec without any difficulties.
What I want to do is to run a mpi application not in Terminal,I mean without using mpiexec or mpirun.
Is this doable? If it is, how can I setup mpd daemon and other necessary stuffs in source level?
Thanks. 

==========
hello.c
==========
#include 
#include 

static int run()
{
	int node;
	MPI_Init(xxx, xxx); /* TODO */
	MPI_Comm_rank(MPI_COMM_WORLD, &node);
	printf("Hello World from Node (%d)\n", node);
	MPI_Finalize();
	return 0;	
}

int main (int argc, char **argv)
{
run();
return 0;
}

13 Replies
Gergana_S_Intel
Employee
2,032 Views

Hi jaewonj,

That's an interesting use case. Do you mind sharing with us why you're looking to start the daemons within the source code?

I, personally, have only seen the start of the MPD daemons and MPI programs via scripts (be it shell or python) but not within compiled code. Perhaps others on this forum have a different experience. I know that with the Intel MPI Library, we don't provide any sort of system-level calls you can use.

Regards,
~Gergana

0 Kudos
TimP
Honored Contributor III
2,032 Views
Of course, it's a common mode of operation to run all of it from a script, either under a queuing system or directly from the user session.
0 Kudos
vastviewec
Beginner
2,032 Views

Hi jaewonj,

That's an interesting use case. Do you mind sharing with us why you're looking to start the daemons within the source code?

I, personally, have only seen the start of the MPD daemons and MPI programs via scripts (be it shell or python) but not within compiled code. Perhaps others on this forum have a different experience. I know that with the Intel MPI Library, we don't provide any sort of system-level calls you can use.

Regards,
~Gergana


Hello Gergana,

I can compile and run a program with INTEL MPI library and run with mpiexec without any difficulties. However, when I tried to run MPI programs via bash shell, I'm getting following error message

mpdroot: cannot connect to local mpd at: /tmp/mpd2.console_root
probable cause: no mpd daemon on this machine
possible cause: unix socket /tmp/mpd2.console_root has been removed
mpiexec_user-desktop (__init__ 1186): forked process failed; status=255


My test command in bash file is very simple

#!/bin/sh
mpdboot &
mpiexec -n 2 ./program.exe
mpdallexit


It seems to me that mpi daemon is not started correctly. Is there a speical way to call mpi in shell scripts?

Thanks very much!

JACKY
0 Kudos
Gergana_S_Intel
Employee
2,032 Views
Quoting - vastviewec
My test command in bash file is very simple

#!/bin/sh
mpdboot &
mpiexec -n 2 ./program.exe
mpdallexit


It seems to me that mpi daemon is not started correctly. Is there a speical way to call mpi in shell scripts?

Hi Jacky,

I would recommend that you amend your script as follows:

#!/bin/sh
mpdboot
mpiexec -n 2 ./program.exe
mpdallexit

Note the missing "&" after mpdboot. That's why the script was failing before: by putting mpdboot in the background, you're notwaiting forthe Intel MPI library to successfully start all MPDs, and *then* use those daemons to run your MPI program. In effect, it would be like running mpiexec with no mpdboot preceding it.

I hope this makes sense. Let me know how it goes and if you have further questions.

Regards,
~Gergana

0 Kudos
vastviewec
Beginner
2,032 Views

Hi Jacky,

I would recommend that you amend your script as follows:

#!/bin/sh
mpdboot
mpiexec -n 2 ./program.exe
mpdallexit

Note the missing "&" after mpdboot. That's why the script was failing before: by putting mpdboot in the background, you're notwaiting forthe Intel MPI library to successfully start all MPDs, and *then* use those daemons to run your MPI program. In effect, it would be like running mpiexec with no mpdboot preceding it.

I hope this makes sense. Let me know how it goes and if you have further questions.

Regards,
~Gergana


Thanks Gergana! The problem is solved by using your solustion!

Regards,

Jacky
0 Kudos
duzyatko2yahoo_com
2,032 Views
I think that it is a bit more convenient to run such programs via scripts.

___________
essay paper help
0 Kudos
jimmy82
Novice
2,032 Views

Previously in MPI implementation, there is a way to specify the environment variables and start each process individually without using MPIexec.

I wonder if anybody has been successful in doing it this way using Intel MPI? Certain applications that I require has to be invoked separately, so I will require such ways of starting each node.

Hope somebody can advise.

Gergana_S_Intel
Employee
2,032 Views

Hi jimmy,

Can you perhaps give an example of an MPI implementation that doesn't use mpiexec. A lot of the ones I know either use that, or mpirun to control the MPI processes and communication.

In terms of environment variables for Intel MPI Library, those can be set whenever you want - either before you start your MPI processes, or on the command line.

Regards,
~Gergana

0 Kudos
jimmy82
Novice
2,032 Views
In the past, I have an application that actually calls mpi_init which blocks till all the applications on the other nodes are also initialised. This is achieved by setting the PMI environment variables. Is this method also applicable for Intel MPI?

The previous implementation that I talked about is using MPICH 1.
0 Kudos
lordme
Beginner
2,032 Views
After loading an Intel MPI module file, compiler wrappers with prefix mpi are in your $PATH. The remaining name component of a compiler wrapper command specifies the underlying compiler to be used (mpigcc uses gcc, mpiifort uses ifort).
___________________

bands rings | swarovski crystal beads

0 Kudos
jaewonj
Novice
2,032 Views
Hello Gergana,

I haven't checked this thread for a while. Thanks for answering my question.

What I wanted to do was to run parallel libraries such as PETSc from within my application. As a quick and dirty workaround, I made a sperate executable and called it from my application using system command. I used binary file stream to communicate with that executable.

I think my question still stands. Do you think it is doable, I mean running MPI app without mpiexec?

Best Regards,
Jaewon
TimP
Honored Contributor III
2,032 Views
Quoting jaewonj

I think my question still stands. Do you think it is doable, I mean running MPI app without mpiexec?

This makes sense only in the context of future versions of MPI, where mpiexec may be implicit in an application which is linked against MPI. If you have available a system() function, and can set up the MPI environment, why not include mpiexec in the command you start under system()?
0 Kudos
jaewonj
Novice
2,032 Views
Thanks tim18. As you said it's possible to invoke mpiexec via system command but I don't like to use filestream.

Best regards,
Jaewon


0 Kudos
Reply