I'm afaid I'm not sure what you mean. Could you clarify why you don't want users to start the mpdboot command? I suppose you can change the permissions on the Intel MPI Library install directory (e.g. /opt/intel/impi/) so that no user would be able to access any of the commands.
Or do you mean that you want to kill an mpdboot command that's already running? You can do a simple Ctrl+C, although that might leave some zombie processes running around. The mpdallexit command would kill any mpd daemons started from mpdboot for the given user. Or you can kill an actual MPI job via mpdkilljob. More info is available in the Intel MPI Library Reference Manual.
Or perhaps I misunderstood your question. Let me know if that's the case.
Ah, I see! Thanks for clarifying, Tim.
In this case, I guess changing the permissions would work. Although, any good job schedulershould be able to restrict access to the head node when jobs are submitted. Anish, what type of job scheduler are you using on the cluster?
In reality, what you want is not to prevent mpdboot from running on the head node, but to restrict users from running MPI jobs on the head node. Those are two different things. The mpdboot command would start the mpd daemons on all nodes you will be using, but mpiexec is the command used to run the MPI job. And it's not necessary that all nodes which have mpd daemons started be used for the MPI job.
This is easily done via the -nolocal option for the mpiexec/mpirun command which will tell the Intel MPI Library to exclude the current node when running the job. To make this a global setting for all Intel MPI jobs run on the cluster by all users, you can create an
$ cat /opt/intel/impi/3.2.2/etc64/mpiexec.conf
More details are in the Reference Manual, as noted in my earlier post. I hope this solves your problem. Let us know how it helps.
Indeed, you are correct in your 2nd case: the -nolocal option will always exclude the node from where the mpiexec command is started. So, if you're *not* starting from the head node, then the head node will *not* be excluded.
Another suggestion I can make is using a machine file to explicitly specify where you want your job to run. In fact, there're several ways to specify which nodes to include, but no easy way to exclude a specific node. And, when you think about it, it's not really the purpose of the MPI implementation to do job scheduling. That's better left to the professional job schedulers out there :)
These are the options that I see for you right now:
That's all I can think of right now. Does anyone else have other ideas?