topic Re: Please explain the sequential usage of mpirun. How to get the names of involved nodes? in Intel® oneAPI Math Kernel Library

Please explain the sequential usage of mpirun. How to get the names of involved nodes?

AjayBadita — Sat, 21 Sep 2019 00:42:11 GMT

_________________

mpirun -n 4 python addNumbers.py &

sleep 10

_________________

Without finishing the first instance of mpirun, I initialised another mpirun. I am confused weather the second mpirun makes use of new compute nodes or the old nodes?

How to get the names of involved nodes? (For e.g. s001-n036, s001-n035, etc.). I want the total 12 nodes (4+8) to be different.

Also please note that I used "&" intentionally at the end of first mpirun.

Re: Please explain the sequential usage of mpirun. How to get the names of involved nodes?

Aswathy_C_Intel — Mon, 23 Sep 2019 19:59:46 GMT

Thanks for reaching out to us.

If you have 2 commands in a single job file, second one will run on the same nodes as of the first one only after its completion.

Please note that '-n' specified in mpirun command is the number of processes not the number of nodes. Also make sure that your code should be written in such a way to utilize mpirun. Or the code will run as many times as specified without any use.

To get the node names, you can write the below line in the job file.

cat $PBS_NODEFILE

Hope this helps.

Re: Please explain the sequential usage of mpirun. How to get the names of involved nodes?

Aswathy_C_Intel — Wed, 25 Sep 2019 15:49:40 GMT

Could you please confirm whether the details provided was helpful?

Please be informed that the thread will get closed within 2 business days assuming that the solution provided was helpful.

Re: Please explain the sequential usage of mpirun. How to get the names of involved nodes?

AjayBadita — Thu, 26 Sep 2019 00:42:41 GMT

I'm busy doing some important thing please don't close this thread. I will inform you shortly.

Re: Please explain the sequential usage of mpirun. How to get the names of involved nodes?

Aswathy_C_Intel — Thu, 26 Sep 2019 12:07:16 GMT

Could please tell us a time frame for checking on this?

If it takes more than a week, we suggest you open a new thread.

Please confirm.

Re: Please explain the sequential usage of mpirun. How to get the names of involved nodes?

AjayBadita — Thu, 26 Sep 2019 12:36:35 GMT

I need two days. Please don't close this thread.

Re: Please explain the sequential usage of mpirun. How to get the names of involved nodes?

Aswathy_C_Intel — Thu, 26 Sep 2019 12:45:12 GMT

Sure. Will wait.

Re: Please explain the sequential usage of mpirun. How to get the names of involved nodes?

AjayBadita — Sun, 29 Sep 2019 18:37:26 GMT

--------------------------------------distrJob1--------------------------------------

#PBS -l nodes=2:ppn=2

cd $PBS_O_WORKDIR

mpirun -n 4 python addNumbers.py

cat $PBS_NODEFILE

--------------------------------------distrJob1--------------------------------------

--------------------------------------distrJob2--------------------------------------

#PBS -l nodes=4:ppn=2

cd $PBS_O_WORKDIR

mpirun -n 8 python addNumbers.py

cat $PBS_NODEFILE

--------------------------------------distrJob2--------------------------------------

--------------------------------------scriptForMPI--------------------------------------

n_0 = 4

n_1=(12-$n_0)

t_1 = 10

qsub distrJob1 &

sleep $t_1

qsub distrJob2

--------------------------------------scriptForMPI--------------------------------------

I started with the script named scriptForMPI and submitting two jobs distrJob1 and distrJob2. My objective is to start n_0=4 processes at once (at zeroth second) using MPI and wait for t_1=10 seconds before starting distrJob2 on another n_1=8 processes. Note that I used "&" at the end of qsub distrJob1 in scriptForMPI to make sure that I don't want to wait for first MPI instance to finish.

scriptForMPI created two jobs with id's distrJob1.356789, distrJob2.356790. The corresponding output and error files are as follows.

------------------------

distrJob1.e356789

[2] DAPL startup: RLIMIT_MEMLOCK too small

[3] DAPL startup: RLIMIT_MEMLOCK too small

------------------------

distrJob1.o356789

########################################################################

# Date: Sun Sep 29 04:27:08 PDT 2019

# Job ID: 356789.v-qsvr-1.aidevcloud

# User: uXXXX

# Resources: neednodes=2:ppn=2,nodes=2:ppn=2,walltime=06:00:00

########################################################################

s001-n093

s001-n097

########################################################################

# End of output for job 356789.v-qsvr-1.aidevcloud

# Date: Sun Sep 29 04:27:15 PDT 2019

########################################################################

------------------------

distrJob2.e356790

------------------------

distrJob2.o356790

########################################################################

# Date: Sun Sep 29 04:27:20 PDT 2019

# Job ID: 356790.v-qsvr-1.aidevcloud

# User: uXXXX

# Resources: neednodes=4:ppn=2,nodes=4:ppn=2,walltime=06:00:00

########################################################################

===================================================================================