How To: how-can-i-utilise-maximum-number-of-nodes-for-parallel-computing-of-jobs-in-intel-devcloud

[WILLIAM_H_Intel4]

QUESTION: 

How can I utilize maximum number of nodes for parallel computing of jobs in Intel DevCloud?

for i in range(1,101,1):

  !qsub myjob -F "0" -o ./my_log_output.txt -e ./my_log_errors.txt

_____

After initializing 100 jobs with the above code I checked the status of queue using "qstat". The result shows that only 20 are running at any given instant and the remaining are in queue, whereas the "pbsnodes" command is outputting 136 nodes.

How can I utilise all these 136 nodes in parallel?

_____

!qstat -a

v-qsvr-1.aidevcloud:

Req'd Req'd Elap

Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time

----------------------- ----------- -------- ---------------- ------ ----- ------ --------- --------- - ---------

298629.v-qsvr-1.aidevc ***** jupyterh jupyterhub-singl 18053 1 1 94gb 04:02:00 R 02:06:30

299845.v-qsvr-1.aidevc ***** batch myjob 1608 1 2 -- 24:00:00 R 00:00:37

299846.v-qsvr-1.aidevc ***** batch myjob 9271 1 2 -- 24:00:00 R 00:00:36

299847.v-qsvr-1.aidevc ***** batch myjob 359 1 2 -- 24:00:00 R 00:00:35

299848.v-qsvr-1.aidevc ***** batch myjob 17650 1 2 -- 24:00:00 R 00:00:33

299849.v-qsvr-1.aidevc ***** batch myjob 361 1 2 -- 24:00:00 R 00:00:32

299850.v-qsvr-1.aidevc ***** batch myjob 14630 1 2 -- 24:00:00 R 00:00:31

299851.v-qsvr-1.aidevc ***** batch myjob 29911 1 2 -- 24:00:00 R 00:00:29

299852.v-qsvr-1.aidevc ***** batch myjob 11618 1 2 -- 24:00:00 R 00:00:28

299853.v-qsvr-1.aidevc ***** batch myjob 19202 1 2 -- 24:00:00 R 00:00:26

299854.v-qsvr-1.aidevc ***** batch myjob 29068 1 2 -- 24:00:00 R 00:00:25

299855.v-qsvr-1.aidevc ***** batch myjob 4379 1 2 -- 24:00:00 R 00:00:24

299856.v-qsvr-1.aidevc ***** batch myjob 1326 1 2 -- 24:00:00 R 00:00:23

299857.v-qsvr-1.aidevc ***** batch myjob 31874 1 2 -- 24:00:00 R 00:00:21

299858.v-qsvr-1.aidevc ***** batch myjob 15858 1 2 -- 24:00:00 R 00:00:20

299859.v-qsvr-1.aidevc ***** batch myjob 23915 1 2 -- 24:00:00 R 00:00:19

299860.v-qsvr-1.aidevc ***** batch myjob 12509 1 2 -- 24:00:00 R 00:00:18

299861.v-qsvr-1.aidevc ***** batch myjob 9314 1 2 -- 24:00:00 R 00:00:16

299862.v-qsvr-1.aidevc ***** batch myjob 7923 1 2 -- 24:00:00 R 00:00:12

299863.v-qsvr-1.aidevc ***** batch myjob 19655 1 2 -- 24:00:00 R 00:00:10

299864.v-qsvr-1.aidevc ***** batch myjob 26290 1 2 -- 24:00:00 R 00:00:01

299865.v-qsvr-1.aidevc ***** batch myjob -- 1 2 -- 24:00:00 Q --

299866.v-qsvr-1.aidevc ***** batch myjob -- 1 2 -- 24:00:00 Q --

299867.v-qsvr-1.aidevc ***** batch myjob -- 1 2 -- 24:00:00 Q --

299868.v-qsvr-1.aidevc ***** batch myjob -- 1 2 -- 24:00:00 Q --

299869.v-qsvr-1.aidevc ***** batch myjob -- 1 2 -- 24:00:00 Q --

ANSWER:

In order to maintain a fair utilization for all those who works in Devcloud , there is a restriction in the max number nodes that can be availed by a single user.