s001-n189 doesn't have arria10 property but can still be returned by qsub with arria10 property

[PSath2]

Using the batch@v-qsvr-fpga queue, I request only those nodes with property "arria10" (i.e. n[137-139]), but can still end up on n189, which doesn't have the arria10 property. This requires the user to manually select a node that has the arria10 card installed and dev resources mounted, rather than using the queue as intended.

Log and initial output for pbsnodes below:

<userid>@login-1:~$ qsub -V -q batch@v-qsvr-fpga -l nodes=arria10:ppn=2  -I
qsub: waiting for job 1861.v-qsvr-fpga.aidevcloud to start
qsub: job 1861.v-qsvr-fpga.aidevcloud ready


########################################################################
#      Date:           Thu Nov 14 09:16:30 PST 2019
#    Job ID:           1861.v-qsvr-fpga.aidevcloud
#      User:           <userid>
# Resources:           neednodes=arria10:ppn=2,nodes=arria10:ppn=2,walltime=06:0                                     0:00
########################################################################

-bash: /opt/a10/intelrtestack/init_env_nosudo.sh: No such file or directory
INTELFPGAOCLSDKROOT is set to /glob/development-tools/versions/intelFPGA_pro/19.                                     3//hld. Using that.

Will use $QUARTUS_ROOTDIR_OVERRIDE= /glob/development-tools/versions/intelFPGA_p                                     ro/19.3//quartus  to find Quartus

Adding /glob/development-tools/versions/intelFPGA_pro/19.3//hld/bin to PATH
Adding /glob/development-tools/versions/intelFPGA_pro/19.3//hld/linux64/lib to L                                     D_LIBRARY_PATH
Adding /glob/development-tools/versions/intelFPGA_pro/19.3//hld/host/linux64/lib                                      to LD_LIBRARY_PATH
Adding /opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/opencl/opencl_bsp/linux64/li                                     b to LD_LIBRARY_PATH
ALTERAOCLSDKROOT=/glob/development-tools/versions/intelFPGA_pro/19.3//hld
<userid>@s001-n189:~$ exit
logout

qsub: job 1861.v-qsvr-fpga.aidevcloud completed
<userid>@login-1:~$ pbsnodes -s v-qsvr-fpga
s001-n137
     state = job-exclusive
     power_state = Running
     np = 2
     properties = xeon,gold6130,skl,ram192gb,net1gbe,fpga,arria10
     ntype = cluster
     jobs = 0-1/1860.v-qsvr-fpga.aidevcloud
     status = rectime=1573751801,macaddr=e4:43:4b:4c:a6:c8,cpuclock=Fixed,varatt                                     r=,jobs=1860.v-qsvr-fpga.aidevcloud(cput=0,energy_used=0,mem=89624kb,vmem=532644                                     kb,walltime=3164,Error_Path=/dev/pts/0,Output_Path=/dev/pts/0,session_id=61841),                                     state=free,netload=353136730090,gres=,loadave=0.15,ncpus=64,physmem=196664632kb,                                     availmem=197276504kb,totmem=198663476kb,idletime=284273,nusers=1,nsessions=1,ses                                     sions=61841,uname=Linux s001-n137 4.15.0-48-generic #51~16.04.1-Ubuntu SMP Fri A                                     pr 5 12:01:12 UTC 2019 x86_64,opsys=linux
     mom_service_port = 15002
     mom_manager_port = 15003

s001-n189
     state = free
     power_state = Running
     np = 2
     properties = xeon,gold6130,skl,ram192gb,net1gbe,fpga,darby
     ntype = cluster
     status = rectime=1573751799,macaddr=e4:43:4b:47:82:84,cpuclock=Fixed,varatt                                     r=,jobs=1861.v-qsvr-fpga.aidevcloud(cput=0,energy_used=0,mem=52756kb,vmem=327020                                     kb,walltime=2,Error_Path=/dev/pts/1,Output_Path=/dev/pts/1,session_id=89167),sta                                     te=free,netload=383350319614,gres=,loadave=0.09,ncpus=64,physmem=196462340kb,ava                                     ilmem=196518736kb,totmem=198559488kb,idletime=34407,nusers=1,nsessions=1,session                                     s=89167,uname=Linux s001-n189 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 30 14:19                                     :46 UTC 2019 x86_64,opsys=linux
     mom_service_port = 15002
     mom_manager_port = 15003
....

[JEYANTHKRI_N_Intel]

Thanks for reaching out to us.

We urge you to post this question in the following url (https://forums.intel.com/s/topic/0TO0P0000001AUUWA2/intel-high-level-des...) in order to get a faster response regarding FPGA .

[STyur]

Dear Paul!

If you have access to "v-qsvr-nda", try to use 

qsub -q batch@v-qsvr-nda -l nodes=1:fpga_compile:ppn=2

for compilation, and for execute on fpga:

qsub -q batch@v-qsvr-nda -l nodes=1:fpga_runtime:ppn=2

[PSath2]

Thanks for the suggestion Sergey, unfortunately I don't have access to that queue.

Jeyanth I am aware of the other forums but this bug is specific to the queuing system on the DevCloud environment, so I maintain that it is most appropriately asked in the DevCloud forum. (The fact that the property is related to FPGA is immaterial, the problem is with qsub/torque.)

[JEYANTHKRI_N_Intel]

Hi ,

We will contact the admin team and get back to you .

[JEYANTHKRI_N_Intel]

Hi ,

Looking at your queue name, it looks like you have gained access to FPGA Intel AI DevCloud (https://software.intel.com/en-us/devcloud/FPGA ). Please check "get help" link under the section "Support"
This is where you will get support for the mentioned issues. Or more specififcally, please post your queries in https://forums.intel.com/s/topic/0TO0P0000001AUUWA2/intel-high-level-design?language=en_US
Please note that even if the bug is specific to the queuing system on the DevCloud environment, any questions on FPGA DevCloud will be answered in the afore-mentioned forum only