Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud
1660 Discussions

Job exceeds queue resource limits error on job submission, devcloud_login says the FPGA is available

MajdiAS
Beginner
4,337 Views

Hello,

 

I am using Devcloud for FPGA, and when I check devcloud_login SNN, it says the Stratix 10 (not OneAPI) nodes are available, but when I submit the job, it cancels after a few seconds. I used the qsub command and it gives me this error: 

qsub: submit error (Job exceeds queue resource limits MSG=cannot locate feasible nodes (nodes file is empty, all systems are busy, or no nodes have the requested feature))

 

I have not changed my job submission resource requirements so I do not expect resources to be an issue.

 

I checked the pbsnodes command and only the s001 and s002 nodes seem to show up. My guess is that the devices are available but the host CPUs for those devices are busy.

Labels (1)
0 Kudos
1 Solution
AthiraM_Intel
Moderator
3,386 Views

Hi,

 

Thanks for the confirmation. Since there is very limited number of Stratix 10 PAC nodes, it is difficult to get free nodes.

 

We are able to use the Stratix 10 PAC nodes. Please find the below screenshots:

AthiraM_Intel_1-1678104380995.png

 

AthiraM_Intel_0-1678104365367.png

 

If there is no free nodes available, please try to access the nodes after some time.

 

Since the issue is resolved, we are closing this case. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

 

 

Thanks

 

View solution in original post

0 Kudos
10 Replies
AthiraM_Intel
Moderator
4,300 Views

Hi,


Thank you for posting in Intel Community.


We are able to reproduce your issue. We are checking on this internally, will let you know the updates soon.



Thanks


0 Kudos
AthiraM_Intel
Moderator
3,647 Views

Hi,


We are sorry for the delay.


qsub command needs to include the "-q batch@v-qsvr-fpga" in order to access the specific node requested. Please see an example below:


qsub -q batch@v-qsvr-fpga -I -l nodes=s005-n006:ppn=2


Please let us know if you face any further issue.



Thanks


0 Kudos
AthiraM_Intel
Moderator
3,610 Views

Hi,


We have not heard back from you. Could you please give us an update?



Thanks


0 Kudos
MajdiAS
Beginner
3,595 Views

Hello,

 

I just tried the command you gave me, it becomes stuck with the following message:

 

qsub: waiting for job xxxxx.v-qsvr-fpga.aidevcloud to start 

 

The node I selected was said to be available when checking 

pbsnodes -l free -s v-qsvr-fpga

 

0 Kudos
AthiraM_Intel
Moderator
3,555 Views

Hi,

 

Could you please try to access the node again. We are able to access node by using the below command:

 

qsub -q batch@v-qsvr-fpga -I -l nodes=<node number>:ppn=2

 

Please find the below screenshot:

AthiraM_Intel_0-1676884866864.png

 

If you face the same issue again, please let us know.

 

 

 

Thanks

 

0 Kudos
MajdiAS
Beginner
3,538 Views

Hello,

 

I am able to access the Arria 10 boards, including the example s005-n007 instance. I do not think I ever had a problem with Arria 10 batch submissions. I had to port my design over to be compatible with Arria 10, which eventually worked.

However, accessing the Stratix 10 instances still seems to be an issue. I tried logging in to s001-n142, a Stratix 10 OneAPI instance, as the non-OneAPI instances were not available. I was receiving the same message as my original post, despite being that instance being listed as available according to devcloud_login. I will check when the regular Stratix 10 instances are available.

 

Let me know if you are able to reproduce the issue with Stratix 10 (non and OneAPI variants).

 

Best,

Majdi

0 Kudos
AthiraM_Intel
Moderator
3,495 Views

Hi,

 

Could you please try to access Stratix 10 nodes now. We are able to access without any issue.

Please find the below screenshot:

AthiraM_Intel_0-1677239284325.png

 

If you are still facing the issue, please let us know.

 

 

Thanks

 

0 Kudos
AthiraM_Intel
Moderator
3,442 Views

Hi,


We have not heard back from you. Could you please give us an update?




Thanks


0 Kudos
MajdiAS
Beginner
3,399 Views

Hello,

 

I can confirm that I can log in to Stratix 10 OneAPI now, but I could not verify Stratix 10 PAC as they have not been available.

 

Best,

Majdi

0 Kudos
AthiraM_Intel
Moderator
3,387 Views

Hi,

 

Thanks for the confirmation. Since there is very limited number of Stratix 10 PAC nodes, it is difficult to get free nodes.

 

We are able to use the Stratix 10 PAC nodes. Please find the below screenshots:

AthiraM_Intel_1-1678104380995.png

 

AthiraM_Intel_0-1678104365367.png

 

If there is no free nodes available, please try to access the nodes after some time.

 

Since the issue is resolved, we are closing this case. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

 

 

Thanks

 

0 Kudos
Reply