qsub job queue seems stuck

[Lan]

Hi,

When I using routing commad "qsub -I" to dispatch a new job today (1/29), the working node is not started as usual.

By using "qstat -Q" command, it returns as follow, please check, thanks!

Queue Max Tot Ena Str Que Run Hld Wat Trn Ext T Cpt
---------------- --- ---- -- -- --- --- --- --- --- --- - ---
flex 30 0 yes yes 0 0 0 0 0 0 E 0
extended 0 0 yes yes 0 0 0 0 0 0 E 0
temp 2 0 yes yes 0 0 0 0 0 0 E 0
jupyterhub 0 0 yes yes 0 0 0 0 0 0 E 0
batch 200 67 yes yes 64 2 1 0 0 0 E 0

[xqyint]

Same problem.

[Thyme]

Hi,

Same problem, job queue seems to be stuck.

Maybe an outage or maintenance? Is there any way to know about server status?

Jupyter session also constantly returns "Spawn failed: Timeout"

[Thermit]

The command pbsnodes would check the status of servers.

Some servers are online, but my jobs are also stucked.

[joshdumo]

Facing the same issue for the last 9 hrs.

[Vincent8]

Same problem here, jobs submitted will not run and i cannot start up the server for jyupter devcloud.

[Christoph9]

I also have this problem. In my case I try to get a job running on a Stratix10 node. qstat -Q shows me that there are 110 jobs queued in total but only 3 running

[venovako]

Same here, with interactive jobs on the "spr" and the "icx" nodes, and batch jobs on the former.

[AthiraM_Intel]

Hi,

Thank you for posting in Intel Communities.

We are sorry for the inconvenience caused. We are working on this issue internally. We will let you know the updates soon.

Thanks

[AthiraM_Intel]

Hi,

The issue is resolved now. Could you please check from your end whether the issue resolved for you and let us know.

We are sorry for the inconvenience caused.

Thanks

[Thermit]

It's Ok now.

[joshdumo]

I could run jobs.

Thanks for resolving.

[venovako]

Thanks, it works now.

[AthiraM_Intel]

Hi,

Glad to know that the issue is resolved. The issue is resolved and we have verified the same from our end also.

If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks.