Unable to login into any computer node.

[black__edgar]

Hi all,

I was working on one of the computer nodes (idc-beta-batch-pvc-node-xx) and was suddenly logged off of it.

I am still able to login into the head node (idc-beta-batch-head-node), but every time I try a computer node I get the following message:

uxxxxx@idc-beta-batch-head-node:~$ srun -p pvc-shared --pty /usr/bin/bash

srun: error: Unable to allocate resources: Invalid account or account/partition combination specified

Is this happening just to me?

Any help will be welcomed.

Thanks.

[Megat_Intel]

Hi Black__edgar,

Thank you for reaching out to Intel® Communities.

This issue seems to be a slurm-related error regarding account or account/partition combination. Try to run the command "sinfo -al" to view the partition and node information available to you. Below are mine:

 

On the other hand, I got a similar error when I specified the wrong account information. Can you please try to provide the account information by using the "-A" option in srun? You can find your account information using the "whoami" command. I show my result here:

 

If the above method does not work, try to use "srun -p pvc-shared -w idc-beta-batch-pvc-node-XX --pty bash" to force yourself onto node XX (change XX to the previous node that you were able to connect). Do let us know if it works.

Regards,

Megat

[black__edgar]

Hi Megat,

Thank you for your reply.

I am going to post here the responses to each one of your suggestions.

+++++++++++++++++++++++++++++++++++++++++++++++++

u107157@idc-beta-batch-head-node:~$ sinfo -al
Wed Oct 11 05:19:08 2023
PARTITION AVAIL TIMELIMIT JOB_SIZE ROOT OVERSUBS GROUPS NODES STATE NODELIST
pvc-shared* up 4:00:00 1 no FORCE:20 all 19 allocated idc-beta-batch-pvc-node-[01-06,08-19,21]
pvc-shared* up 4:00:00 1 no FORCE:20 all 1 idle idc-beta-batch-pvc-node-20

u107157@idc-beta-batch-head-node:~$

+++++++++++++++++++++++++++++++++++++++++++++++++

+++++++++++++++++++++++++++++++++++++++++++++++++

u107157@idc-beta-batch-head-node:~$ whoami
u107157
u107157@idc-beta-batch-head-node:~$ srun -A u107157 -p pvc-shared --pty /usr/bin/bash
srun: error: Unable to allocate resources: Invalid account or account/partition combination specified

u107157@idc-beta-batch-head-node:~$

+++++++++++++++++++++++++++++++++++++++++++++++++

+++++++++++++++++++++++++++++++++++++++++++++++++

u107157@idc-beta-batch-head-node:~$ srun -p pvc-shared -w idc-beta-batch-pvc-node-20 --pty /usr/bin/bash
srun: error: Unable to allocate resources: Invalid account or account/partition combination specified
u107157@idc-beta-batch-head-node:~$

+++++++++++++++++++++++++++++++++++++++++++++++++

My problem is still there.

Thank you for helping me,

Edgar Black

[Megat_Intel]

Hi Edgar Black,

Thank you for the information provided.

We suspect a submitted job may have triggered the system and restricted the account. We are still checking into it to see if this is the case. Please let us know what was the last job you submitted and the Batch Service instance End Date.

On another note, a quick fix for this issue is to terminate the Batch Instance and launch a new one.

Regards,

Megat

[black__edgar]

Hi Megat,

I was running an example of a sycl program when I was suddenly kicked out of the computer node and back in the head node.

I am still unable to login into a computer node.

Thanks for the help,

Edgar

[Megat_Intel]

Hi Edgar,

We have informed the relevant team about this issue for further investigation and will update you as soon as possible.

In the meantime, please let us know the End Date of your Batch Service instance.

Regards,

Megat

[black__edgar]

Hi Megat,

I don't know how to get the End Date of your Batch Service instance.

Could you please help me find it?

Thanks,

Edgar

[Megat_Intel]

Hi Edgar,

You can check your instance Reservation details in the "View Instance" Tab on the https://scheduler.cloud.intel.com/#/systems page, or you can see the End Date when you log in to the head node. I show the examples below:

 

 

Regards,

Megat

[black__edgar]

Hi Megat,

My End Date of your Batch Service instance is:

         Your User Account expires on : Oct 29, 2023

I got that from the head node.

Thanks for your help,

Edgar

PD: I have no access to the "View Instance" Tab. That is related to a previous problem I had in which my account was terminated prematurely. I had no access to the head node and no access to the "View Instance" Tab. My access to the head node was established, but the problem with the "View Instance" Tab remains. I attached a video showing the problem with my access to the "View Instance" Tab.

Thanks again,

Edgar Black

[Megat_Intel]

Hi Edgar,

The best solution right now is to terminate the Batch Access Instance and launch back a new Intel® Max Series GPU (PVC) on 4th Gen Intel® Xeon® processors - 1100 series (4x) (Batch Processing/Scheduled access) Instance. You can terminate the instance in the "View Instance" tab.

When terminating the instance, all the files you created or uploaded will be deleted. Please ensure you have backup all the important data first before terminating the instance.

On the other hand, you mentioned before you are unable to go to the "View Instance" tab in the https://scheduler.cloud.intel.com/ page. Is it still the case now? If yes, please follow the steps below:

1. Go to your browser, refresh the page, and wait for 1-2 minutes.
2. Go to the Web Developer Tools (Ctrl + Shift + I), click on the Network tab.
3. Open the IDC Console (https://scheduler.cloud.intel.com/) and try to go to the "View Instance" tab.
4. If you are then redirected to the homepage, after that click on the settings button and click "Save all as HAR"
5. Download the HAR file.

Here is an example:

 

Please provide us with the HAR file for further investigation. You can send the HAR file to my email: megatx.muhammad.firdaus.bin.sahrir@intel.com

Regards,

Megat

[black__edgar]

Please, let me know if you got the HAR files.

[black__edgar]

Hi Megat,

For me, the problem with the "View Instance" tab persists. Therefore, I cannot terminate the Batch Access Instance and launch a new one.

I followed the steps to create the HAR file. I did it on two different computers with the same result.

I am sending those two files for further investigation to the provided email address.

Thank you very much for all the attention you have given to my problem.

Edgar Black

[Megat_Intel]

Hi Edgar Black,

Yes, I have received both the HAR files that you sent, Thanks.

We have escalated this issue to the relevant team for further investigation. We will get back to you as soon as we receive any updates. We thank you for your patience.

Regards,

Megat

[black__edgar]

Hi Megat,

Any news about my problem?

Thanks,

Edgar Black

[Zulkifli_Intel]

Hi black__edgar

Thank you for your patience.

Batch Mode has been removed from IDC Beta. Therefore, we suggest you use IDC 1.0 (cloud.intel.com)

1. In Console Home, under Quick Start click Training and Workshops
2. Click Launch Jupyter Lab (top right corner)

Do let us if you encounter any issues.

Sincerely,

Zulkifli

[black__edgar]

Hi Zulkifli,

The Jupyter Lab seems to be working. Thanks.

I noticed that I was assigned a new username and linux account.

Is there an easy way to transfer the files I had in my previous account to this new account?

Thanks again,

Edgar Black

[Megat_Intel]

Hi Edgar Black,

I'm glad that you are able to connect to the Jupyter Lab instance.

Yes, you are assigned a new instance for Jupyter Lab. Regarding file transfers, the easiest way to transfer files between instances is to download the files from your old instance and upload the files back to the new instance.

You can refer to the How to Upload File from Local System to Intel® Developer Cloud Instance page for more information on file transfers. Hope this helps.

Regards,

Megat

[black__edgar]

Hi Megat,

Thank you for the information.

I will try it as soon as I get my juniper working again. Today it has refused to start.

See the attached picture.

[black__edgar]

Hi Megat,

Are you sure that the information presented in the "How to Upload File from Local System to Intel® Developer Cloud Instance" page works in the jupyterLab?

[Megat_Intel]

Hi Edgar Black,

I apologize for the inconvenience.

To upload files to the JupyterLab instance, you can just click on the 'Upload Files' button or drag and drop the files to the instance as shown below:

On the other hand, since the ssh/fstp/scp are incoming only, you can use https in and out (e.g., git) to download the files. Hope this helps.

Regards,

Megat

[black__edgar]

Hi Megat,

Yes, after a short while, I noticed the upload button.

I am beginning to use the system again.

Thank you.