Login Node Versus Compute Node in Intel® DevCloud
- What is the difference between login node and compute node?
Login node uses a lightweight general-purpose processor. Compute node uses an Intel® Xeon® Gold 6128 processor that is capable of handling heavy workloads.
All the tasks that need extensive memory and compute resources have to be run on compute node, not on login node.
- How are the login/compute nodes placed in Intel® AI DevCloud?
The following diagram gives the overall architecture of Intel® AI DevCloud:
- Why can’t I run compute-intensive tasks on the login node?
Login node is lightweight and not capable of handling heavy workloads. It is primarily intended to save your data; hence, a limit is enforced on both memory and compute on login nodes. This ensures that a memory error is thrown if you try to run any heavy tasks on login node.
- How do I check to see whether I’m on login node or compute node?
Please review the following screenshots:
One has “n0xx” in the prompt while the other does not. Whenever there is “n0xx” after c009 in the prompt it indicates that you are on the compute node. When there is no “n0xx” in the prompt it means that you are on the login node.
- How do I run compute-intensive jobs in Intel AI DevCloud?
There are six different methods by which you can run jobs on the compute node of Intel AI DevCloud:
- Using qsub
When the user submits a job using “qsub <JOB_SCRIPT>” from the login node, the job waits in the queue till the scheduler picks it up and finally runs it on the compute node.
- Using qsub –I
The user runs the job in interactive mode using “qsub –I”. This creates a job with default settings and provides the user with a terminal from the compute node. The user can directly run the commands there.
- Using JupyterHub*
Navigate to https://hub.colfaxresearch.com. Give credentials and start the server. Create a new notebook and run the code from there.
- Using qsub from JupyterHub
Navigate to https://hub.colfaxresearch.com. Give credentials and start the server. Create a new notebook. Submit the job using the qsub command Details are available in the Welcome.ipynb file in the home folder of Intel AI DevCloud.
- Directly run from JupyterHub terminal
Navigate to https://hub.colfaxresearch.com. Give credentials and start the server. Start a new terminal (find details in the article JupyterHub Terminal Versus SSH Terminal). The JupyterHub terminal is that of a compute node. The user can directly run the commands of a compute-intensive job in the terminal.
- Using qsub from the JupyterHub terminal
Submit the job using “qsub <JOB_SCRIPT>” from the JupyterHub terminal. This submits the job, and the job waits in the queue until the scheduler picks it up and executes it on compute node.
- What is the qsub mode and the interactive qsub mode of job submission?
In qsub mode, jobs are created and submitted using the command “qsub <JOB_SCRIPT>” from the login node. “JOB_SCRIPT” contains the job commands.
In interactive qsub mode, a job is created using the command “qsub –I”. After running this command we get a new terminal from the compute node allocated for this job. The user can then run job commands directly in the terminal.
- How do I enter interactive qsub mode?
When using secure shell (SSH) to access Intel AI DevCloud using a PuTTY*/Linux* terminal, enter the login node first. To run a job in interactive qsub mode, type “qsub –I”. This creates a new job and gives a terminal from the compute node allocated for this job. See the following screenshot:
- When should I use qsub mode and when should I use interactive qsub mode?
qsub mode is suitable when you have a tested running code and just need to run it and store the results.
Interactive qsub mode is suitable when you are still in the process of creating a running code and expect errors and would like to fix it simultaneously, just as if you were doing it on a local machine.
- How do I check job status and logs in qsub mode versus interactive qsub mode?
In qsub mode, a user can submit the job, work on other things, and come back later to check updates. Job status can be checked with the qstat command and logs can be checked using the log file/qpeek command.
In interactive qsub mode, once the terminal has expired, the logs written while the job/command was running are deleted.
Link Copied
