Login Node Versus Compute Node in Intel® DevCloud
Login node uses a lightweight general-purpose processor. Compute node uses an Intel® Xeon® Gold 6128 processor that is capable of handling heavy workloads.
All the tasks that need extensive memory and compute resources have to be run on compute node, not on login node.
The following diagram gives the overall architecture of Intel® AI DevCloud:
Login node is lightweight and not capable of handling heavy workloads. It is primarily intended to save your data; hence, a limit is enforced on both memory and compute on login nodes. This ensures that a memory error is thrown if you try to run any heavy tasks on login node.
Please review the following screenshots:
One has “n0xx” in the prompt while the other does not. Whenever there is “n0xx” after c009 in the prompt it indicates that you are on the compute node. When there is no “n0xx” in the prompt it means that you are on the login node.
There are six different methods by which you can run jobs on the compute node of Intel AI DevCloud:
When the user submits a job using “qsub <JOB_SCRIPT>” from the login node, the job waits in the queue till the scheduler picks it up and finally runs it on the compute node.
The user runs the job in interactive mode using “qsub –I”. This creates a job with default settings and provides the user with a terminal from the compute node. The user can directly run the commands there.
Navigate to https://hub.colfaxresearch.com. Give credentials and start the server. Create a new notebook and run the code from there.
Navigate to https://hub.colfaxresearch.com. Give credentials and start the server. Create a new notebook. Submit the job using the qsub command Details are available in the Welcome.ipynb file in the home folder of Intel AI DevCloud.
Navigate to https://hub.colfaxresearch.com. Give credentials and start the server. Start a new terminal (find details in the article JupyterHub Terminal Versus SSH Terminal). The JupyterHub terminal is that of a compute node. The user can directly run the commands of a compute-intensive job in the terminal.
Submit the job using “qsub <JOB_SCRIPT>” from the JupyterHub terminal. This submits the job, and the job waits in the queue until the scheduler picks it up and executes it on compute node.
In qsub mode, jobs are created and submitted using the command “qsub <JOB_SCRIPT>” from the login node. “JOB_SCRIPT” contains the job commands.
In interactive qsub mode, a job is created using the command “qsub –I”. After running this command we get a new terminal from the compute node allocated for this job. The user can then run job commands directly in the terminal.
When using secure shell (SSH) to access Intel AI DevCloud using a PuTTY*/Linux* terminal, enter the login node first. To run a job in interactive qsub mode, type “qsub –I”. This creates a new job and gives a terminal from the compute node allocated for this job. See the following screenshot:
qsub mode is suitable when you have a tested running code and just need to run it and store the results.
Interactive qsub mode is suitable when you are still in the process of creating a running code and expect errors and would like to fix it simultaneously, just as if you were doing it on a local machine.
In qsub mode, a user can submit the job, work on other things, and come back later to check updates. Job status can be checked with the qstat command and logs can be checked using the log file/qpeek command.
In interactive qsub mode, once the terminal has expired, the logs written while the job/command was running are deleted.