When I submit jobs using "qsub" command, it gets terminated after some time. It seems to be due to walltime getting exceeded. How can we increase walltime so that job will get completed.
Below is the error log.
=>> PBS: job killed: walltime 21642 exceeded limit 21600
*** Aborted at 1532971520 (unix time) try "date -d @1532971520" if you are using GNU date ***
PC: @ 0x2b3084a63adb mkldnn::impl::cpu::ref_lrn_bwd_t<>::execute_backward()
*** SIGTERM (@0x6f4) received by PID 80152 (TID 0x2b3083449980) from PID 1780; stack trace: ***
@ 0x2b3089e0f5e0 (unknown)
@ 0x2b3084a63adb mkldnn::impl::cpu::ref_lrn_bwd_t<>::execute_backward()
@ 0x2b30898fbac3 __kmp_invoke_microtask
@ 0x2b30898ca257 __kmp_invoke_task_func
@ 0x2b30898cb498 __kmp_fork_call
@ 0x2b30898a27de __kmpc_fork_call
@ 0x2b3084a634ec mkldnn::impl::cpu::ref_lrn_bwd_t<>::execute_backwa
16364@c002-n003 ~]$ git status | more
fatal: Not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
[u16364@c002-n003 ~]$ cd git_repo/cpp_crack_app/
[u16364@c002-n003 cpp_crack_app]$ git status | more
# On branch master
# Changes not staged for commit:
# (use "git add ..." to update what will be committed)
# (use "git checkout -- ..." to discard changes in working directory)
# modified: build/Makefile
# modified: src/crack.cpp
# modified: src/main.cpp
Thanks for reaching out to us.
You can increase the walltime by adding it in bash script before running in qsub mode as shown below:
# PBS -l walltime=24:00:00
Then run "qsub "
Please note that the max walltime is 24 hours.
Hope this helps.
For a Jupyter terminal, the maximum session time is 4 hrs.
We will run a qsub job that requires more time from the jupyter terminal and get back to you on the results.
It is observed that post the maximum session time (4 hrs), the log files are not generated. So it is not ideal to submit a qsub job via Jupyter terminal which takes more than the session time.
Hope that helps.