Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
1,119 Views

Job submitted via qsub getting killed.

When I submit jobs using "qsub" command, it gets terminated after some time. It seems to be due to walltime getting exceeded. How can we increase walltime so that job will get completed.

Below is the error log.

 

=>> PBS: job killed: walltime 21642 exceeded limit 21600

 

*** Aborted at 1532971520 (unix time) try "date -d @1532971520" if you are using GNU date ***

 

PC: @ 0x2b3084a63adb mkldnn::impl::cpu::ref_lrn_bwd_t<>::execute_backward()

 

*** SIGTERM (@0x6f4) received by PID 80152 (TID 0x2b3083449980) from PID 1780; stack trace: ***

 

@ 0x2b3089e0f5e0 (unknown)

 

@ 0x2b3084a63adb mkldnn::impl::cpu::ref_lrn_bwd_t<>::execute_backward()

 

@ 0x2b30898fbac3 __kmp_invoke_microtask

 

@ 0x2b30898ca257 __kmp_invoke_task_func

 

@ 0x2b30898cb498 __kmp_fork_call

 

@ 0x2b30898a27de __kmpc_fork_call

 

@ 0x2b3084a634ec mkldnn::impl::cpu::ref_lrn_bwd_t<>::execute_backwa

16364@c002-n003 ~]$ git status | more

 

fatal: Not a git repository (or any parent up to mount point /home)

 

Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

 

[u16364@c002-n003 ~]$ cd git_repo/cpp_crack_app/

 

[u16364@c002-n003 cpp_crack_app]$ git status | more

 

# On branch master

 

# Changes not staged for commit:

 

# (use "git add ..." to update what will be committed)

 

# (use "git checkout -- ..." to discard changes in working directory)

 

#

 

# modified: build/Makefile

 

# modified: src/crack.cpp

 

# modified: src/main.cpp

 

#
Tags (1)
0 Kudos
7 Replies
Highlighted
Beginner
4 Views

Re: Job submitted via qsub getting killed.

I have tried with "# PBS -l walltime=48:00:00" option. Still it didn't resolved.

Please do correct me if this option wrong.

0 Kudos
Highlighted
Community Manager
4 Views

Re: Job submitted via qsub getting killed.

Hi Joseph,

 

 

Thanks for reaching out to us.

 

 

You can increase the walltime by adding it in bash script before running in qsub mode as shown below:

 

# PBS -l walltime=24:00:00 

 

 

Then run "qsub "

 

 

Please note that the max walltime is 24 hours.

 

 

Hope this helps.

 

 

Regards,

 

Deepthi
0 Kudos
Highlighted
Beginner
4 Views

Re: Job submitted via qsub getting killed.

Will this work even with a job triggered via Jupyter terminal and extend the walltime to 24hours?

0 Kudos
Highlighted
Community Manager
4 Views

Re: Job submitted via qsub getting killed.

Hi Joseph,

For a Jupyter terminal, the maximum session time is 4 hrs.

We will run a qsub job that requires more time from the jupyter terminal and get back to you on the results.

Regards,

 

Deepthi
0 Kudos
Highlighted
Community Manager
4 Views

Re: Job submitted via qsub getting killed.

Hi Joseph,

It is observed that post the maximum session time (4 hrs), the log files are not generated. So it is not ideal to submit a qsub job via Jupyter terminal which takes more than the session time.

 

Hope that helps.

 

Regards,

 

Deepthi

 

0 Kudos
Highlighted
Community Manager
4 Views

Re: Job submitted via qsub getting killed.

Hi Joseph,

 

Could you please confirm if the queries got cleared.

 

 

Regards,

 

Deepthi Raj
0 Kudos
Highlighted
Community Manager
4 Views

Re: Job submitted via qsub getting killed.

Hi Joseph,

 

 

Hope the queries are cleared. We are closing this thread. Please raise a new thread if you are facing any other issues.

 

 

Regards,

 

Deepthi Raj.
0 Kudos