10.3 Suspend/Resume Handling
When supported by the resource manager, Moab can suspend and resume jobs. A user can suspend his/her own jobs, but only an administrator can resume them. By default, a job is suspended for one minute before it can resume. You can modify this default time using the MINADMINSTIME parameter.
A job must be marked as suspendable for Moab to suspend and resume it. To do so, either submit the job with the suspendable flag attached to it or configure a credential to pass the flag to its associated jobs. These methods are demonstrated in the examples below:
msub -l flags=suspendable
GROUPCFG[default] JOBFLAGS=SUSPENDABLE
Once the job is suspendable, Moab allows you to suspend jobs using the two following methods: (1) manually on the command line and (2) automatically in the moab.cfg file.
To manually suspend jobs, use the mjobctl command as demonstrated in the following examplee:
> mjobctl -s job05
Moab suspends job05, preventing it from running immediately in the job queue.
If you are an administrator and want to resume a job, use the mjobctl command as demonstrated in the following example:
> mjobctl -r job05
Moab removes job05 from a suspended state and allows it to run.
You can also configure the Moab preemption policy to suspend and resume jobs automatically by setting the PREEMPTPOLICY parameter to SUSPEND. A sample Moab configuration looks like this:
PREEMPTPOLICY SUSPEND ... USERCFG[tom] JOBFLAGS=SUSPENDABLE
Moab suspends jobs submitted by user tom if necessary to make resources available for jobs with higher priority.
If your resource manager has a native interface, you must configure JOBSUSPENDURL to suspend and resume jobs.
For more information about suspending and resuming jobs in Moab, see the following sections:
- manual preemption with the mjobctl command
- Job preemption