The CANCEL attribute for PREEMPTPOLICY cancels active jobs. If the PREEMPTPOLICY is set to CANCEL, then preempted jobs will cancel, regardless of any JOBFLAGS (such as REQUEUEABLE or SUSPENDABLE).

You should not allow preemption with interactive jobs unless PREEMPTPOLICY is set to CANCEL.

The following outlines some benefits of using CANCEL and also lists some things you should be aware of if you choose to use it.

Advantages:

• This attribute is the easiest to configure and use.

Cautions:

• Canceled jobs are not automatically restarted or requeued. Users must resubmit canceled jobs.

To use CANCEL

The following steps explain and illustrate how to set up preemption with CANCEL.

1. Make the following configurations to the moab.cfg file:

GUARANTEEDPREEMPTION TRUE
 
PREEMPTPOLICY CANCEL
 
QOSCFG[test1] QFLAGS=PREEMPTEE MEMBERULIST=john PRIORITY=100 
QOSCFG[test2] QFLAGS=PREEMPTOR MEMBERULIST=john PRIORITY=10000
						

• Set GUARANTEEDPREEMPTION to TRUE. (This locks the job on a node and keeps trying to preempt.)
• Make sure that JOBNODEMATCHPOLICY is not set to EXACTNDODE.
• Set PREEMPTPOLCY to CANCEL.

PREEMPTPOLICY CANCEL

• Make sure that the PREEMPTEE job has a lower priority than the PREEMPTOR job.

QOSCFG[test1] QFLAGS=PREEMPTEE MEMBERULIST=john PRIORITY=100
QOSCFG[test2] QFLAGS=PREEMPTOR MEMBERULIST=john PRIORITY=10000

2. Submit a job to the PREEMPTEE QoS (test1). For example:

[john@g06]# echo sleep 600 | msub -l walltime=600 -l qos=test1 -l procs=128

Examine the following output for showq:

Moab.7
[john@g06]# showq
 
active jobs------------------------ 
JOBID     USERNAME    STATE      PROCS    REMAINING    STARTTIME 
Moab.7    john        Running    128      00:01:59     Thu Nov 10 12:28:44
 
1 active job     128 of 128 processors in use by local jobs (100.00%) 
2 of 2 nodes active (100.00%)
 
eligible jobs---------------------- 
JOBID     USERNAME    STATE     PROCS      WCLIMIT      QUEUETIME
 
0 eligible jobs
 
blocked jobs----------------------- 
JOBID     USERNAME    STATE     PROCS      WCLIMIT      QUEUETIME
 
0 blocked jobs
 
Total job: 1

3. Now submit a job to the PREEMPTOR QoS (test2). For example:

[john@g06]$ echo sleep 120 | msub -l procs=128,walltime=120 -l qos=test2

Examine the following output for showq:

Moab.8
[john@g06]# showq
 
active jobs------------------------
JOBID     USERNAME    STATE      PROCS    REMAINING    STARTTIME
Moab.7    john        Canceling  128      00:01:56     Thu Nov 10 12:28:44
Moab.8    john        Running    128      00:02:00     Thu Nov 10 12:28:48
 
2 active jobs 128 of 128 processors in use by local jobs (100.00%)
2 of 2 nodes active (100.00%)
 
eligible jobs---------------------- 
JOBID     USERNAME    STATE     PROCS      WCLIMIT      QUEUETIME
 
0 eligible jobs
 
blocked jobs----------------------- 
JOBID     USERNAME    STATE     PROCS      WCLIMIT      QUEUETIME
 
0 blocked jobs
 
Total jobs: 2

Note that test1 is canceled when test2 is submitted.

4. Examine the checkjob outputs for these two jobs:

checkjob test1:

[john@g06]$ checkjob Moab.9 
job Moab.9
 
State: Removed
Completeion Code: -1 Time: Thu Nov 10 12:28:48 
Creds: user:john group:john qos:test1 
WallTime: 00:00:02 of 00:02:00 
SubmitTime: Thu Nov 10 12:28:44 
(Time Queued Total: 00:00:07 Eligible: 00:00:00)
 
Total Requested Tasks: 128
 
Req[0] TaskCount: 128 Partition: licenses 
NodeCount: 2
 
Allocated Nodes: 
node[01-02]*64
 
 
IWD: /opt/native 
SubmitDir: /opt/native 
Executable: /opt/native/spool/moab.job.UFe8sQ
 
StartCount: 1 
Flags: GLOBALQUEUE,PROCSPECIFIED 
Attr: PREEMPTEE 
StartPriority: 100 

Note: State: Removed

checkjob test2:

[john@g06]$ checkjob Moab.10 
job Moab.10
 
State: Running 
Creds: user:john group:john qos:test2 
WallTime: 00:00:00 of 00:02:00 
SubmitTime: Thu Nov 10 12:36:31 
(Time Queued Total: 00:00:00 Eligible: 00:00:00)
 
StartTime: Thu Nov 10 12:28:48 
Total Requested Tasks: 128
 
Req[0] TaskCount: 128 Partition: licenses
 
Allocated Nodes: 
node[01-02]*64
 
 
IWD: /opt/native 
SubmitDir: /opt/native 
Executable: /opt/native/spool/moab.job.CZavjU
 
StartCount: 1 
Flags: HASPREEMPTED,PREEMPTOR,GLOBALQUEUE,PROCSPECIFIED 
StartPriority: 10000 
Reservation 'Moab.10' (-00:00:07 -> 00:01:53 Duration: 00:02:00)

See Also

• Choosing a PREEMPTPOLICY type