21.3 Using Owner Preemption
Context
Owner preemption allows jobs submitted by a reservation owner to preempt jobs submitted by other users (for more information, see Configuring and Managing Reservations).
Owner preemption is enabled with the OWNERPREEMPT reservation flag.
For information about PREEPMPTEE and PREEMPTOR flags, see Preemption Flags
To enable owner preemption
- Make the following configurations to the
- Set GUARANTEEDPREEMPTION to TRUE. (This causes Moab to lock PREEMPTOR jobs until JOBRETRYTIME expires.)
- Make sure that JOBNODEMATCHPOLICY is not set to EXACTNODE, which is not currently supported for preemption (for more information, see Testing and Troubleshooting Preemption).
- Set the PREEMPTPOLICY type (for more information, see PREEMPTPOLICY Types).
- Set the OWNERPREEMPT flag.
- Specify an owner.
Optionally, if you want the owner preemption to override any PREEMPTMINTIME settings for PREEMPTEE jobs, you can set the OWNERPREEMPTIGNOREMINTIME flag as well.
If the non-owner job does not have a RESTARTABLE or REQUEUEABLE flag set, the job will cancel.
- Submit a job to a user who is not the owner (in this example, jane).
- Now submit a job for the owner (in this example, john).
For example:
GUARANTEEDPREEMPTION TRUE PREEMPTPOLICY <policy> SRCFG[myrez] FLAGS=OWNERPREEMPT HOSTLIST=node01 SRCFG[myrez] OWNER=USER:john SRCFG[myrez] USERLIST=jane,john PERIOD=INFINITY QOSCFG[test1] QFLAGS=PREEMPTEE JOBFLAGS=restartable MEMBERULIST=john PRIORITY=100 QOSCFG[test2] QFLAGS=PREEMPTOR MEMBERULIST=john PRIORITY=10000
[jane@g06]$ echo sleep 600 | msub -l walltime=600 -l procs=64
(Optional) Examine the following output for showq and checkjob for jane's job:
Moab.1
[jane@g06]$ showq
active jobs------------------------
JOBID USERNAME STATE PROCS REMAINING STARTTIME
Moab.1 jane Running 64 00:09:57 Mon Nov 14 12:07:52
1 active job 64 of 64 processors in use by local jobs (100.00%)
1 of 1 nodes active (100.00%)
eligible jobs----------------------
JOBID USERNAME STATE PROCS WCLIMIT QUEUETIME
0 eligible jobs
blocked jobs-----------------------
JOBID USERNAME STATE PROCS WCLIMIT QUEUETIME
0 blocked jobs
Total job: 1
root@g06]# checkjob Moab.1 job Moab.1 State: Running Creds: user:jane group:jane WallTime: 00:01:02 of 00:10:00 SubmitTime: Mon Nov 14 12:07:52 (Time Queued Total: 00:00:00 Eligible: 00:00:00) StartTime: Mon Nov 14 12:07:52 Total Requested Tasks: 64 Req[0] TaskCount: 64 Partition: FLEXlm NodeCount: 1 Allocated Nodes: [node01:64] IWD: /opt/native SubmitDir: /opt/native Executable: /opt/native/spool/moab.job.FoZfIU StartCount: 1 Flags: GLOBALQUEUE,PROCSPECIFIED StartPriority: 1 Reservation 'Moab.1' (-00:01:24 -> 00:08:36 Duration: 00:10:00)
[john@g06]$ echo sleep 600 | msub -l walltime=600 -l procs=50
[john@g06]$ echo sleep 600 | msub -l walltime=600 -l procs=50
(Optional) Examine the following output for showq and checkjob for john's job:
Moab.2
[john@g06]$ showq
active jobs------------------------
JOBID USERNAME STATE PROCS REMAINING STARTTIME
Moab.1 jane Canceling 64 00:07:43 Mon Nov 14 12:07:52
Moab.2 john Running 50 00:09:59 Mon Nov 14 12:10:08
2 active jobs 64 of 64 processors in use by local jobs (100.00%)
1 of 1 nodes active (100.00%)
eligible jobs----------------------
JOBID USERNAME STATE PROCS WCLIMIT QUEUETIME
0 eligible jobs
blocked jobs-----------------------
JOBID USERNAME STATE PROCS WCLIMIT QUEUETIME
0 blocked jobs
Total jobs: 2
Note that jane's job is canceled once john's job is submitted.
[john@g06]$ checkjob Moab.2
job Moab.2
State: Running
Creds: user:john group:john
WallTime: 00:00:31 of 00:10:00
SubmitTime: Mon Nov 14 12:10:08
(Time Queued Total: 00:00:00 Eligible: 00:00:00)
StartTime: Mon Nov 14 12:10:08
Total Requested Tasks: 50
Req[0] TaskCount: 50 Partition: FLEXlm
NodeCount: 1
Allocated Nodes:
[node01:50]
IWD: /opt/native
SubmitDir: /opt/native
Executable: /opt/native/spool/moab.job.jf1N4a
StartCount: 1
Flags: HASPREEMPTED,GLOBALQUEUE,PROCSPECIFIED
StartPriority: 1
Reservation 'Moab.2' (-00:00:48 -> 00:09:12 Duration: 00:10:00)
Note the new HASPREEMPTED flag.
(Optional) Now look at the showq for jane's job (after):
[root@g06]# checkjob Moab.1
job Moab.1
State: Removed
Completion Code: -1 Time: Mon Nov 14 12:10:08
Creds: user:jane group:jane
WallTime: 00:02:47 of 00:10:00
SubmitTime: Mon Nov 14 12:07:52
(Time Queued Total: 00:00:00 Eligible: 00:00:00)
Total Requested Tasks: 64
Req[0] TaskCount: 64 Partition: FLEXlm
NodeCount: 1
Allocated Nodes:
[node01:64]
IWD: /opt/native
Executable: /opt/native/spool/moab.job.FoZfIU
Execution Partition: FLEXlm
Flags: GLOBALQUEUE,PROCSPECIFIED
StartPriority: 0
Note that the state is now Removed.
Related Topics