This section breaks down what you need to do to set up preemption with QoS. It explains how to configure the moab.cfg file and contains information about setting PREEMPTEE and PREEMPTOR QFLAGS. For details, see:

• Understanding PREEMPTEE & PREEMPTOR flags
• Configuring moab.cfg
• Additional PREEMPTEE & PREEMPTOR information

Understanding PREEMPTEE & PREEMPTOR flags

Using QoS, you can specify preemption rules and control access to preemption privileges.

This allows you to increase system throughput, improve job response time for specific classes of jobs, or enable various political policies. You enable all policies by specifying some QoS credentials with the QFLAGS[link] PREEMPTEE, and others with PREEMPTOR.

PREEMPTEE
Description	Indicates that the job can be preempted by a higher-priority job.
Use	Use for lower-priority jobs that can be preempted.
Notes	This may delay some node actions. When reprovisioning, the system job may expire before the provision action occurs; while the action will still occur, the job will not show it.
Example	QOSCFG[test1] QFLAGS=PREEMPTEE MEMBERULIST=<user> PRIORITY=100

PREEMPTOR
Description	Indicates that the job should take priority and preempt any PREEMPTEE jobs.
Use	Use for jobs that need to take precedence over lower-priority jobs.
Notes	PREEMPTOR jobs, either queued or running, should always have a higher priority than PREEMPTEE jobs. When you configure job as a PREEMPTOR, you should also increase its priority (for details, see PREEMPTPRIOJOBSELECTWEIGHT and PREEMPTRTIMEWEIGHT).
Example	QOSCFG[test2] QFLAGS=PREEMPTOR MEMBERULIST=<user> PRIORITY=10000

QoS-based preemption only occurs when the following three conditions are satisfied:

• The PREEMPTOR job has the PREEMPTOR attribute set.
• The PREEMPTEE job has the PREEMPTEE attribute set.
• The PREEMPTOR job has a higher priority than the PREEMPTEE job.

In the following example, the test1 job is set as PREEMPTEE, while the test2 job is set as PREEMPTOR. This configuration will allow test2 to preempt test1. (Note that test2 has a much higher priority than test1.)

GUARANTEEDPREEMPTION TRUE
PREEMPTPOLICY <policy>

QOSCFG[test1] QFLAGS=PREEMPTEE MEMBERULIST=root PRIORITY=100 
QOSCFG[test2] QFLAGS=PREEMPTOR MEMBERULIST=root PRIORITY=10000

Configuring moab.cfg

To configure the moab.cfg file

This is an example of a moab.cfg file.

GUARANTEEDPREEMPTION TRUE
PREEMPTPOLICY <policy>

QOSCFG[test1] QFLAGS=PREEMPTEE MEMBERULIST=<user> PRIORITY=100 
QOSCFG[test2] QFLAGS=PREEMPTOR MEMBERULIST=<user> PRIORITY=10000

1. Set GUARANTEEDPREEMPTION to TRUE. (This locks the job on a node and keeps trying to preempt.)
2. Make sure that JOBNODEMATCHPOLICY is not set to EXACTNDODE.
3. If it is not already, set NODEACCESSPOLICY to SHARED.
4. Set the PREEMPTPOLCY policy type (for more information, see Choosing a PREEMPTPOLICY type).
5. Set up QFLAGS to mark jobs as PREEMPTEE (a lower-priority job that can be preempted by a higher-priority job), or as PREEMPTOR (a higher-priority job that can preempt a lower-priority job). As in the example:

QOSCFG[test1] QFLAGS=PREEMPTEE MEMBERULIST=<user> PRIORITY=100
QOSCFG[test2] QFLAGS=PREEMPTOR MEMBERULIST=<user> PRIORITY=10000

6. Make sure that the PREEMPTEE job has a lower priority than the PREEMPTOR job. As in the example:

QOSCFG[test1] QFLAGS=PREEMPTEE MEMBERULIST=<user> PRIORITY=100
QOSCFG[test2] QFLAGS=PREEMPTOR MEMBERULIST=<user> PRIORITY=10000

Additional PREEMPTEE & PREEMPTOR information

You can set the RESERVATIONPOLICY parameter to NEVER. With this configuration, PREEMPTEE jobs can start whenever idle resources become available. These jobs will be allowed to run until a PREEMPTOR job arrives, at which point the PREEMPTEE jobs are preempted, freeing the resource. This configuration allows near immediate resource access for the PREEMPTOR jobs. Using this approach, a cluster can maintain near 100% system utilization while still delivering excellent turnaround time to the most important jobs.

In environments where job checkpointing or job suspension incur significant overhead, you might want to constrain the rate at which job preemption is allowed. You can use the JOBPREEMPTMINACTIVETIME parameter to throttle job preemption. In essence, this parameter prevents a newly started or newly resumed job from being eligible for preemption until it has executed for a specified amount of time. Conversely, you can exclude jobs from preemption after they have run for a certain amount of time by using the JOBPREEMPTMAXACTIVETIME parameter.

See Also

• Managing preemption