You are here: Appendices > Appenidx X SCHEDCFG Flags

Appendix X SCHEDCFG flags

Flag Description
AGGREGATENODEFEATURES

AGGREGATENODEFEATURES causes Moab to aggregate features reported by the different RMs. For example, if you have two RMs reporting different features for the same node, Moab will add both features together (instead of one being overwritten by the other).

In order to set features manually, you can use mnodectl -m features (for details, see mnodectl).

ALLOWCREDENTIALSWITHSPACES ALLOWCREDENTIALSWITHSPACES lets Moab ignore POSIX standards and allows groups, users, and accounts with spaces in their names.
ALLOWINFINITEJOBS ALLOWINFINITEJOBS allows infinite wallclock times to be accepted. Previously, jobs with infinite job times were allowed by default.
ALLOWMULTICOMPUTE ALLOWMULTICOMPUTE tells Moab how to resolve conflicting information from different resource managers. If ALLOWMULTICOMPUTE is specified, Moab will use the STATE and OS information from the resource manager that reports the node as online.
CANCELFAILEDDEPENDENCYJOBS

Automatically cancels dependency jobs that will never run because of an unmet requirement. For example, if you ran a job with both an afterok and afternotok job attached to it and that job was successful, the afterok job would run, leaving the afternotok job idle in the queue. If you set CANCELFAILEDDEPENDENCYJOBS, Moab will cancel the job with the failed dependency and remove it from the queue. For more information about job dependencies, see Job Dependencies.

If you want to cancel all jobs that a specified <job_id> depends on, use mjobctl -c flags=follow-dependency <job_id> instead.

DISABLEPERJOBNODESETS Disables a job's ability to override the system specified node set. See 13.3 Resource Manager Extensions for more information.
DISABLEPARTIALNODERESERVATIONS Blocks partial node reservations.
ENABLEDYNAMICNODES Enables the ability to automatically remove nodes from Moab that are no longer reported by the resource manager.
ENABLEJOBTRIGGERSONRSV Enable job start triggers based on the job's reservation and not the actual start of the job (allows for negative offset job start triggers).
ENABLEMOABJOBENV Puts the Moab job variables on every job.
ENABLESLURMMEMPERCPU

By default Moab calls sbatch or srun with a --mem= request in a SLURM environment. When you set ENABLESLURMMEMPERCPU, Moab instead calls --mem-per-cpu=. This is to allow sites with policies that require the other parameter to use --mem-per-cpu.

ENFORCERESERVEDNODES Without this flag Moab tries to optimize the reservation for a job before it starts, meaning a job may start on nodes that weren't part of its reservation. With this flag Moab tries to start jobs only on the nodes that were reserved.
ENFORCESAMENODESET The same nodeset is not enforced across job requirements by default, rather each requirement is scheduled separately and the nodesets are determined on a per-req basis. To have Moab enforce the same nodeset across all job requirements set this flag.
EXTENDEDGROUPSUPPORT Allows Moab to consider a user's secondary Linux groups when dealing with reservation ACLs.
FASTGROUPLOOKUP Moab will use the system call getgrouplist to gather group information. This can significantly improve performance on some LDAP systems.
FASTRSVSTARTUP

Speeds up start time if there are existing reservations.

When you set the FASTRVSSTARTUP flag, Moab will also set the DISABLEPARTIALNODERESERVATIONS flag.

FASTRSVSTARTUP is incompatible with partial node reservations.

FASTRSVSTARTUP maintains the resource manager-reported node order at all times.

On very large systems, if there is a reservation in the checkpoint file on all the nodes, it would take a really long time for Moab to start up. For every node in the reservation, Moab checks every other node. With this flag, Moab just uses the nodelist that was checkpointed to create the reservation. It speeds up the startup process because it doesn't have to check every node. Where Moab would take 8 - 10 minutes to start up with an 18,000 node reservation without the flag, Moab can start up in 2-3 minutes with the flag.

With the flag you will see one difference in checknode. A reservation that uses all the procs on a node initially shows that all the procs are blocked. Without the flag, and as jobs fill on the node, the blocked resources will be configured - dedicated (ex. 5/6). With the flag, the blocked resources will always be what the reservation is blocking and won't change when jobs fill on the node.

Without flag:
Reservations:
brian.1x1 User -00:12:52 -> INFINITY ( INFINITY)
Blocked Resources@-00:00:02 Procs: 5/6 (83.33%) Mem: 0/5000 (0.00%)
Blocked Resources@00:04:58 Procs: 6/6 (100.00%) Mem: 0/5000 (0.00%)
m.2x1 Job:Running -00:00:02 -> 00:04:58 (00:05:00)
Jobs: m.2

 

With flag:
Reservations:
brian.1x1 User -00:00:15 -> INFINITY ( INFINITY)
Blocked Resources@-00:00:02 Procs: 6/6 (100.00%) Mem: 0/5000 (0.00%)
Blocked Resources@00:04:58 Procs: 6/6 (100.00%) Mem: 0/5000 (0.00%)
m.1x1 Job:Running -00:00:02 -> 00:04:58 (00:05:00)
Jobs: m.1

FILELOCKHA This is a High Availability feature. FILELOCKHA prevents scheduling conflicts between multiple Moab servers.
FREECOMPLETEDJOBSUBMITSTRING Moab frees the job submit string for completed jobs, decreasing the amount of memory needed during operation. This is useful in environments with large job scripts that can create a large memory footprint.
IGNOREPIDFILELOCK Moab will not fail if it cannot get a lock on the .moab.pid file. This is useful when Moab is running on a shared filesystem where file locking can be unpredictable.
INTERACTIVEWCACCURACY Moab will assume all interactive jobs are 100% accurate with respect to walltime. This is useful for sites that don't enforce walltimes for interactive jobs but don't want users punished for inaccurate interactive job walltimes.
JOBSUSERSVWALLTIME Allows jobs submitted without a walltime request or default walltime received from a class or queue but with an ADVRES:reservation to inherit their walltime limit from the reservation instead of the Moab default. The job walltime limit is then the remaining time of the reservation to which the job was submitted.
NOCLASSUPDATE While running against Torque, Moab will not update classes when it refreshes each iteration. Moab loads the classes at startup, but does not refresh them until the next time it is restarted.
NORMALIZETASKDEFINITIONS

Instructs Moab to normalize all tasks that it receives via an mshow -a command. Moab normalizes the task definition to one processor and then changes the tasks requested to the number of processors requested. For example, when the following is received by Moab:

mshow -a -w mintasks=1@procs:4+mem:4096

It is changed to this:

mshow -a -w mintasks=4@procs:1+,mem:1024,tpn=4
OPTIMIZEDBACKFILL Turns on an optimization within the FIRSTFIT backfill algorithm that checks whether there is a system-wide reservation blocking most jobs from running. This flag speeds up backfill scheduling (in the case where there is a system-wide reservation blocking most users) by checking access to the reservation sooner rather than later. This flag will be the default in future versions of Moab.
PRIORITYPOLICYBLOCKING

By default, a job that violates a policy is placed into the blocked queue. Jobs with a lower priority, but that do not violate the policy, will run. This can lead to situations in which small jobs starve out larger, higher priority jobs.

When you set the PRIORITYPOLICYBLOCKING flag, Moab allows the job that violates the policy to continue consuming the policy slots while it remains blocked. With the policy slots consumed, the smaller, lower priority jobs will not run. The higher priority job will continue to consume the policy slots until it has consumed enough to actually run.

Note that because the blocked job consumes policy slots, this will inevitably lead to lower system utilization.

SETDEFAULTHOSTLISTEXACTSET

By default, all Hostlist requests will be interpreted as an exactset request. See HOSTLIST for more information.

SHOWCOMPLETEDDEPENDENCIES Continues showing dependencies on a job even after the dependencies have been satisfied.
SHOWREQUESTEDPROCS Shows requested processors regardless of NodeAccessPolicy in showq. When SINGLEJOB NODEACCESSPOLICY is used and the job requests one processor, showq displays the job with one processor.
SHOWUSERJOBSONLY Causes Moab, when a non-admin user runs showq, to return only that user's jobs. If an administrator runs showq when this flag is set, Moab returns the jobs of all users; no restrictions are placed on administrators.
STRICTSPOOLDIRPERMISSIONS Enforces at least a 511 permission on the Moab spool directory.
SUSPENDEDJOBNODEBFINELIGIBLE

Turns off back fill on nodes where a job was suspended due to preemption.

When enabled, CHECKSUSPENDEDJOBPRIORITY must be set to True. See CHECKSUSPENDEDJOBPRIORITY.

UNMIGRATEONDEFER Forces Moab to unmigrate a job in a grid if it enters a deferred state.
USERMCOMPLETEDJOBSTATS Calculate job statistics using the RM reported metrics rather than Moab's internal metrics.

© 2016 Adaptive Computing