Moab Workload Manager > Appendices > Appendix A: Moab Parameters

Appendix A: Moab Parameters

See the Parameters Overview in the Moab Admin Manual for further information about specifying parameters.

Index: A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

ACCOUNTCFG[<ACCOUNTID>]
Format List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:
General Credential Flags, CHARGERATE, PRIORITY, ENABLEPROFILING, MEMBERULIST, PLIST, QDEF, QLIST, usage limit, or a fairness usage limit specification ( FSCAP, FSTARGET, and FSWEIGHT).
Default ---
Description Specifies account specific attributes. See the account overview for general information and the job flag overview for a description of legal flag values.
Example
ACCOUNTCFG[projectX] MAXJOB=50 QDEF=highprio

Up to 50 jobs submitted under the account ID projectX will be allowed to execute simultaneously and will be assigned the QOS highprio by default.

ACCOUNTINGINTERFACEURL
Format <URL> where protocol can be one of exec or file
Default ---
Description Specifies the interface to use for real-time export of Moab accounting/auditing information. See Exporting Events in Real-Time for more information.
Example
ACCOUNTINGINTERFACEURL exec:///$TOOLSDIR/dumpacc.pl
ACCOUNTWEIGHT
Format <INTEGER>
Default 1
Description Specifies the priority weight to be applied to the specified account priority. See Credential (CRED) Factor.
Example
ACCOUNTWEIGHT 100
ADMIN1, ADMIN2, ADMIN3
Description

These parameters are deprecated. Use ADMINCFG.

ADMINCFG[X]
Format One or more <ATTR>=<VALUE> pairs where <ATTR> is one of the following: ENABLEPROXY, USERS, GROUPS, SERVICES, or NAME
Default ---
Description Allows a site to configure which services and users belong to a particular level of administration. Note: The first user listed in the ADMINCFG[1] users list is considered to be the primary admin. The option USERS=ALL is allowed. The groups list adds the groups' users as if they were listed individually as USERS. To prevent Moab from assigning a primary user from the first group listed, you must specify a primary user first using the USERS attribute, then list the desired groups.
Example
ADMINCFG[1] USERS=root,john 
ADMINCFG[1] GROUPS=admin 
ADMINCFG[1] SERVICES=ALL 
ADMINCFG[1] NAME=batchadmin
ADMINCFG[3] USERS=bob,carol,smoore
ADMINCFG[3] GROUPS=science,math 
ADMINCFG[3] SERVICES=mjobctl,mcredctl,runjob
ADMINCFG[3] NAME=helpdesk

Members of the batchadmin admin role and members of the admin group are allowed to run all commands. Members of the helpdesk role and science and math groups are allowed to run mjobctl. They are also able to view and modify credential objects (i.e. users, groups, accounts, etc.) See the security overview for more details.

ADMINCFG[4] USERS=ALL SERVICES=checknode

All users can execute mdiag -n or checknode to get information on any node.

AGGREGATENODEACTIONS
Format <BOOLEAN>
Default FALSE
Description Consolidates queued node actions into as few actions as possible to reduce communication burden with resource manager. Node actions are queued until the AGGREGATENODEACTIONSTIME setting.
Note

This may delay some node actions. When reprovisioning, the system job may expire before the provision action occurs; while the action will still occur, the job will not show it.

Example
AGGREGATENODEACTIONS TRUE

Queues node actions together when possible.

AGGREGATENODEACTIONSTIME
Format <SECONDS>
Default 60
Description The delay time for the AGGREGATENODEACTIONS parameter to aggregate requests before sending job batches.
Example
AGGREGATENODEACTIONSTIME 120

Sets the AGGREGATENODEACTIONS delay to two minutes.

ALLOWMULTIREQNODEUSE
Format <BOOLEAN>
Default FALSE
Description By default Moab does not allow different requirements on the same job to occupy the same node. For example, if a job is submitted with nodes=2:ppn=8+4:fast:ppn=16, it's possible that some of the tasks requested could overlap onto the same node. This parameter instructs Moab to allow overlapping the same node, or not. This parameter also applies to the various -w clauses of an mshow -a command.
Example
ALLOWMULTIREQNODEUSE TRUE
ALLOWROOTJOBS
Format <BOOLEAN>
Default FALSE
Description Specifies whether batch jobs from the root user (UID=0) are allowed to be executed. Note: The resource manager must also support root jobs.
Example
ALLOWROOTJOBS TRUE

Jobs from the root user can execute.

ALLOWVMMIGRATION
Format <BOOLEAN>
Default FALSE
Description Enables Moab to migrate VMs.
Example
ALLOWVMMIGRATION TRUE
ALWAYSEVALUATEALLJOBS
Format <BOOLEAN>
Default FALSE
Description When scheduling priority jobs, Moab stops scheduling when it encounters the first job that cannot run and cannot get a reservation. ALWAYSEVALUATEALLJOBS directs Moab to continue scheduling until all priority jobs (jobs that do not violate any limits) are evaluated.
Example
ALWAYSEVALUATEALLJOBS TRUE
AMCFG
Format One or more key-value pairs as described in the Accounting Manager Overview.
Default ---
Description Specifies the interface and policy configuration for the scheduler-accounting manager interface. Described in detail in the Configuring the Accounting Manager Interface.
Example
AMCFG[mam] SERVER=mam://master.ufl.edu STARTFAILUREACTION=HOLD TIMEOUT=15
APPLICATIONLIST
Format Space-delimited list of generic resources.
Default ---
Description Specifies which generic resources represent actual applications on the cluster/grid. See Managing Consumable Generic Resources for more information.
Example
NODECFG[node01] GRES=calclab:1,powerhouse:1 RCSOFTWARE=calclab:1,powerhouse:1
NODECFG[node02] GRES=calclab:1,powerhouse:1 RCSOFTWARE=calclab:1,powerhouse:1
APPLICATIONLIST calclab,powerhouse

The generic resources calclab and powerhouse will now be recognized and treated as application software.

ARRAYJOBPARLOCK
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, all sub jobs of an array are locked to a single partition. The default behavior when scheduling array sub jobs is to span the jobs across partitions when possible. The ARRAYJOBPARLOCK job flag can be used to specify partition locking at submit time. The ARRAYJOBPARSPAN job flag overrides this parameter.
Example
ARRAYJOBPARLOCK TRUE
ASSIGNVLANFEATURES
Format <BOOLEAN>
Default FALSE
Description When set to TRUE, this forces all VMs to be contained in VLANs.
Example
ASSIGNVLANFEATURES TRUE
ATTRATTRWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight to be applied to jobs with the specified job attribute. See Attribute (ATTR) Factor.
Example
ATTRATTRWEIGHT 100
ATTRGRESWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight to be applied to jobs requesting the specified generic resource. See Attribute (ATTR) Factor.
Example
ATTRGRESWEIGHT 200
ATTRSTATEWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight to be applied to jobs with the specified job state. See Attribute (ATTR) Factor.
Example
ATTRSTATEWEIGHT 200
ATTRWEIGHT
Format <INTEGER>
Default 1
Description Specifies the priority component weight to be applied to the ATTR subcomponents. See Attribute (ATTR) Factor.
Example
ATTRWEIGHT      2
ATTRSTATEWEIGHT 200
BACKFILLDEPTH
Format <INTEGER>
Default 0 (no limit)
Description Specifies the number of idle jobs to evaluate for backfill. The backfill algorithm will evaluate the top <X> priority jobs for scheduling. By default, all jobs are evaluated.
Example
BACKFILLDEPTH 128

Evaluate only the top 128 highest priority idle jobs for consideration for backfill.

BACKFILLPOLICY
Format One of FIRSTFIT or NONE
Default FIRSTFIT
Description Specifies which backfill algorithm will be used. See Configuring Backfill for more information.
Example
BACKFILLPOLICY  NONE
BFCHUNKDURATION
Format [[[DD:]HH:]MM:]SS
Default 0 (chunking disabled)
Description Specifies the duration during which freed resources will be aggregated for use by larger jobs. Used in conjunction with BFCHUNKSIZE. See Configuring Backfill for more information.
Example
BFCHUNKDURATION 00:05:00
BFCHUNKSIZE     4

Aggregate backfillable resources for up to 5 minutes, making resources available only to jobs of size 4 or larger.

BFCHUNKSIZE
Format <INTEGER>
Default 0 (chunking disabled)
Description Specifies the minimum job size which can utilize chunked resources. Used in conjunction with BFCHUNKDURATION. See Configuring Backfill for more information.
Example
BFCHUNKDURATION 00:05:00
BFCHUNKSIZE     4

Aggregate backfillable resources for up to 5 minutes, making resources available only to jobs of size 4 or larger.

BFMINVIRTUALWALLTIME
Format [[[DD:]HH:]MM:]SS
Default ---
Description

Specifies the minimum job wallclock time for virtual scaling (optimistic-like backfilling.) Any job with a wallclock time less than this setting will not be virtually scaled. The value specified relates to a job's original walltime and not its virtually-scaled walltime.

Example
BFMINVIRTUALWALLTIME  00:01:30
BFPRIORITYPOLICY
Format One of RANDOM, DURATION, or HWDURATION
Default ---
Description Specifies policy to use when prioritizing backfill jobs for preemption
Example
BFPRIORITYPOLICY  DURATION

Use length of job in determining which backfill job to preempt.

BFVIRTUALWALLTIMECONFLICTPOLICY
Format One of the following: PREEMPT
Default ---
Description Specifies how to handle scheduling conflicts when a virtually scaled job "expands" to its original wallclock time. This occurs when the job is within one scheduling iteration - RMPOLLINTERVAL - of its virtually scaled wallclock time expiring.
Example
BFVIRTUALWALLTIMECONFLICTPOLICY  PREEMPT
BFVIRTUALWALLTIMESCALINGFACTOR
Format <DOUBLE>
Default 0 (virtual scaling disabled)
Description

Specifies the factor by which eligible jobs' wallclock time is virtually scaled (optimistic-like backfilling).

If you do not want scaling, set BFVIRTUALWALLTIMESCALINGFACTOR to "0" (default). Setting to "1" is not recommended as it impacts performance. When set to "1", Moab will exercise the code paths of scaling but no actual scaling will occur.

Example
BFVIRTUALWALLTIMESCALINGFACTOR .4
BYPASSCAP
Format <INTEGER>
Default 0
Description Specifies the max weighted value allowed from the bypass count subfactor when determining a job's priority (see Priority Factors for more information).
Example
BYPASSWEIGHT 5000
BYPASSCAP    30000
BYPASSWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight to be applied to a job's backfill bypass count when determining a job's priority (see Priority Factors for more information).
Example
BYPASSWEIGHT 5000
CHECKPOINTDIR
Format <STRING>
Default ---
Description Specifies the directory for temporary job checkpoint files (usually of the form jobid.cp). This is not the directory for Moab's checkpoint file (.moab.ck).
Example
CHECKPOINTDIR  /tmp/moabcheckpoint
CHECKPOINTEXPIRATIONTIME
Format [[[DD:]HH:]MM:]SS or UNLIMITED
Default 3,000,000 seconds
Description Specifies how 'stale' checkpoint data can be before it is ignored and purged.
Example
CHECKPOINTEXPIRATIONTIME 1:00:00:00

Expire checkpoint data which has been stale for over 1 day.

CHECKPOINTFILE
Format <STRING>
Default moab.ck
Description Name (absolute or relative) of the Moab checkpoint file.
Example
CHECKPOINTFILE /var/adm/moab/moab.ck

Maintain the Moab checkpoint file in the file specified.

CHECKPOINTINTERVAL
Format [[[DD:]HH:]MM:]SS
Default 00:05:00
Description

Time between automatic Moab checkpoints.

If RMPOLLINTERVAL does not specify both a minimum and maximum poll time, Moab will ignore CHECKPOINTINTERVAL and checkpoint every iteration.

Example
CHECKPOINTINTERVAL 00:15:00

Moab should checkpoint state information every 15 minutes.

CHECKPOINTWITHDATABASE
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, Moab stores checkpoint information to a database rather than to the .moab.ck flat text file.
Example
CHECKPOINTWITHDATABASE    TRUE
CHECKSUSPENDEDJOBPRIORITY
Format <BOOLEAN>
Default TRUE
Description Prevents Moab from starting a job on any node containing a suspended job of higher priority.
Example
CHECKSUSPENDEDJOBPRIORITY    FALSE
CHILDSTDERRCHECK
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, child processes Moab executes are considered failed if their standard error stream contains the text "ERROR".
Example
CHILDSTDERRCHECK    TRUE
CLASSCFG[<CLASSID>]
Format

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

General Credential Flags, DEFAULT.ATTR, DEFAULT.DISK, DEFAULT.FEATURES, DEFAULT.GRES, DEFAULT.MEM, DEFAULT.NODE, DEFAULT.NODESET, DEFAULT.PROC, ENABLEPROFILING, EXCL.FEATURES, EXCLUDEUSERLIST, HOSTLIST, JOBEPILOG, JOBPROLOG, MAXPROCPERNODE, MAX.NODE, MAX.PROC, MAX.WCLIMIT, MIN.NODE, MIN.PROC, MIN.TPN, MIN.WCLIMIT, PARTITION, PRIORITY, PRIORITYF, QDEF, QLIST, REQ.FEATURES, REQUIREDACCOUNTLIST, REQUIREDUSERLIST, RESFAILPOLICY, SYSPRIO, WCOVERRUN, usage limit, or fairshare usage limit specification.

Default ---
Description Specifies class specific attributes (see Credential Overview for details).
Example
CLASSCFG[batch] MAXJOB=50 QDEF=highprio

Up to 50 jobs submitted to the class batch will be allowed to execute simultaneously and will be assigned the QOS highprio by default.

CLASSWEIGHT
Format <INTEGER>
Default 1
Description Specifies the weight to be applied to the class priority of each job (see Credential (CRED) Factor and credential priority).
Example
CLASSWEIGHT 10
CLIENTCFG[<X>]
Format One or more of <ATTR>-<VALUE> pairs where <X> indicates the specified peer and <ATTR> is one of the following: AUTH, AUTHCMD, AUTHTYPE, HOST, KEY, or DEFAULTSUBMITPARTITION.
Default ---
Description Specifies the shared secret key and authentication method which Moab will use to communicate with the named peer daemon. See Security Overview for more information. Note: The AUTHTYPE and KEY attributes of this parameter may only be specified in the moab-private.cfg config file.
Example
CLIENTCFG[silverB] KEY=apple7 AUTH=admin1

Moab will use the session key apple7 for peer authentication and for encrypting and decrypting messages sent from silverB. Also, client connections from this interface will be authorized at an admin 1 level.

CLIENTMAXCONNECTIONS
Format <INTEGER>
Default 128
Description Changes the maximum number of connections that can simultaneously connect to Moab. The value can be increased during runtime, but it cannot be decreased. The value cannot be reduced below the default value of 128.
Example
CLIENTMAXCONNECTIONS 256

Doubles the maximum number of connections.

CLIENTMAXPRIMARYRETRY
Format <INTEGER> or INFINITY
Default 1
Description Specifies how many times the client command will attempt to retry its connection to the primary server if Moab is not available.
Example
CLIENTMAXPRIMARYRETRY 5
CLIENTMAXPRIMARYRETRYTIMEOUT 1000

The client command will attempt to retry its connection to the primary server 5 times with 1 second intervals before giving up. Note: If INFINITY is specified, Moab will attempt 2,140,000,000 times.

CLIENTMAXPRIMARYRETRYTIMEOUT
Format <INTEGER> (milliseconds)
Default 2000
Description Specifies how much time to wait until the client command will attempt to retry its connection to the primary server if Moab is not available.
Example
CLIENTMAXPRIMARYRETRY 3
CLIENTMAXPRIMARYRETRYTIMEOUT 500

The client command will attempt to retry its connection to the primary server 3 times with .5 second intervals before giving up.

CLIENTTIMEOUT
Format [[[DD:]HH:]MM:]SS
Default 00:00:30
Description Time which Moab client commands will wait for a response from the Moab server. See Client Configuration for more information. Note: May also be specified as an environment variable.
Example
CLIENTTIMEOUT 00:15:00

Moab clients will wait up to 15 minutes for a response from the server before timing out.

CLIENTUIPORT
Format <INTEGER>
Default N/A
Description Port on which to listen when UIMANAGEMENTPOLICY FORK is specified. This is typically Moab's configured listen port + 1.
Example
UIMANAGEMENTPOILCY FORK
CLIENTUIPORT 42560

Moab is typically configured to listen on port 42559.

CREDDISCOVERY
Format TRUE
Default FALSE
Description Specifies that Moab should create otherwise unknown credentials when it discovers them in the statistics files.
Example
CREDDISCOVERY TRUE
CREDWEIGHT
Format <INTEGER>
Default 1
Description Specifies the credential component weight associated with the credential priority. See Credential (CRED) Factor for more information.
Example
CREDWEIGHT 2
DATASTAGEHOLDTYPE
Format Any Job Hold type
Default DEFER
Description Specifies what to do if a job's data staging operations fail.
Example
DATASTAGEHOLDTYPE  BATCH
DEADLINEPOLICY
Format One of CANCEL, HOLD, IGNORE, or RETRY
Default NONE
Description Specifies what to do when a requested deadline cannot be reached (see Job Deadlines).
Example
DEADLINEPOLICY  IGNORE
DEFAULTCLASSLIST
Format Space-delimited list of one or more <STRING>s.
Default ---
Description Specifies the default classes supported on each node for RM systems which do not provide this information.
Example
DEFAULTCLASSLIST serial parallel
DEFAULTSUBMITPARTITION
Format See parameter CLIENTCFG[].
Default ---
Description If a user submits a job using msub which does not specify host, feature, or partition constraints, then the msub client will insert the specified default submit partition into the newly submitted job as a hard requirement.
Example
CLIENTCFG[DEFAULT] DEFAULTSUBMITPARTITION=partition1
DEFERCOUNT
Format <INTEGER>
Default 24
Description Specifies the number of times a job can be deferred before it will be placed in batch hold.
Example
DEFERCOUNT 12
DEFERSTARTCOUNT
Format <INTEGER>
Default 1
Description Specifies the number of times a job will be allowed to fail in its start attempts before being deferred. JOBRETRYTIME overrides DEFERSTARTCOUNT; DEFERSTARTCOUNT only begins when the JOBRETRYTIME window elapses. Note: A job's startcount will increase each time a start request is made to the resource manager regardless of whether or not this request succeeded. This means start count increases if job starts fail or if jobs are started and then rejected by the resource manager. (For related information, see Reservation Policies, DEFERTIME, RESERVATIONRETRYTIME, NODEFAILURERESERVETIME, JOBRETRYTIME, and GUARANTEEDPREEMPTION.)
Example
DEFERSTARTCOUNT 3
DEFERTIME
Format [[[DD:]HH:]MM:]SS
Default 1:00:00
Description Specifies the amount of time a job will be held in the deferred state before being released back to the Idle job queue. Note: A job's defer time will be restarted if Moab is restarted. (For related information, see Reservation Policies, DEFERSTARTCOUNT, RESERVATIONRETRYTIME, NODEFAILURERESERVETIME, JOBRETRYTIME, and GUARANTEEDPREEMPTION.)
Example
DEFERTIME 0:05:00
DELETESTAGEOUTFILES
Format <BOOLEAN>
Default FALSE
Description Specifies whether the scheduler should delete explicitly specified stageout files after they are successfully staged. By default, such files are not deleted but are left on the nodes where the job ran.
Example
DELETESTAGEOUTFILES TRUE
Example of an explicit stageout request
msub x=MSTAGEOUT:ssh://source_node/tmp/file,file:///results_folder
job.cmd

With this parameter set to TRUE, /tmp/file on source_node is deleted after it is copied to the specified destination ( file:///results_folder). If the parameter is not set, or if it is set to FALSE, /tmp/file remains on source_node after the job terminates.

DEPENDFAILUREPOLICY
Format HOLD or CANCEL
Default HOLD
Description Specifies what happens to a job if its dependencies cannot be fulfilled; that is, what happens when a job depends on another job to complete successfully but the other job fails.
Example
DEPENDFAILUREPOLICY CANCEL

If job A is submitted with depend=afterok:B and job B fails, job A is canceled.

DIRECTORYSERVER
Format <HOST>[:<PORT>]
Default ---
Description Specifies the interface for the directory server.
Example
DIRECTORYSERVER calli3.icluster.org:4702
DISABLEEXCHLIST
Format <BOOLEAN>
Default FALSE
Description If the resource manager rejects a job and the value is set to TRUE, then the node is not added to the job's exclude host list.
Example
DISABLEEXCHLIST TRUE
DISABLEINTERACTIVEJOBS
Format <BOOLEAN>
Default FALSE
Description

Disallows interactive jobs submitted with msub -I.

Note: It is possible for users to submit interactive jobs directly to a resource manager, which can bypass the DISABLEINTERACTIVEJOBS parameter. However, some resource managers (such as TORQUE) will check with Moab before allowing an interactive job.

Example
DISABLEINTERACTIVEJOBS TRUE
DISABLEREGEXCACHING
Format <BOOLEAN>
Default FALSE
Description Turns off regular expression caching. Turning off regular expression caching preserves memory with hostlist reservations and speeds up start time.
Example
DISABLEREGEXCACHING TRUE
DISABLEREQUIREDGRESNONE
Format <BOOLEAN>
Default FALSE
Description When set to TRUE, this causes Moab to reject msub requests that have a gres of "none". ENFORCEGRESSACCESSS must also be set to TRUE for this feature to work.
Example
##########  moab.cfg  ##########
ENFORCEGRESACCESS TRUE         
DISABLEREQUIREDGRESNONE TRUE   
################################

> msub -A ee -l nodes=1,ttc=5,walltime=600,partition=g02 -l gres=none
ERROR: cannot submit job - cannot locate required resource 'none'
DISABLESAMECREDPREEMPTION
Format Comma-delimited list of one or more credentials: ACCT, CLASS, GROUP, QOS, or USER
Default ---
Description This parameter prevents specified credentials from preempting its own jobs.
Example
DISABLESAMECREDPREEMPTION  QOS,USER
DISABLESCHEDULING
Format <BOOLEAN>
Default FALSE
Description Specifies whether or not the scheduler will schedule jobs. If set to TRUE, Moab will continue to update node and job state but will not start, preempt, or otherwise modify jobs. The command mschedctl -r will clear this parameter and resume normal scheduling.
Example
DISABLESCHEDULING FALSE
DISABLESLAVEJOBSUBMIT
Format <BOOLEAN>
Default TRUE
Description This parameter can be added to the moab.cfg file on a slave Moab server (in a grid configuration) to prevent users from submitting jobs to the master Moab server from the slave Moab server. Some grid configurations allow the user to submit jobs on the slave that are migrated to the master and submitted from the master. Other grid configurations do not allow the jobs to be migrated to the master from the slave, in which case, jobs submitted from the slave remain idle on the slave and never run. This parameter will reject the job submissions on the slave to prevent the submission of jobs that will never run.
Example
DISABLESLAVEJOBSUBMIT TRUE
example (node04 is a slave and node06 is the master)
[test@node04 moab-slurm]$ echo sleep 100 | msub
ERROR:    cannot submit job from slave
DISABLETHRESHOLDTRIGGERS
Format <BOOLEAN>
Default FALSE
Description This makes Moab not fire threshold-based triggers, but will log the intended action to the event logs. Similar to DISABLEVMDECISIONS.
Example
DISABLETHRESHOLDTRIGGERS TRUE
DISABLEVMDECISIONS
Format <BOOLEAN>
Default FALSE
Description This makes Moab not take any automatic decisions with respect to VM's, namely powering on/off nodes and migrating VMs. Intended actions will instead be logged in the event logs. Similar to DISABLETHRESHOLDTRIGGERS.
Example
DISABLEVMDECISIONS TRUE
DISKWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight to be applied to the amount of dedicated disk space required per task by a job (in MB).
Example
RESWEIGHT  10
DISKWEIGHT 100

If a job requires 12 tasks and 512 MB per task of dedicated local disk space, Moab will increase the job's priority by 10 * 100 * 12 * 512

DISPLAYFLAGS
Format One or more of the following values (space delimited):

ACCOUNTCENTRIC, HIDEBLOCKED, HIDEDRAINED, NODECENTRIC, or USEBLOCKING

Default ---
Description

Specifies flags that control how Moab client commands display varied information.

ACCOUNTCENTRIC will display account information in showq, rather than group information.

HIDEBLOCKED will prevent showq from listing information about blocked jobs which are not owned by the user if the user is not an admin.

HIDEDRAINED prevents mdiag -n from displaying nodes and mvmctl -q from displaying VMs in the DRAINED state. An override option of mdiag -n -w nodestate=drained lists only those nodes with a DRAINED state. Similarly, an override option of mvmctl -q -w state=drained lists only those VMs with a DRAINED state.

NODECENTRIC will display node allocation information instead of processor allocation information in showq.

USEBLOCKING disables threading for commands that support it; those commands include showq, mdiag -n, and mdiag -j.

Example
DISPLAYFLAGS NODECENTRIC
DISPLAYPROXYUSERASUSER
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, Moab shows the proxy users instead of the real user on some queries of system jobs that have proxy users. Commands affected include mjobctl -q diag and checkjob.
Example
DISPLAYPROXYUSERASUSER TRUE                                                      
DONTCANCELINTERACTIVEHJOBS
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, Moab does not cancel interactive jobs that are held.
Example
DONTCANCELINTERACTIVEHJOBS TRUE
DONTENFORCEPEERJOBLIMITS
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, only the scheduler that is running the job can cancel the job or enforce other limits.
Example
DONTENFORCEPEERJOBLIMITS TRUE
EMULATIONMODE
Format SLURM
Default ---
Description Specifies whether or not the scheduler will perform the automatic setup of a particular resource manager environment.
Example
EMULATIONMODE SLURM

Moab will perform the automated setup steps as if it were interfacing with a slurm resource manager (automatic QOS creation).

ENABLEFAILUREFORPURGEDJOB
Format <BOOLEAN>
Default FALSE
Description

By default, when a job is purged or removed by the TORQUE resource manager for a walltime violation, the job takes on a state of Completed and a completion code of 0. If TRUE, the job state is set to Removed and has a completion code of 98. ENABLEFAILUREFORPURGEDJOB is for the TORQUE resource manager only.

For ENABLEFAILUREFORPURGEDJOB to return Removed job states, you must reset the TORQUE server attribute keep_completed to 0 in qmgr. See "Queue Attributes" in the TORQUE Administrator Guide for more information.

Example
ENABLEFAILUREFORPURGEDJOB TRUE

Jobs that are purged or removed by TORQUE are given a state of Removed and a completion code of 98.

ENABLEFSVIOLATIONPREEMPTION
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, Moab will allow jobs within the same class/queue to preempt when the preemptee is violating a fairshare target and the preemptor is not.
Example
ENABLEFSVIOLATIONPREEMPTION TRUE
ENABLEHIGHTHROUGHPUT
Format <BOOLEAN>
Default FALSE
Description

Configures Moab so that it will accept msub submissions, start jobs, process triggers, etc., in a manner which minimizes their processing time. The downside is that Moab will return minimal information about these jobs at submit time.

If ENABLEHIGHTHROUGHPUT is TRUE, you must set NODEALLOCATIONPOLICY to FIRSTAVAILABLE.

Example
ENABLEHIGHTHROUGHPUT TRUE

Moab can now accept hundreds of jobs per second using msub instead of 20-30.

ENABLEJOBARRAYS
Format <BOOLEAN>
Default TRUE
Description If set to TRUE, job arrays will be enabled .
Example
ENABLEJOBARRAYS TRUE
ENABLENEGJOBPRIORITY
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, the scheduler allows job priority value to range from -INFINITY to MMAX_PRIO; otherwise, job priority values are given a lower bound of '1'. For more information, see REJECTNEGPRIOJOBS.
Example
ENABLENEGJOBPRIORITY TRUE

Job priority may range from -INFINITY to MMAX_PRIO.

ENABLENODEADDRLOOKUP
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, the scheduler will use the default host name service lookup mechanism (i.e., /etc/hosts, DNS, NIS, etc.) to determine the IP address of the nodes reported by the resource manager. This information is used to correlate information reported by multi-homed hosts.
Example
ENABLENODEADDRLOOKUP TRUE
ENABLEPOSUSERPRIORITY
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, the scheduler will allow users to specify positive job priority values which will be honored. In other words, users can specify a priority that falls in the range of -1024 to +1023, inclusive. If set to FALSE (the default), user priority values are given an upper bound of '0' when users request a positive priority. See USERPRIOWEIGHT.
Example
ENABLEPOSUSERPRIORITY TRUE

Users may now specify positive job priorities and have them take effect (e.g. msub -p 100 job.cmd).

ENABLESPVIOLATIONPREEMPTION
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, Moab will allow jobs within the same class/queue to preempt when the preemptee is violating a soft usage policy and the preemptor is not.
Example
ENABLESPVIOLATIONPREEMPTION TRUE
ENABLEVMDESTROY
Format <BOOLEAN>
Default FALSE
Description If set to TRUE, enables the automatic destruction of a VM when the VM wall time is expired or when the VM is stale and configured to be destroyed (for more information, see VMSTALEACTION).
Example
ENABLEVMDESTROY TRUE
ENFORCEACCOUNTACCESS
Format <BOOLEAN>
Default FALSE
Description Specifies whether or not Moab will enforce account access constraints without an accounting manager.
Example
ENFORCEACCOUNTACCESS TRUE
ENFORCEGRESACCESS
Format <BOOLEAN>
Default FALSE
Description If a user submits a job with a non-existent gres (e.g. in the case of a typo) and ENFORCEGREACCESS is not set in moab.cfg, or is set to FALSE, then the requested gres will be created (but will not exist on any nodes) and the job will be deferred (similar to ENFORCEACCOUNTACCESS).
Example
ENFORCEGRESACCESS TRUE
EVENTSERVER
Format <HOST>[:<PORT>]
Default ---
Description Specifies the interface for the event server.
Example
EVENTSERVER  calli3.icluster.org:4702
FEATURENODETYPEHEADER
Format <STRING>
Default ---
Description Specifies the header used to specify node type via node features (i.e. LL features or PBS node attributes).
Example
FEATURENODETYPEHEADER xnt

Moab will interpret all node features with the leading string xnt as a nodetype specification - as used by the accounting manager and other accounting managers, and assign the associated value to the node. i.e., xntFast.

FEATUREPARTITIONHEADER
Format <STRING>
Default ---
Description Specifies the header used to specify node partition via node features (i.e. LL features or PBS node attributes).
Example
FEATUREPARTITIONHEADER xpt

Moab will interpret all node features with the leading string xpt as a partition specification and assign the associated value to the node. i.e., xptGold.

FEATUREPROCSPEEDHEADER
Format <STRING>
Default ---
Description Specifies the header used to extract node processor speed via node features (i.e., LL features or PBS node attributes). Note: Adding a trailing '$' character will specify that only features with a trailing number be interpreted. For example, the header 'sp$' will match 'sp450' but not 'sport'.
Example
FEATUREPROCSPEEDHEADER xps

Moab will interpret all node features with the leading string xps as a processor speed specification and assign the associated value to the node. i.e., xps950.

FEATURERACKHEADER
Format <STRING>
Default ---
Description Specifies the header used to extract node rack index via node features (i.e., LL features or PBS node attributes). Note: Adding a trailing '$' character will specify that only features with a trailing number be interpreted. For example, the header 'rack$' will match 'rack4' but not 'racket'.
Example
FEATURERACKHEADER rack

Moab will interpret all node features with the leading string rack as a rack index specification and assign the associated value to the node. i.e., rack16.

FEATURESLOTHEADER
Format <STRING>
Default ---
Description Specifies the header used to extract node slot index via node features (i.e., LL features or PBS node attributes). Note: Adding a trailing '$' character will specify that only features with a trailing number be interpreted. For example, the header 'slot$' will match 'slot12' but not 'slotted'.
Example
FEATURESLOTHEADER slot

Moab will interpret all node features with the leading string slot as a slot index specification and assign the associated value to the node. i.e., slot16.

FEEDBACKPROGRAM
Format <STRING>
Default ---
Description Specifies the name of the program to be run at the completion of each job. If not fully qualified, Moab will attempt to locate this program in the 'tools' subdirectory.
Example
FEEDBACKPROGRAM /var/moab/fb.pl

Moab will run the specified program at the completion of each job.

FILEREQUESTISJOBCENTRIC
Format <BOOLEAN>
Default FALSE
Description Specifies whether a job's file request is a total request for the job or a per task request.
Example
FILEREQUESTISJOBCENTRIC TRUE

Moab will treat file requests as a total request per job.

FILTERCMDFILE
Format <BOOLEAN>
Default TRUE
Description Running the msub command performs the following operations on the submission script:
  • Replace all comments with spaces (excludes Resource Manager directives)
  • Strip empty lines
  • Replace \r with \n
  • Lock job to a PBS resource manager if $PBS is found in the script

Include the FILTERCMDFILE parameter in the moab.cfg file that resides on the clients.

FILTERCMDFILE must be FALSE for REJECTDOSSCRIPTS to work correctly.

Example
FILTERCMDFILE FALSE

Running the msub command does not perform the actions detailed earlier.

FORCENODEREPROVISION
Format <BOOLEAN>
Default FALSE
Description When set to TRUE, this config option causes Moab to reprovision a node, even if it is to the same operating system (in essence rewriting the OS).
Example
FORCENODEREPROVISION TRUE
FORCERSVSUBTYPE
Format <BOOLEAN>
Default FALSE
Description Specifies that admin reservations must have a subtype associated with them.
Example
FORCERSVSUBTYPE TRUE

Moab will require all admin reservations to include a subtype.

FREETIMELOOKAHEADDURATION
Format [[[DD:]HH:]MM:]SS
Default 2 Months
Description Specifies how far ahead Moab will look when calculating free time on a node.
Example
FREETIMELOOKAHEADDURATION 7:00:00:00
                                                                

Moab will look 1 week ahead when it calculates free time on a node.

FSACCOUNTWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight assigned to the account subcomponent of the fairshare component of priority.
Example
FSACCOUNTWEIGHT 10
FSCAP
Format <DOUBLE>
Default 0 (NO CAP)
Description Specifies the maximum allowed absolute value for a job's total pre-weighted fairshare component.
Example
FSCAP 10.0

Moab will bind a job's pre-weighted fairshare component by the range +/- 10.0.

FSCLASSWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight assigned to the class subcomponent of the fairshare component of priority.
Example
FSCLASSWEIGHT 10
FSDECAY
Format <DOUBLE>
Default 1.0
Description Specifies decay rate applied to past fairshare interval when computing effective fairshare usage. Values may be in the range of 0.01 to 1.0. A smaller value causes more rapid decay causing aged usage to contribute less to the overall effective fairshare usage. A value of 1.0 indicates that no decay will occur and all fairshare intervals will be weighted equally when determining effective fairshare usage. See Fairshare Overview.
Example
FSPOLICY   DEDICATEDPS
FSDECAY    0.8
FSDEPTH    8

Moab will apply a decay rate of 0.8 to all fairshare windows.

FSDEPTH
Format <INTEGER>
Default 8
Description Note: The number of available fairshare windows is bounded by the MAX_FSDEPTH value (32 in Moab). See Fairshare Overview.
Example
FSDEPTH 12
FSENABLECAPPRIORITY
Format <BOOLEAN>
Default FALSE
Description Fairshare priority will increase to target and stop.
Example
FSENABLECAPPRIORITY TRUE
FSGROUPWEIGHT
Format <INTEGER>
Default 0
Description
Example
FSGROUPWEIGHT 4
FSINTERVAL
Format [[[DD:]HH:]MM:]SS
Default 12:00:00
Description Specifies the length of each fairshare window.
Example
FSINTERVAL 12:00:00

Track fairshare usage in 12 hour blocks.

FSJPUWEIGHT
Format <INTEGER>
Default 0
Description Specifies the fairshare weight assigned to jobs per user.
Example
FSJPUWEIGHT 10
FSMOSTSPECIFICLIMIT
Format <BOOLEAN>
Default FALSE
Description When checking policy usage limits in a fairshare tree, if the most specific policy limit is passed then do not check the same policy again at higher levels in the tree. For example, if a user has a MaxProc policy limit then do not check the MaxProc policy limit on the account for this same user.
Example
FSMOSTSPECIFICLIMIT  TRUE
FSPOLICY
Format <POLICY>[*] where <POLICY> is one of the following: DEDICATEDPS, DEDICATEDPS%, DEDICATEDPES, or UTILIZEDPS.
Default ---
Description

Specifies the unit of tracking fairshare usage. The following options are:

  • DEDICATEDPS (dedicated processor seconds delivered) tracks dedicated processor seconds.
  • DEDICATEDPS% (dedicated processor seconds available) to specify that percentage based fairshare should be used. See Fairshare Overview and Fairshare Consumption Metrics for more information.
  • DEDICATEDPES (dedicated processor-equivalent seconds delivered) tracks dedicated processor-equivalent seconds.
  • UTILIZEDPS (utilized processor seconds delivered) tracks the number of utilized processor seconds.
Example
FSPOLICY DEDICATEDPES

Moab will track fairshare usage by dedicated processor-equivalent seconds.

FSPPUWEIGHT
Format <INTEGER>
Default 0
Description Specifies the fairshare weight assigned to processors per user.
Example
FSPPUWEIGHT 10
FSPSPUWEIGHT
Format <INTEGER>
Default 0
Description Specifies the fairshare weight assigned to processor-seconds per user.
Example
FSPSPUWEIGHT 10
FSQOSWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight assigned to the QOS fairshare subcomponent.
Example
FSQOSWEIGHT 16
FSTARGETISABSOLUTE
Format <BOOLEAN>
Default FALSE
Description Specifies whether Moab should base fairshare targets off of delivered cycles or up/available cycles.
Example
FSTARGETISABSOLUTE TRUE
FSTREE
Format List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:
SHARES or MEMBERLIST
Default ---
Description Specifies the share tree distribution for job fairshare prioritization (see Hierarchical Share Tree Overview).
Example
FSTREE[geo]  SHARES=16  MEMBERLIST=geo103,geo313,geo422
FSTREEACLPOLICY
Format OFF, PARENT, or FULL
Default FULL
Description Specifies how Moab should interpret credential membership when building the FSTREE (see Hierarchical Share Tree Overview).
Example
FSTREEACLPOLICY PARENT

Credentials will be given access to their parent node when applicable.

FSTREEISREQUIRED
Format <BOOLEAN>
Default FALSE
Description Specifies whether a job must have an applicable node in the partition's FSTREE in order to execute within that partition (see Hierarchical Share Tree Overview).
Example
FSTREEISREQUIRED TRUE

Jobs must have an applicable node in the FSTREE in order to execute.

FSTREEUSERISREQUIRED
Format <BOOLEAN>
Default FALSE
Description Specifies whether the user must be given explicit access to a branch in the FSTREE (see Hierarchical Share Tree Overview).
Example
FSTREEUSERISREQUIRED TRUE

Users must be given explicit access to FSTREE nodes in order to gain access to the FSTREE.

FSUSERWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight assigned to the user fairshare subfactor.
Example
FSUSERWEIGHT 8
FSWEIGHT
Format <INTEGER>
Default 1
Description Specifies the priority weight assigned to the summation of the fairshare subfactors (see Priority Factor and Fairshare overviews).
Example
FSWEIGHT 500
GEVENTCFG[<GEVENT>]
Format List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is ACTION, ECOUNT, REARM, or SEVERITY. See Responding to Generic Events for details on values you can assign to each attribute.
Default ---
Description Specifies how the scheduler should behave when various cluster events are detected. See the Generic Events Overview for more information.
Example
GEVENTCFG[hitemp] ACTION=avoid,record,notify  REARM=00:10:00
GEVENT[nodeerror] SEVERITY=3

If a hitemp event is detected, Moab adjusts the node allocation policy to minimize the allocation of the node. Moab also sends emails to cluster administrators and reports the event in the Moab event log.

GRESCFG[<GRES>]
Format

List of zero or more space delimited <ATTR >=<VALUE> pairs where <ATTR> is one of the following:

TYPE, PRIVATE, INVERTTASKCOUNT, FEATUREGRES, or STARTDELAY

Default ---
Description

Specifies associations of generic resources into resource groups.

When PRIVATE is set to TRUE, Moab puts the requested generic resource on a separate job request.

By default a private request is a request with 1 task with X number of generic resources per task. When INVERTTASKCOUNTandPRIVATE are set to TRUE, Moab makes the generic resource's private request a request with X number of tasks with 1 generic resource per task.

See 12.6 Managing Consumable Generic Resources for more information.

Example
GRESCFG[scsi1] TYPE=fastio
GRESCFG[scsi2] TYPE=fastio
GRESCFG[scsi3] TYPE=fastio

The generic resources scsi1, scsi2, and scsi3 are all associated with the generic resource type fastio.

GRESTOJOBATTR
Format Comma delimited list of generic resources
Default ---
Description The list of generic resources will also be interpreted as JOB features. See Managing Reservations.
Example
GRESTOJOBATTR  matlab,ccs

Jobs which request the generic resources matlab or ccs will have a corresponding job attribute assigned to them.

GROUPCFG[<GROUPID>]
Format

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

General Credential Flags, PRIORITY, ENABLEPROFILING, QLIST, QDEF, PLIST, FLAGS, usage limits, or a fairshare usage limit specification.

Default ---
Description Specifies group specific attributes. See the flag overview for a description of legal flag values.
Example
GROUPCFG[staff] MAXJOB=50 QDEF=highprio

Up to 50 jobs submitted by members of the group staff will be allowed to execute simultaneously and will be assigned the QOS highprio by default.

GROUPWEIGHT
Format <INTEGER>
Default 1
Description Specifies the priority weight assigned to the specified group priority (See Credential (CRED) Factor).
Example
GROUPWEIGHT 20
GUARANTEEDPREEMPTION
Format <BOOLEAN>
Default FALSE
Description

Causes Moab to lock PREEMPTOR jobs until JOBRETRYTIME expires (essentially, waiting for the PREEMPTEE jobs to finish).

It may take some time for the PREEMPTEE jobs to clear out. During that time, the PREEMPTOR job might want to look elsewhere to run, which would be disruptive as it might preempt another set of jobs. If you wish it prevent this, it is recommended that you set GUARANTEEDPREEMPTION to TRUE.

For related information, see About preemption, Reservation Policies, DEFERSTARTCOUNT, DEFERTIME, RESERVATIONRETRYTIME, NODEFAILURERESERVETIME, and JOBRETRYTIME.

Example
GUARANTEEDPREEMPTION TRUE
HALOCKCHECKTIME
Format [[[DD:]HH:]MM:]SS
Default 9
Description Specifies how frequently the secondary server checks the timestamp on the lock file. See High Availability Overview for more info.
Example
HALOCKCHECKTIME 00:00:15

The Moab fallback server will check the health of the Moab primary server every 15 seconds.

HALOCKUPDATETIME
Format [[[DD:]HH:]MM:]SS
Default 3
Description Specifies how frequently the primary server checks the timestamp on the lock file. See High Availability Overview for more info.
Example
HALOCKUPDATETIME 00:00:03

The Moab primary server will check the timestamp of the lock file every 3 seconds.

HIDEVIRTUALNODES
Format <BOOLEAN>
Default ---
Description Enables VM management; also used to reveal hypervisors.
Example
HIDEVIRTUALNODES TRANSPARENT
IDCFG
Format One or more of the following attribute/value pairs: BLOCKEDCREDLIST, CREATECRED, CREATECREDURL, REFRESHPERIOD, RESETCREDLIST or SERVER.
Default ---
Description

This parameter enables the identity manager interface allowing credential, policy, and usage information to be shared with an external information service.

Only one identity manager can be configured at a time.

Example
IDCFG[info] SERVER=exec:///usr/local/bin/dbquery.pl REFRESHPERIOD=30:00

Moab will refresh credential info every half hour using the STDOUT of the specified script.

IGNOREMDATASTAGING
Format <BOOLEAN>
Default FALSE
Description When set to TRUE, Moab will ignore any resource manager specific data staging on a job and assume the resource manager is processing the request. Currently, this only applies to PBS.
Example
IGNORERMDATASTAGING TRUE
IGNORECLASSES
Format [!]<CLASS>[,<CLASS>]...
Default ---
Description By default, if using the TORQUE resource manager, jobs from all listed classes are ignored and not scheduled, tracked, or otherwise processed by Moab. If the not(i.e., '!') character is specified, only jobs from listed classes are processed. See the Moab Side-by-Side Analysis for more information.
Example
IGNORECLASSES dque,batch

Moab will ignore jobs from classes dque and batch.

IGNOREJOBS
Format [!]<JOBID>[,<JOBID>]...
Default ---
Description By default, listed jobs are ignored and not scheduled, tracked, or otherwise processed by Moab. If the not(i.e., '!') character is specified, only listed jobs are processed. See the Moab Side-by-Side Analysis for more information.
Example
IGNOREJOBS !14221,14223

Moab will ignore jobs all classes except 14221 and 14223.

IGNORENODES
Format [!]<NODE>[,<NODE>]...
Default ---
Description By default, all listed nodes are ignored and not scheduled, tracked, or otherwise processed by Moab. If the not(i.e., '!') character is specified, only listed nodes are processed. See the Moab Side-by-Side Analysis for more information.
Example
IGNORENODES !host3,host4

Moab will only process nodes host3 and host4.

IGNOREPREEMPTEEPRIORITY
Format <BOOLEAN>
Default FALSE
Description By default, preemptor jobs can only preempt preemptee jobs if the preemptor has a higher job priority than the preemptee. When this parameter is set to true, the priority constraint is removed allowing any preemptor to preempt any preemptees once it reaches the top of the eligible job queue.
Example
IGNOREPREEMPTEEPRIORITY TRUE

A preemptor job can preempt any preemptee jobs when it is at the top of the eligible job queue.

IGNOREUSERS
Format [!]<USERNAME>[,<USERNAME>]...
Default ---
Description By default, if using the TORQUE resource manager, jobs from all listed users are ignored and not scheduled, tracked, or otherwise processed by Moab. If the not(i.e., '!') character is specified, only jobs from listed users are processed. (See the Moab Side-by-Side Analysis for more information.)
Example
IGNOREUSERS testuser1,annapolis

Moab will ignore jobs from users testuser1 and annapolis.

#INCLUDE
Format <STRING>
Default ---
Description Specifies another file which contains more configuration parameters. If <STRING> is not an absolute path, Moab will search its home directory for the file.
Example
#INCLUDE moab.acct

Moab will process the parameters in moab.acct as well as moab.cfg

INSTANTSTAGE
Description

This parameter is deprecated. Use JOBMIGRATEPOLICY.

INVALIDFSTREEMSG
Format "<STRING>"
Default "no valid fstree node found"
Description Specifies the error message that should be attached to jobs that cannot run because of a fairshare tree configuration violation.
Example
INVALIDFSTREEMSG        "account is invalid for requested partition"
JOBACTIONONNODEFAILURE
Format CANCEL, FAIL, HOLD, IGNORE, NOTIFY, or REQUEUE
Default ---
Description Specifies the action to take if Moab detects that a node allocated to an active job has failed (state is down). By default, Moab only reports this information via diagnostic commands. If this parameter is set, Moab will cancel or requeue the active job. See Reallocating Resources When Failures Occur for more information.

Note: The HOLD value is only applicable when using checkpointing.
Example
JOBACTIONONNODEFAILURE REQUEUE

Moab will requeue active jobs which have allocated nodes which have failed during the execution of the job.

JOBAGGREGATIONTIME
Format [[[DD:]HH:]MM:]SS
Default 0
Description

Specifies the minimum amount of time the scheduler should wait after receiving a job event until it should process that event. This parameter allows sites with bursty job submissions to process job events in groups decreasing total job scheduling cycles and allowing the scheduler to make more intelligent choices by aggregating job submissions and choosing between the jobs. See Considerations for Large Clusters.

 

Example
JOBAGGREGATIONTIME 00:00:04
RMPOLLINTERVAL     30,30

Moab will wait 4 seconds between scheduling cycles when job events have been received and will wait 30 seconds between scheduling cycles otherwise.

JOBCFG
Format <ATTR>=<VAL> where <ATTR> is one of FLAGS, GRES, NODERANGE, PRIORITYF, PROCRANGE, QOS, RARCH, RFEATURES, ROPSYS, SELECT, or TARGETLOAD
Default ---
Description Specifies attributes for jobs which satisfy the specified profile. The SELECT attribute allows users to specify the job template by using msub -l template=.

The JOBCFG parameter supports the following attributes:

NONE, ACCOUNT, ACTION, AUTOSIZE, CLASS, CPULIMIT, DESCRIPTION, DGRES, FAILUREPOLICY, GROUP, IFLAGS, JOBSCRIPT MEM (for MEM=<value>), MEMORY (for MEMORY=$LEARN), NODEACCESSPOLICY, NODEMOD, PARTITION, PREF, QOS, RESTARTABLE, RM, RMSERVICEJOB, SELECT, SOFTWARE, SRM, TEMPLIMIT, TFLAGS, USER, VMUSAGE, WALLTIME, WORK

It also supports the following Wiki attributes:

ARGS, DMEM, DDISK, DWAP, ERROR, EXEC, EXITCODE, GATTR, GEVENT, IWD, JNAME, NAME, PARTITIONMASK, PRIORITYF, RDISK, RSWAP, RAGRES, RCGRES, TASKPERNODE, TRIGGER, VARIABLE, NULL

Note: The index to the JOBCFG parameter can either be an admin-chosen job template name or the exact name of job reported by one or more workload queries. See Wiki Attributes and Job Template Extensions.

Example
JOBCFG[sql] RFEATURES=sqlnode QOS=service

When the sql job is detected, it will have the specified default qos and node feature attributes set.

JOBCPURGETIME
Format [[[DD:]HH:]MM:]SS
Default 00:05:00
Description Specifies the amount of time Moab will preserve detailed information about a completed job (see showq -c and checkjob).
Example
JOBCPURGETIME 02:00:00

Moab will maintain detailed job information for 2 hours after a job has completed.

JOBCTRUNCATENLCP
Format <BOOLEAN>
Default TRUE
Description Specifies whether Moab will store only the first node of the node list for a completed job in the checkpoint file.
Example
JOBCTRUNCATENLCP TRUE

JOBCTRUNCATENLCP reduces the amount of memory Moab uses to store completed job information.

JOBEXTENDSTARTWALLTIME
Format <BOOLEAN>
Default ---
Description

Extends the job walltime when Moab starts the job up to the lesser of the maximum or the next reservation (rounded down to the nearest minute).

JOBEXTENDSTARTWALLTIME TRUE and JOBEXTENDDURATION cannot be configured together. If they are in the same moab.cfg or are both active, then the JOBEXTENDDURATION will not be honored.

Example
JOBEXTENDSTARTWALLTIME TRUE

Submit job with a minimum wallclock limit and a walltime; for example:

echo sleep 500 | msub -A ee -l
nodes=5,minwclimit=5:00,walltime=30:00,partition=g02

At job start, Moab recognizes the nodes assigned to the specified job and extends the walltime for the job (one time at job start) up to the lesser of the maximum walltime requested or the least amount of time available for any of the nodes until the next reservation on that node.

JOBFAILRETRYCOUNT
Format <INTEGER>
Default 0
Description

Specifies the number of times a job is requeued and restarted by Moab if the job fails (if the job itself returns a non-zero exit code). Some types of jobs may succeed if automatically retried several times in short succession. This parameter was created with these types of jobs in mind. Note that the job in question must also be restartable (the job needs to have the "RESTARTABLE" flag set on it) and the RM managing the job must support requeuing and starting completed jobs.

If a job fails too many times, and reaches the number of retries given by JobFailRetryCount, then a UserHold is placed on the job and a message is attached to it signifying that the job has a "restart count violation."

Example
JOBFAILRETRYCOUNT  7

Any job with a RESTARTABLE flag is requeued, if it fails, up to 7 times before a UserHold is placed on it.

JOBIDWEIGHT
Format <INTEGER>
Default ---
Description Specifies the weight to be applied to the job's id. See Attribute (ATTR) Factor.
Example
JOBIDWEIGHT -1

Later jobs' priority will be negatively affected.

JOBMATCHCFG
Format <ATTR>=<VAL> where <ATTR> is one of JMIN, JMAX, JDEF, JSET, or JSTAT
Default ---
Description Specifies the job templates which must be matched and which will be applied in the case of a match.
Example
JOBMATCHCFG[sql] JMIN=interactive JSTAT=istat
JOBMAXHOLDTIME
Format [[[DD:]HH:]MM:]SS
Default ---
Description Specifies the amount of time a job can be held before it is canceled automatically.
Example
JOBMAXHOLDTIME 02:00:00

Moab will keep jobs in any HOLD state for 2 hours before canceling them.

JOBMAXNODECOUNT
Format <INTEGER>
Default 1024
Description Specifies the maximum number of nodes which can be allocated to a job. After changing this parameter, Moab must be restarted. Note: This value cannot exceed either MMAX_NODE or MMAX_TASK_PER_JOB. If larger values are required, these values must also be increased. Moab must be restarted before changes to this command will take effect. The command mdiag -S will indicate if any job node count overflows have occurred. See Consideration for Large Clusters.
Example
JOBMAXNODECOUNT 4000
JOBMAXOVERRUN
Format [[[[DD:]HH:]MM:]SS,][[[DD:]HH:]MM:]SS
Default (no soft limit), 10 minutes (hard limit)
Description

Soft and hard limit of the amount of time Moab will allow a job to exceed its wallclock limit before it first sends a mail to the primary admin (soft limit) and then terminates the job (hard limit). See WCVIOLATIONACTION or Usage-based Limits.

Note

If you run Moab with the TORQUE resource manager, you must set the $ignwalltime parameter to true in the /var/spool/torque/mom_priv/config file, otherwise the pbs_mom will kill any job that exceeds its walltime. See the TORQUE documentation for more information.

Example
JOBMAXOVERRUN 15:00,1:00:00

Jobs may exceed their wallclock limit by up to 1 hour, but Moab will send an email to the primary administrator when a job exceeds its walltime by 15 minutes.

JOBMAXPREEMPTPERITERATION
Format <INTEGER>
Default 0 (No Limit)
Description Maximum number of jobs allowed to be preempted per iteration.
Example
JOBMAXPREEMPTPERITERATION 10
JOBMAXSTARTPERITERATION
Format <INTEGER>
Default 0 (No Limit)
Description Maximum number of jobs allowed to start per iteration.
Example
JOBMAXSTARTPERITERATION 10
JOBMAXSTARTTIME
Format [[[DD:]HH:]MM:]SS
Default -1 (NO LIMIT)
Description length of time a job is allowed to remain in a 'starting' state. If a 'started' job does not transition to a running state within this amount of time, Moab will cancel the job, believing a system failure has occurred.
Example
JOBMAXSTARTTIME 2:00:00

Jobs may attempt to start for up to 2 hours before being canceled by the scheduler

JOBMAXTASKCOUNT
Format <INTEGER>
Default 4096
Description Specifies the total number of tasks allowed per job.
Example
JOBMAXTASKCOUNT 226000
JOBMIGRATEPOLICY
Format One of the following: IMMEDIATE, JUSTINTIME, or AUTO
Default AUTO
Description Upon using the msub command to submit a job, you can allocate the job to immediately (IMMEDIATE) migrate to the resource manager, or you can instruct Moab to only migrate the job to the resource manager when it is ready to run (JUSTINTIME). Specifying AUTO allows MOAB to determine on a per-job basis whether to use IMMEDIATE or JUSTINTIME.
Example
JOBMIGRATEPOLICY JUSTINTIME
JOBNAMEWEIGHT
Format <INTEGER>
Default ---
Description Specifies the weight to be applied to the job's name if the Name contains an integer. See Attribute (ATTR) Factor.
Example
JOBNAMEWEIGHT 1
JOBNODEMATCHPOLICY
Format EXACTNODE or EXACTPROC
Default ---
Description Specifies additional constraints on how compute nodes are to be selected. EXACTNODE indicates that Moab should select as many nodes as requested even if it could pack multiple tasks onto the same node. EXACTPROC indicates that Moab should select only nodes with exactly the number of processors configured as are requested per node even if nodes with excess processors are available.
Example
JOBNODEMATCHPOLICY EXACTNODE

In a PBS/Native job with resource specification nodes=<x>:ppn=<y>, Moab will allocate exactly <y> task on each of <x> distinct nodes.

JOBPREEMPTMAXACTIVETIME
Format [[[DD:]HH:]MM:]SS
Default 0
Description The amount of time in which a job may be eligible for preemption. See Job Preemption.
Example
JOBPREEMPTMAXACTIVETIME 00:05:00

A job is preemptable for the first 5 minutes of its run time.

JOBPREEMPTMINACTIVETIME
Format [[[DD:]HH:]MM:]SS
Default 0
Description The minimum amount of time a job must be active before being considered eligible for preemption. See Job Preemption.
Example
JOBPREEMPTMINACTIVETIME 00:05:00

A job must execute for 5 minutes before Moab will consider it eligible for preemption.

JOBPRIOACCRUALPOLICY
Format ACCRUE or RESET
Default ACCRUE
Description

Specifies how Moab should track the dynamic aspects of a job's priority. ACCRUE indicates that the job will accrue queuetime based priority from the time it is submitted unless it violates any of the policies not specified in JOBPRIOEXCEPTIONS. RESET indicates that it will accrue priority from the time it is submitted unless it violates any of the JOBPRIOEXCEPTIONS. However, with RESET, if the job does violate JOBPRIOEXCEPTIONS then its queuetime based priority will be reset to 0.

Note

JOBPRIOACCRUALPOLICY is a global parameter, but can be configured to work only in QOSCFG:

QOSCFG[arrays]  JOBPRIOACCRUALPOLICY=ACCRUE

The following old JOBPRIOACCRUALPOLICY values have been deprecated and should be adjusted to the following values:

  • QUEUEPOLICY= ACCRUE and JOBPRIOEXCEPTIONSSOFTPOLICY,HARDPOLICY
  • QUEUEPOLICYRESET= RESET and JOBPRIOEXCEPTIONSSOFTPOLICY,HARDPOLICY
  • ALWAYS= ACCRUE and JOBPRIOEXCEPTIONSALL
  • FULLPOLICY= ACCRUE and JOBPRIOEXCEPTIONSNONE
  • FULLPOLICYRESET= RESET and JOBPRIOEXCEPTIONSNONE
Example
JOBPRIOACCRUALPOLICY   RESET

Moab will adjust the job's dynamic priority subcomponents, i.e., QUEUETIME, XFACTOR, and TARGETQUEUETIME, etc. each iteration that the job does not violate any JOBPRIOEXCEPTIONS, if it is found in violation, its queuetime will be reset to 0.

JOBPRIOEXCEPTIONS
Format Comma delimited list of any of the following: DEFER, DEPENDS, SOFTPOLICY, HARDPOLICY, IDLEPOLICY, USERHOLD, BATCHHOLD, and SYSTEMHOLD (ALL or NONE can also be specified on their own)
Default NONE
Description

Specifies exceptions for calculating a job's dynamic priority (QUEUETIME, XFACTOR, TARGETQUEUETIME). Normally, when a job violates a policy, is placed on hold, or has an unsatisfied dependency, it will not accrue priority. Exceptions can be configured to allow a job to accrue priority in spite of any of these violations. With DEPENDS a job will increase in priority even if there exists an unsatisfied dependency. With SOFTPOLICY, HARDPOLICY, or IDLEPOLICY a job can accrue priority despite violating a specific limit. With DEFER, USERHOLD, BATCHHOLD, or SYSTEMHOLD a job can accrue priority despite being on hold.

Note

JOBPRIOEXCEPTIONS is a global parameter, but can be configured to work only in QOSCFG:

QOSCFG[arrays]  JOBPRIOEXCEPTIONS=IDLEPOLICY

Example
JOBPRIOEXCEPTIONS BATCHHOLD,SYSTEMHOLD,DEPENDS

Jobs will accrue priority in spite of batchholds, systemholds, or unsatisfied dependencies.

JOBPRIOF
Format <ATTRIBUTE>[<VALUE>]=<PRIORITY> where <ATTRIBUTE> is one of ATTR, GRES or STATE
Default ---
Description Specifies attribute priority weights for jobs with specific attributes, generic resource requests, or states. State values must be one of the standard Moab job states. See Attribute-Based Job Prioritization.
Example
JOBPRIOF         STATE[Running]=100  STATE[Suspended]=1000  ATTR[PREEMPTEE]=200  GRES[biocalc]=5
ATTRATTRWEIGHT   1
ATTRSTATEWEIGHT  1

Moab will adjust the job's dynamic priority subcomponents.

JOBPURGETIME
Format [[[DD:]HH:]MM:]SS
Default 0 (purge immediately if the resource manager does not report the job)
Description The amount of time Moab will keep a job record which is no longer reported by the resource manager. Useful when using a resource manager which drops information about a job due to internal failures. See JOBCPURGETIME.
Example
JOBPURGETIME 00:05:00

Moab will maintain a job record for 5 minutes after the last update regarding that object received from the resource manager.

JOBREJECTPOLICY
Format One or more of CANCEL, HOLD, IGNORE (beta), MAIL, or RETRY
Default HOLD
Description Specifies the action to take when the scheduler determines that a job can never run. CANCEL issues a call to the resource manager to cancel the job. HOLD places a batch hold on the job preventing the job from being further evaluated until released by an administrator. ( Note: Administrators can dynamically alter job attributes and possibly fix the job with mjobctl -m.) With IGNORE(currently in beta), the scheduler will allow the job to exist within the resource manager queue but will neither process it nor report it. MAIL will send email to both the admin and the user when rejected jobs are detected. If RETRY is set, then Moab will allow the job to remain idle and will only attempt to start the job when the policy violation is resolved. Any combination of attributes may be specified. See QOSREJECTPOLICY.
Example
JOBREJECTPOLICY  MAIL,CANCEL
JOBREMOVEENVVARLIST
Format Comma-delimited list of strings
Default ---
Description

Moab will remove the specified environment variables from the job's environment before migrating the job to its destination resource manager. This is useful when jobs submit themselves from one cluster to another with the full environment.

Note

This parameter is currently only supported with TORQUE resource managers.

Example
JOBREMOVEENVVARLIST PBS_SERVER,TZ

Moab will remove the environment variables PBS_SERVER and TZ before submitting jobs.

JOBRETRYTIME
Format [[[DD:]HH:]MM:]SS
Default 00:00:60
Description Period of time Moab will continue to attempt to start a job which has failed to start due to transient failures or which has successfully started and was then rejected by the resource manager due to transient failures. (For related information, see Reservation Policies, DEFERSTARTCOUNT, DEFERTIME, RESERVATIONRETRYTIME, NODEFAILURERESERVETIME, and GUARANTEEDPREEMPTION.)
Example
JOBRETRYTIME   00:05:00

Moab will try for up to 5 minutes to restart jobs if the job start has failed due to transient errors.

LIMITEDJOBCP
Format <BOOLEAN>
Default FALSE
Description Specifies whether there should be limited job checkpointing (see Consideration for Large Clusters). With LIMITEDJOBCP enabled, Moab will only checkpoint a job if it is modified with mjobctl or if it has been submitted with msub but has not migrated. In all other cases, Moab does not checkpoint the job and all Moab-specific information (such as messages attached to the job) is lost. No TORQUE-specific information will be lost.
Example
LIMITEDJOBCP TRUE

Moab will only maintain scheduler checkpoint information for jobs with explicitly modified job attributes. Some minor job performance and usage statistics may be lost.

LIMITEDNODECP
Format <BOOLEAN>
Default FALSE
Description Specifies whether there should be limited node checkpointing (see Consideration for Large Clusters).
Example
LIMITEDNODECP TRUE

Moab will only maintain scheduler checkpoint information for nodes with explicitly modified job attributes. (some minor node performance and usage statistics may be lost)

LOADALLJOBCP
Format <BOOLEAN>
Default FALSE
Description Specifies whether Moab should load, during startup, all non-completed jobs in the checkpoint files regardless of whether or not their corresponding resource managers are active. For example, this allows source peers to continue showing remote jobs in the queue based on checkpointed info, even though the destination peer is offline.
Example
LOADALLJOBCP TRUE

Moab will load, at startup, all non-completed jobs from all checkpoint files.

LOCKFILE
Format <STRING>
Default ---
Description Specifies the path for the lock (pid) file used by Moab.
Example
LOCKFILE /var/spool/moab/lock
LOGDIR
Format <STRING>
Default log
Description Specifies the directory in which log files will be maintained. If specified as a relative path, LOGDIR will be relative to $(MOABHOMEDIR) See Logging Overview for more information.
Example
LOGDIR /var/spool/moab

Moab will record its log files directly into the /var/spool/moab directory

LOGFACILITY
Format Colon delimited list of one or more of the following: CORE, SCHED, SOCK, UI, LL, CONFIG, STAT, SIM, STRUCT, FS, CKPT, BANK, RM, PBS, WIKI, ALL
Default ALL
Description Specifies which types of events to log (see Logging Overview).
Example
LOGFACILITY RM:PBS

Moab will log only events involving general resource manager or PBS interface activities.

LOGFILE
Format <STRING>
Default moab.log
Description Name of the Moab log file. This file is maintained in the directory pointed to by <LOGDIR> unless <LOGFILE> is an absolute path (see Logging Overview)
Example
LOGFILE moab.test.log

Log information will be written to the file moab.test.log located in the directory pointed to by the LOGDIR parameter.

LOGFILEMAXSIZE
Format <INTEGER>
Default 10000000
Description Maximum allowed size (in bytes) of the log file before it will be rolled (see Logging Overview).
Example
LOGFILEMAXSIZE 50000000

Log files will be rolled when they reach 50 MB in size

LOGFILEROLLDEPTH
Format <INTEGER>
Default 3
Description Number of old log files to maintain (i.e., when full, moab.log will be renamed moab.log.1, moab.log.1 will be renamed moab.log.2, ...). See Logging Overview.
Example
LOGFILEROLLDEPTH 5

Moab will maintain and roll the last 5 log files.

LOGLEVEL
Format <INTEGER> (0-9)
Default 0
Description Specifies the verbosity of Moab logging where 9 is the most verbose ( Note: each logging level is approximately an order of magnitude more verbose than the previous level). See Logging Overview.
Example
LOGLEVEL 4

Moab will write all Moab log messages with a threshold of 4 or lower to the moab.log file.

LOGLEVELOVERRIDE
Format <BOOLEAN>
Default FALSE
Description When this parameter is on, if someone runs a command with --loglevel=<x>, that loglevel, if higher than the current loglevel, is used on the scheduler side for the duration of the command. All logs produced during that time are put into a separate log file (this creates a "gap" in the normal logs). This can be very useful for debugging, but it is recommend that this be used only when diagnosing a specific problem so that users can't affect performance by submitting multiple --loglevel commands.
Note

This parameter does not work with threaded commands (such as showq, mdiag -n, and mdiag -j).

Example
LOGLEVELOVERRIDE    TRUE
LOGPERMISSIONS
Format <INTEGER>
Default 644
Description Specifies the octal number that represents read, write, and execute permissions.
Example
LOGPERMISSIONS 600

Allows the file owner to read and write permissions, but denies rights to the group and others.

LOGROLLACTION
Format <STRING>
Default ---
Description Specifies a script to run when the logs roll. The script is run as a trigger and can be viewed using mdiag -T. For example, a script can be specified that always moves the first rolled log file, moab.log.1, to an archive directory for longer term storage.
Example
LOGROLLACTION /usr/local/tools/logroll.pl
MAILPROGRAM
Format [<Full_Path_To_Mail_Command> | DEFAULT | NONE ][@<DEFAULTMAILDOMAIN>]
Default NONE
Description

If set to NONE, no mail is sent. If set to DEFAULT, Moab sends mail via the system's default mail program (usually /usr/bin/sendmail). If set to the local path of a mail program, Moab uses the specified mail program to send mail.

By default, Moab mail notification is disabled. To enable, you must set MAILPROGRAM to DEFAULT or specify some other locally available mail program. If the default mail domain is set, emails will be routed to this domain unless a per-user domain is specified using the EMAILADDRESS attribute of the USERCFG parameter. If neither of these values is set, Moab uses "@localhost" as the mail domain. See Notify Admins.

For jobs, the email address used on the msub -M option overrides all other user email addresses. Additionally, administrators are notified in the case of job violations.

Example
MAILPROGRAM DEFAULT

Moab sends mail via the system's default mail program, /usr/bin/sendmail.

MAILPROGRAM /usr/local/bin/[email protected]

Moab sends mail via the mail program located at /usr/local/bin/sendmail with default mail domain @mydomain.com

MAXGRES
Format <INTEGER>
Default 512
Description Specifies how many generic resources Moab should manage.
Example
MAXGRES 1024
MAXGMETRIC
Format <INTEGER>
Default 10
Description Specifies how many generic metrics Moab should manage.
Example
MAXGMETRIC 20
MAXJOB
Format <INTEGER>
Default 4096
Description

Specifies the maximum quantity of jobs for which Moab should allocate memory used for tracking jobs. If Moab is tracking the maximum quantity of jobs specified by this parameter, it rejects subsequent jobs submitted by any user since it has no memory left with which to track newly submitted jobs.

If a user submitted a job with the msub command, this rejection behavior requires the user to resubmit the job at a later time after other jobs have completed, which frees memory in which Moab can place later-submitted jobs. If a user submitted a job with the TORQUE qsub command, TORQUE will automatically resubmit the job to Moab until Moab accepts it.

The mdiag -S command indicates if any job overflows have occurred.

If this parameter's value is changed, it does not go into effect until Moab restarts. Moab reads the parameter only on initial startup and uses its value to allocate the memory it uses to track jobs.

Example
MAXJOB 45000
MAXNODE
Format <INTEGER>
Default 5120
Description

Specifies the maximum number of compute nodes supported.

Example
MAXNODE 10000                                                   
MAXRSVPERNODE
Format <INTEGER>
Default 48
Description

Specifies the maximum number of reservations on a node.

For large SMP systems (>512 processors/node), Adaptive Computing advises adjusting the value to approximately twice the average sum of admin, standing, and job reservations present.

A second number, led by a comma, can also be specified to set a maximum number of reservations for nodes that are part of the SHARED partition.

The maximum possible value of MAXRSVPERNODE is 8192 for a global node and 4096 for any other node.
Moab must be restarted for any changes to this parameter to take effect. The command mdiag -S indicates whether any node reservation overflows have occurred. See Considerations for Large Clusters.

Warning

Do not lower the MAXRSVPERNODE value while there are active jobs in the queue. This can lead to queue instability and certain jobs could become stuck or disconnected from the system.

Example
MAXRSVPERNODE 64

64 is the maximum number of reservations on a single node.

MAXRSVPERNODE 100,7000

100 is the maximum number of reservations on a single node, and 7000 is the maximum number of reservations for global nodes.

MEMREFRESHINTERVAL
Format [[[DD:]HH:]MM:]:SS | job:<COUNT>
Default ---
Description Specifies the time interval or total job query count at which Moab will perform garbage collection to free memory associated with resource manager API's which possess memory leaks (i.e., Loadleveler, etc.).
Example
# free memory associated with leaky RM API
MEMREFRESHINTERVAL 24:00:00

Moab will perform garbage collection once every 24 hours.

MEMWEIGHT
Format <INTEGER>
Default 0
Description Specifies the coefficient to be multiplied by a job's MEM (dedicated memory in MB) factor. See Resource Priority Overview.
Example
RESWEIGHT 10
MEMWEIGHT 1000

Each job's priority will be increased by 10 * 1000 * <request memory>.

MESSAGEQUEUEADDRESS
Format The IP address of the machine on which Moab is generating events.
Default * (all)
Description When a user subscribes to the events Moab provides and delivers via zeroMQ, s/he must do so by specifying tcp://<ipAddress>:<port>. MESSAGEQUEUEADDRESS specifies the <ipAddress>, which must match the IP address of the machine on which Moab is installed. To specify the port, see MESSAGEQUEUEPORT.
Example
MESSAGEQUEUEADDRESS    10.1.0.10

To subscribe to Moab events, users must use tcp://10.1.0.10:<port>.

MESSAGEQUEUEPORT
Format The port of the machine on which Moab is generating events.
Default 5563
Description When a user subscribes to the events Moab provides and delivers via zeroMQ, s/he must do so by specifying tcp://<ipAddress>:<port>. MESSAGEQUEUEPORT specifies the <port>, which must match the port of the machine on which Moab is installed. To specify the IP address, see MESSAGEQUEUEADDRESS.
Example
MESSAGEQUEUEPORT    1010

To subscribe to Moab events, users must use tcp://<ipAddress>:1010.

MESSAGEQUEUESECRETKEY
Format <STRING>
Default ---
Description

Causes Moab to encrypt the events delivered via zeroMQ using the Advanced Encryption Standard (AES) algorithm. Must be a Base64-encoded, 128-bit (16-byte) key. Messages will be encrypted using AES in CBC mode where inputs are padded with PKCS5 padding. The initialization vector is calculated by using an MD5 hash of the key specified in MESSAGEQUEUESECRETKEY.

MESSAGEQUEUESECRETKEY can only be specified in the moab-private.cfg file.

Example
MESSAGEQUEUESECRETKEY 1r6RvfqJa6voezy5wAx0hw==
MINADMINSTIME
Format <INTEGER>
Default 60 seconds
Description Specifies the minimum time a job will be suspended if suspended by an administrator or by a scheduler policy.
Example
MINADMINSTIME 00:10:00

Each job suspended by administrators or policies will stay in the suspended state for at least 10 minutes.

MISSINGDEPENDENCYACTION
Format CANCEL, HOLD, or RUN
Default HOLD
Description Controls what Moab does with a dependent job when its dependency job cannot be found when Moab evaluates the dependent job for scheduling. This only affects jobs whose dependent job cannot be found.
Example
MISSINGDEPENDENCYACTION CANCEL

Any job that has a dependent job that cannot be found is canceled.

MSUBQUERYINTERVAL
Format <INTEGER>
Default 5 seconds
Description

Specifies the length of the interval (in seconds) between job queries when using msub -K. Jobs submitted with the -K option query the scheduler every MSUBQUERYINTERVAL seconds until the job is completed.

MSUBQUERYINTERVAL can exist as an environment variable. Any value in moab.cfg overrides the environment variable.

Note: If MSUBQUERYINTERVAL is set to 0, the -K option will be disabled. Jobs will still submit correctly, but the client will not continue to check on the job.

Example
MSUBQUERYINTERVAL  60

If a user uses the msub -K command, the client remains open and queries the server every 60 seconds until the job completes.

NODEACCESSPOLICY
Format

One of the following:

SHARED, SHAREDONLY, SINGLEJOB, SINGLETASK, SINGLEUSER, or UNIQUEUSER

Default SHARED
Description Specifies how node resources will be shared by various tasks (See the Node Access Overview for more information).
Example
NODEACCESSPOLICY SINGLEUSER

Moab will allow resources on a node to be used by more than one job provided that the jobs are all owned by the same user.

NODEALLOCATIONPOLICY
Format

One of the following:

FIRSTAVAILABLE, LASTAVAILABLE, MINRESOURCE, CPULOAD, LOCAL, CONTIGUOUS, MAXBALANCE, PRIORITY, or PLUGIN.

Default LASTAVAILABLE
Description

Specifies how Moab should allocate available resources to jobs. See Node Allocation Overview for more information.

If ENABLEHIGHTHROUGHPUT is TRUE, you must set NODEALLOCATIONPOLICY to FIRSTAVAILABLE.

Example
NODEALLOCATIONPOLICY MINRESOURCE

Moab will apply the node allocation policy MINRESOURCE to all jobs by default.

NODEALLOCRESFAILUREPOLICY
Format

One of the following:

CANCEL, HOLD, IGNORE, MIGRATE, NOTIFY, or REQUEUE

Default NONE
Description Specifies how Moab should handle active jobs which experience node failures during execution. See the RESFAILPOLICY resource manager extension or the Node Availability Overview.
Example
NODEALLOCRESFAILUREPOLICY REQUEUE

Moab will requeue jobs which have allocated nodes fail during execution.

NODEAVAILABILITYPOLICY
Format

<POLICY>[:<RESOURCETYPE>] ...

where <POLICY> is one of COMBINED, DEDICATED, or UTILIZED

and <RESOURCETYPE> is one of PROC, MEM, SWAP, or DISK

Default COMBINED
Description Specifies how available node resources are reported. Moab uses the following calculations to determine the amount of available resources:

Dedicated(use what Moab has scheduled to be used):
Available = Configured - Dedicated
Utilized(use what the resource manager is reporting is being used):
Available = Configured - Utilized
Combined(use the larger of dedicated and utilized):
Available = Configured - (MAX(Dedicated,Utilized))

Moab marks a node as busy when it has no available processors, so NODEAVAILABILTYPOLICY, by affecting how many processors are reported as available, also affects node state. See Node Availability Policies for more information.
Example
NODEAVAILABILITYPOLICY DEDICATED:PROCS COMBINED:MEM

Moab will ignore resource utilization information in locating available processors for jobs but will use both dedicated and utilized memory information in determining memory availability.

NODEBUSYSTATEDELAYTIME
Format [[[DD:]HH:]MM:]SS
Default 0:01:00 (one minute)
Description Length of time Moab will assume busy nodes will remain unavailable for scheduling if a system reservation is not explicitly created for the node.
Example
NODEBUSYSTATEDELAYTIME 0:30:00

Moab will assume busy nodes are not available for scheduling for at least 30 minutes from the current time. Thus, these nodes will never be allocated to starting jobs. Also, these nodes will only be available for reservations starting more than 30 minutes in the future.

NODECATCREDLIST
Format <LABEL>=<NODECAT>[,<NODECAT>]...[ <LABEL>=<NODECAT>[,<NODECAT>]...]...

where <LABEL> is any string and <NODECAT> is one of the defined node categories.

Default ---
Description If specified, Moab will generate node category groupings and each iteration will assign usage of matching resources to pseudo-credentials with a name matching the specified label. See the Node Categorization section of the Admin manual for more information.
Example
NODECATCREDLIST down=BatchFailure,HardwareFailure,NetworkFailure idle=Idle
                                                        

Moab will create a down user, group, account, class, and QoS and will associate BatchFailure, HardwareFailure, and NetworkFailure resources with these credentials. Additionally, Moab will assign all Idle resources to matching idle credentials.

NODECFG[X]
Format List of space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:
ACCESS, CHARGERATE, FEATURES, FLAGS, GRES, MAXJOB, MAXJOBPERUSER, MAXLOAD, MAXPE, NODEINDEX, NODETYPE, OSLIST, PARTITION, POWERPOLICY, PRIORITY, PRIORITYF, PROCSPEED, RACK, RADISK, SLOT, SPEED, or TRIGGER
Default ---
Description Specifies node-specific attributes for the node indicated in the array field. See the General Node Administration Overview for more information.
Example
NODECFG[nodeA] MAXJOB=2 SPEED=1.2

Moab will only allow 2 simultaneous jobs to run on node nodeA and will assign a relative machine speed of 1.2 to this node.

NODEDOWNSTATEDELAYTIME
Format [[[DD:]HH:]MM:]SS
Default -1 (never)
Description Length of time Moab will assume down, drained (offline), or corrupt nodes will remain unavailable for scheduling if a system reservation is not explicitly created for the node. The default specification of "-1" causes Moab to never create job reservations on down nodes. See Node Availability for more information.
Example
NODEDOWNSTATEDELAYTIME 0:30:00

Moab will assume down, drained, and corrupt nodes are not available for scheduling for at least 30 minutes from the current time. Thus, these nodes will never be allocated to starting jobs. Also, these nodes will only be available for reservations starting more than 30 minutes in the future.

NODEDOWNTIME
Format [[[DD:]HH:]MM:]SS
Default ---
Description The maximum time a previously reported node remains unreported by a resource manager before the node is considered to be in the down state. This can happen if communication with a resource manager or a peer server is lost for more than the specified length of time, or if there is communication with the resource manager but it fails to report the node status.
Example
NODEDOWNTIME 10:00

If Moab loses communication with the resource manager for more than 10 minutes, it sets the state of all nodes belonging to that resource manager to DOWN.

NODEDRAINSTATEDELAYTIME
Format [[[DD:]HH:]MM:]SS
Default 3:00:00 (three hours)
Description Length of time Moab will assume drained nodes will remain unavailable for scheduling if a system reservation is not explicitly created for the node. Specifying "-1" will cause Moab to never create job reservations on drained nodes. See Node Availability for more information.
Example
NODEDRAINSTATEDELAYTIME 0:30:00

Moab will assume down, drained, and corrupt nodes are not available for scheduling for at least 30 minutes from the current time. Thus, these nodes will never be allocated to starting jobs. Also, these nodes will only be available for reservations starting more than 30 minutes in the future.

NODEFAILURERESERVETIME
Format [[[DD:]HH:]MM:]SS
Default 0:05:00
Description Duration of reservation Moab will place on any node in which it detects a failure from the resource manager (0 indicates no reservation will be placed on the node). See Node Availability for more information. See also RMCFG[] NODEFAILURERSVPROFILE. (For related information, see Reservation Policies, DEFERSTARTCOUNT, DEFERTIME, RESERVATIONRETRYTIME, JOBRETRYTIME, and GUARANTEEDPREEMPTION.)
Example
NODEFAILURERESERVETIME 10:00

Moab will reserve failed nodes for 10 minutes.

NODEIDFORMAT
Format <STRING>
Default *$N*
Description Specifies how a node id can be processed to extract possible node, rack, slot, and cluster index information. The value of the parameter may include the markers $C (cluster index), $N (node index), $R (rack index), or $S (slot index) separated by *(asterisk - representing any number of non-numeric characters) or other characters to indicate this encoding. See Node Selection for more information on use of node, rack, and slot indices.
Example
NODEIDFORMAT *$R*$S

Moab will extract rack and slot information from the cluster node ids (i.e. tg-13s08).

NODEIDLEPOWERACTION
Format [STANDBY | SUSPEND | SLEEP | HIBERNATE | SHUTDOWN | OFF]
Default OFF
Description Specifies what to do with a node that exceeds the NODEIDLEPOWERTHRESHOLD limit.
Example
PARCFG[ALL] NODEIDLEPOWERACTION STANDBY

Nodes that exceed the NODEIDLEPOWERTHRESHOLD limit are placed in standby.

NODEIDLEPOWERTHRESHOLD
Format <INTEGER>
Default 60 seconds
Description Specifies how long to allow a node to be idle before performing a power action. Increasing the idle duration prevents power on/off thrashing.
Example
NODEIDLEPOWERTHRESHOLD 300

Moab will wait 5 minutes before performing a power action on a node that has become idle.

NODEIDLEPURGETIME
Format <seconds>
Default 0
Description

When dynamic nodes are created in Moab, they are usually created with a request ID.

NODEIDLEPURGETIME is the amount of time for all nodes with the same request ID to be idle before Moab begins firing the node end trigger for each iteration.

If one or more of the nodes with the same request ID becomes non-idle, Moab stops firing the node end trigger for all of the nodes with the same request ID until the NODEIDLEPURGETIME is once again met.

A value of 0 disables this feature.

Example
NODEIDLEPURGETIME 300
NODEMAXLOAD
Format <DOUBLE>
Default 0.0
Description Specifies that maximum cpu load on an idle or running node. If the node's load reaches or exceeds this value, Moab will mark the node busy.
Example
NODEMAXLOAD 0.75

Moab will adjust the state of all idle and running nodes with a load >= .75 to the state busy.

NODEMEMOVERCOMMITFACTOR
Format <DOUBLE>
Default ---
Description The parameter overcommits available and configured memory and swap on a node by the specified factor (for example: mem/swap * factor). Used to show that the node has more mem and swap than it really does. Only works for PBS RM types.
Example
NODEMEMOVERCOMMITFACTOR .5

Moab will overcommit the memory and swap of the node by a factor of 0.5.

NODEPOLLFREQUENCY
Format <INTEGER>
Default 0 (Poll Always)
Description Specifies the number of scheduling iterations between scheduler initiated node manager queries. If set to ' -2, Moab will never query the node manager daemons. If set to ' -1', Moab will only query on the first iteration. Note: this parameter is most often used with OpenPBS and PBSPro. It is not required when using TORQUE, LoadLeveler, LSF, or SGE as the resource managers.
Example
NODEPOLLFREQUENCY 5

Moab will update node manager based information every 5 scheduling iterations.

NODESETATTRIBUTE
Format FEATURE or VARATTR
Default ---
Description Specifies the type of node attribute by which node set boundaries will be established. See Node Set Overview.
Example
NODESETPOLICY     ONEOF
NODESETATTRIBUTE  FEATURE

Moab will create node sets containing nodes with common features.

NODESETDELAY
Format [[[DD:]HH:]MM:]SS
Default 0:00:00
Description

Causes Moab to attempt to span a job evenly across nodesets unless doing so delays the job beyond the requested NODESETDELAY.

Note

Must use with NODESETPLUS set to SPANEVENLY; if you do not want to use SPANEVENLY, use NODESETISOPTIONAL instead of NODESETDELAY.

Example
NODESETPLUS SPANEVENLY
NODESETDELAY     5:00

Moab tries to span the job evenly across nodesets unless doing so delays the job by 5 minutes.

NODESETISOPTIONAL
Format <BOOLEAN>
Default TRUE
Description Specifies whether or not Moab will start a job if a requested node set cannot be satisfied. See Node Set Overview.
Example
NODESETISOPTIONAL TRUE

Moab will not block a job from running if its node set cannot be satisfied.

NODESETLIST
Format <ATTR>[{ :,|}<ATTR>]...
Default ---
Description Specifies the list of node attribute values which will be considered for establishing node sets. See Node Set Overview.
Example
NODESETPOLICY     ONEOF
NODESETATTRIBUTE  FEATURE
NODESETLIST       switchA,switchB

Moab will allocate nodes to jobs either using only nodes with the switchA feature or using only nodes with the switchB feature.

NODESETPLUS
Format DELAY or SPANEVENLY
Default ---
Description

Specifies how Moab distributes jobs among nodesets. See Node Set Overview.

Neither SPANEVENLY nor DELAY values of the NODESETPLUS parameter will work with multi-req jobs or preemption.

Example
NODESETPLUS SPANEVENLY

Moab attempts to fit all jobs on a single nodeset or to span them evenly across a number of nodesets, unless doing so would delay a job beyond the requested NODESETDELAY.

NODESETPLUS DELAY

Moab attempts to schedule the job within a nodeset for the configured NODESETDELAY. If Moab cannot find space for the job to start within NODESETDELAY (Moab considers future workload to determine if space will open up in time and might create a future reservation), then Moab schedules the job and ignores the nodeset requirement.

NODESETPOLICY
Format ANYOF, FIRSTOF, or ONEOF
Default ---
Description Specifies how nodes will be allocated to the job from the various node set generated. See Node Set Overview.
Example
NODESETPOLICY     ONEOF
NODESETATTRIBUTE  NETWORK

Moab will create node sets containing nodes with common network interfaces.

NODESETPRIORITYTYPE
Format one of AFFINITY, BESTFIT, WORSTFIT, or MINLOSS
Default MINLOSS
Description Specifies how resource sets will be selected when more than one feasible resource can be found. See Node Set Overview.
Example
NODESETPRIORITYTYPE BESTFIT
NODESETATTRIBUTE    PROCSPEED

Moab will select the resource set that most closely matches the set of resources requested.

NODESYNCTIME
Format [[[DD:]HH:]MM:]SS
Default 00:10:00
Description Specifies the length of time after which Moab will sync up a node's expected state with an unexpected reported state. IMPORTANT Note: Moab will not start new jobs on a node with an expected state which does not match the state reported by the resource manager.
Example
NODESYNCTIME 1:00:00
NODETOJOBATTRMAP
Format Comma delimited list of node features
Default ---
Description Job requesting the listed node features will be assigned a corresponding job attribute. These job attributes can be used to enable reservation access, adjust job priority or enable other capabilities.
Example
NODETOJOBATTRMAP  fast,big

Jobs requesting node feature fast or big will be assigned a corresponding job attribute.

NODEUNTRACKEDRESDELAYTIME
Format [[[DD:]HH:]MM:]SS
Default 0:00:00
Description

Length of time Moab will assume untracked generic resources will remain unavailable for scheduling if a system reservation is not explicitly created for the node.

If NODEUNTRACKEDRESDELAYTIME is enabled and there is an untracked resource preventing a job from running, then the job remains in the idle queue instead of being deferred.

Example
NODEUNTRACKEDRESDELAYTIME 0:30:00

Moab will assume untracked generic resources are not available for scheduling for at least 30 minutes from the current time. Thus, these nodes will never be allocated to starting jobs. Also, these nodes will only be available for reservations starting more than 30 minutes in the future.

NODEVMFEATURECHECKTIME
Format [[[DD:]HH:]MM:]SS
Default 0:10:00
Description The length of time between each Moab check on node and VM features. If a running VM requires a feature but the resource manager is no longer reporting that feature on the VM's host node, Moab migrates the VM to a node that has the feature. If no other node has that feature, no migration occurs.
Example
NODEVMFEATURECHECKTIME 10:00

Moab checks node and VM features every 10 minutes.

NODEVMREQATTRCHECKTIME
Format [[[DD:]HH:]MM:]SS
Default 0:10:00
Description The length of time between each Moab check on a VM's requested node attributes. If a running VM requires node attributes but the resource manager is no longer reporting requested attributes on the VM's host node, Moab migrates the VM to a node that has the requested attributes. If no other node has the requested attributes, no migration occurs.
Example
NODEVMREQATTRCHECKTIME 10:00

Moab checks requested node attributes of a node running a VM every 10 minutes.

NODEWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight which will be applied to a job's requested node count before this value is added to the job's cumulative priority. Note: this weight currently only applies when a nodecount is specified by the user job. If the job only specifies tasks or processors, no node factor will be applied to the job's total priority. This will be rectified in future versions.
Example
NODEWEIGHT 1000
NOLOCALUSERENV
Format <BOOLEAN>
Default FALSE
Description If TRUE, specifies that a user's UserID, GroupID, and HomeDirectory are available on the Moab server host.
Example
NOLOCALUSERENV TRUE
NOJOBHOLDNORESOURCES
Format <BOOLEAN>
Default FALSE
Description If TRUE, Moab does not place a hold on jobs that don't have feasible resources. For example, suppose there are 20 processors available for ClassA and 50 processors for the entire system. If a job requests 21 or more processors from ClassA, or 51 or more processors from the entire system, Moab idles the job (instead of putting a hold on it) until the resources become available.
Example
NOJOBHOLDNORESOURCES TRUE
NOTIFICATIONPROGRAM
Format <STRING>
Default ---
Description Specifies the name of the program to handle all notification call-outs.
Example
NOTIFICATIONPROGRAM   tools/notifyme.pl
NOWAITPREEMPTION
Format <BOOLEAN>
Default ---
Description Generally when a job is trying to preempt another, it just waits for the original jobs that it chose to preempt to end. If this parameter is on, the preemptor will continue trying to preempt jobs until it can get in.
Example
NOWAITPREEMPTION   TRUE
OSCREDLOOKUP
Format NEVER
Default ---
Description

Disables all Moab OS credential lookups, including UID, GID, user to group mappings, and any other OS specific information.

Setting OSCREDLOOKUP by itself does not allow job submission; additional configuration is required. When submitting jobs from user accounts that do not exist on the head node (where Moab HPC Suite and TORQUE are running), you must also set the PROXYJOBSUBMISSION flag in addition to specifying configuration settings in the resource manager configuration file. See the example that follows for information on required resource manager settings.

Example
OSCREDLOOKUP NEVER
RMCFG[] FLAGS=PROXYJOBSUBMISSION

To allow job submission, in the TORQUE configuration file (torque.cfg):

VALIDATEPATH FALSE

Run the following qmgr directive:

set server disable_server_id_check = True

Restart both Moab HPC Suite and pbs_server.

PARALLOCATIONPOLICY
Format One of BestFit, BestFitP, FirstStart, LoadBalance, LoadBalanceP, Random, or RoundRobin
Default FirstStart
Description Specifies the approach to use to allocate resources when more than one eligible partition can be found. See Grid Scheduling Policies for more information.
Example
PARALLOCATIONPOLICY  LOADBALANCE

New jobs will be started on the most lightly allocated partition.

PARCFG
Format NODEPOWEROFFDURATION, NODEPOWERONDURATION, NODEALLOCATIONPOLICY or one or more key-value pairs as described in the Partition Overview
Default ---
Description Specifies the attributes, policies, and constraints for the given partition.
Example
PARCFG[oldcluster] MAX.WCLIMIT=12:00:00

Moab will not allow jobs to run on the oldcluster partition which has a wallclock limit in excess of 12 hours.

PBSACCOUNTINGDIR
Format <PATH>
Default ---
Description When specified, Moab will write out job events in standard PBS/ TORQUE tracejob format to the specified directory using the standard PBS/TORQUE log file naming convention.
Example
PBSACCOUNTINGDIR /var/spool/torque/sched_logs/

Job events will be written to the specified directory (can be consumed by PBS's tracejob command).

PERPARTITIONSCHEDULING
Format <BOOLEAN>
Default FALSE
Description

By default Moab's scheduling routine schedules each job on each partition using the following algorithm:

prioritize

foreach (job)

     find the partition on which that job should run

     schedule job

In this model, a job's priority is the same on each partition as it uses a single global priority. Because a job's priority is the same on every partition, Moab prioritizes the queue once and then schedules the prioritized queue across all partitions.

When PERPARTITIONSCHEDULING TRUE is set, the following algorithm is used:

foreach (partition)

     prioritize

     foreach (job)

          schedule job

In this case, each partition may have a unique priority configuration and Moab will re-prioritize the jobs for each partition on the system. Each job is prioritized and scheduled on each partition. See PARCFG for more information. Also, note that Moab will order the partitions as they are discovered in the moab.cfg file. Partitions should be explicitly ordered via PARCFG in the moab.cfg file.

Example
PERPARTITIONSCHEDULING TRUE
PARCFG[p1] CONFIGFILE=/opt/moab/etc/p1.cfg
PARCFG[p2] CONFIGFILE=/opt/moab/etc/p2.cfg

Rather than prioritizing the job queue once, Moab prioritizes the job queue for each partition, p1 and p2 respectively, and schedules each partition in turn using the policies located in their respective configuration files. (See Per-Partition Settings for more information).

PEWEIGHT
Format <INTEGER>
Default 0
Description Specifies the coefficient to be multiplied by a job's PE (processor equivalent) priority factor.
Example
RESWEIGHT 10
PEWEIGHT  100

Each job's priority will be increased by 10 * 100 * its PE factor.

PREEMPTPOLICY
Format one of the following:
CANCEL, REQUEUE, SUSPEND, or CHECKPOINT
Default REQUEUE
Description Specifies how preemptable jobs will be preempted.

Note: If this policy is set to REQUEUE, preemptible jobs should be marked as RESTARTABLE. If this policy is set to SUSPEND, preemptible jobs should be marked as SUSPENDABLE. Note: Moab uses preemption escalation to preempt resources if the specified preemption facility is not applicable. This means if the policy is set to SUSPEND and the job is not SUSPENDABLE, Moab may attempt to requeue or even cancel the job.

Example
PREEMPTPOLICY CHECKPOINT

Jobs that are to be preempted will be checkpointed and restarted at a later time.

PREEMPTPRIOJOBSELECTWEIGHT
Format <DOUBLE>
Default 256.0
Description

Determines which jobs to preempt based on size or priority. The higher the value, the more emphasis is placed on the priority of the job, causing the lower priority jobs to be preempted first. The lower the value, the more emphasis is placed on the size of the job, causing the smaller jobs to be preempted first. If set to 0, job priority will be ignored, job size will take precedence and the smallest jobs will be preempted.

The special setting of -1 places the emphasis solely on resource utilization. This means that jobs will be preempted in a manner that keeps the resource utilization at the highest level, regardless of job priority or size.

Example
PREEMPTPRIOJOBSELECTWEIGHT 220.5
PREEMPTRTIMEWEIGHT
Format <DOUBLE>
Default 0
Description If set to anything other than 0, a job's remaining time is added into the calculation of which jobs will be preempted. If a positive weight is specified, jobs with a longer remaining time are favored. If a negative weight is specified, jobs with a shorter remaining time are favored.
Example
PREEMPTRTWEIGHT 1.5
PREEMPTSEARCHDEPTH
Format <INTEGER>
Default unlimited
Description Specifies how many preemptible jobs will be evaluated as potential targets for serial job preemptors. See Preemption Overview for more information.
Example
PREEMPTSEARCHDEPTH 8

Serial job preemptors will only consider the first 8 feasible preemptee jobs when determining the best action to take.

PRIORITYTARGETDURATION
Format [[[DD:]HH:]MM:]SS
Default ---
Description Specifies the ideal job duration which will maximize the value of the WALLTIMEWEIGHT priority factor. If specified, this factor will be calculated as the distance from the ideal. Consequently, in most cases, the associated subcomponent weight should be set to a negative value.
Example
WALLTIMEWEIGHT         -2500
PRIORITYTARGETDURATION  1:00:00
PRIORITYTARGETPROCCOUNT
Format <INTEGER>{+|-|%}
Default ---
Description Specifies the ideal job requested proc count which will maximize the value of the PROCWEIGHT priority factor. If specified, this factor will be calculated as the distance from the ideal ( proc count - ideal = coefficient of PROCWEIGHT). Consequently, in most cases, the associated subcomponent weight should be set to a negative value.
Example
PROCWEIGHT               -1000
PRIORITYTARGETPROCCOUNT  64
PROCWEIGHT
Format <INTEGER>
Default 0
Description Specifies the coefficient to be multiplied by a job's requested processor count priority factor.
Example
PROCWEIGHT 2500
PROFILECOUNT
Format <INTEGER>
Default 600
Description

Specifies the number of statistical profiles to maintain.

PROFILECOUNT must be set high enough that at least one day of statistics is maintained. The statistics time window can be determined by measuring PROFILEDURATION * PROFILECOUNT. If PROFILEDURATION is one hour then PROFILECOUNT must be at least 24 so 24 hours worth of statistics are maintained. If PROFILEDURATION is 30:00 then PROFILECOUNT must be set to at least 48. If PROFILECOUNT is not high enough for at least one day of statistics, Moab adjusts it automatically.

Example
PROFILECOUNT 300
PROFILEDURATION
Format [[[DD:]HH:]MM:]SS
Default 00:30:00
Description Specifies the duration of each statistical profile. The duration cannot be more than 24 hours, and any specified duration must be a factor of 24. For example, factors of 1/4, 1/2, 1, 2, 3, 4, 6, 8, 12, and 24 are acceptable durations.
Example
PROFILEDURATION 24:00:00
PURGETIME
Format [[[DD:]HH:]MM:]SS
Default 0
Description The amount of time Moab will keep a job or node record for an object no longer reported by the resource manager. Useful when using a resource manager which 'drops' information about a node or job due to internal failures. Note: This parameter is superseded by JOBPURGETIME.
Example
PURGETIME 00:05:00

Moab will maintain a job or node record for 5 minutes after the last update regarding that object received from the resource manager.

PUSHCACHETOWEBSERVICE
Format <BOOLEAN>
Default FALSE
Description

Specifies whether or not you want to send cache objects (nodes, jobs, services, etc.) to Moab Web Services.

Example
PUSHCACHETOWEBSERVICE TRUE
QOSCFG[<QOSID>]
Format

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

General Credential Flags, PRIORITY, ENABLEPROFILING, FSTARGET, JOBPRIOACCRUALPOLICY, JOBPRIOEXCEPTIONS, MEMBERULIST, QTWEIGHT, QTTARGET, XFWEIGHT, XFTARGET, PREEMPTMINTIME, PREEMPTMAXTIME, PREEMPTQTTHRESHOLD, PREEMPTXFTHRESHOLD, PREEMPTEES, RSVQTTHRESHOLD, RSVXFTHRESHOLD, ACLBLTHRESHOLD, ACLQTTHRESHOLD, ACLXFTHRESHOLD, PLIST, QFLAGS, or a usage limit.

Default ---
Description Specifies QOS specific attributes. See the flag overview for a description of legal flag values. See the QOS Overview section for further details.
Example
QOSCFG[commercial] PRIORITY=1000 MAXJOB=4 MAXPROC=80

Moab will increase the priority of jobs using QOS commercial, and will allow up to 4 simultaneous QOS commercial jobs with up to 80 total allocated processors.

QOSDEFAULTORDER
Format Comma-delimited list of QOS names.
Default ---
Description Sets a global QOS default order for all QOS's which overrides any specific default QOS. If the order is defined as b,a,c and a user has access to c,a and submits a job without requesting a specific QOS, the job is assigned a as the default QOS.
Example
QOSDEFAULTORDER  b,a,c

If the job does not have a QOS specified, it is assigned a QOS from the QOSDEFAULTORDER list (if the user has access to one of them).

QOSISOPTIONAL
Format <BOOLEAN>
Default FALSE
Description An entity's default QOS will be the first QOS specified in the QLIST parameter. When this parameter is set to TRUE the default QOS for the associated credential (user, account, class, etc.) will not be automatically set to the first QOS specified in the QLIST.
Example
QOSISOPTIONAL TRUE
USERCFG[bob]  QLIST=high,low

Moab will set the QOSList for user bob to high and low but will not set the QDEF. Should bob decide to submit to a particular QOS he will have to do so manually.

QOSREJECTPOLICY
Format One or more of CANCEL, HOLD, IGNORE, or MAIL
Default HOLD( IGNORE for SLURM users)
Description Specifies the action to take when Moab determines that a job cannot access a requested QoS. CANCEL issues a call to the resource manager to cancel the job. HOLD places a batch hold on the job preventing the job from being further evaluated until released by an administrator. ( Note: Administrators can dynamically alter job attributes and possibly fix the job with mjobctl -m.) With IGNORE, Moab will ignore the QoS request and schedule the job using the default QoS for that job. MAIL will send email to both the admin and the user when QoS request violations are detected. Most combinations of attributes may be specified; however, if both MAIL and IGNORE are specified, Moab will not implement MAIL. Similarly, while CANCEL and HOLD are mutually exclusive, CANCEL will supersede HOLD if both are specified. (see JOBREJECTPOLICY).
Example
QOSREJECTPOLICY  MAIL,CANCEL
QOSWEIGHT
Format <INTEGER>
Default 1
Description Specifies the weight to be applied to the qos priority of each job (see Credential (CRED) Factor).
Example
QOSWEIGHT 10
QUEUETIMECAP
Format <DOUBLE>
Default 0 (NO CAP)
Description Specifies the maximum allowed absolute pre-weighted queuetime priority factor.
Example
QUEUETIMECAP    10000
QUEUETIMEWEIGHT 10

A job that has been queued for 40 minutes will have its queuetime priority factor calculated as 'Priority = QUEUETIMEWEIGHT * MIN(10000,40)'.

QUEUETIMEWEIGHT
Format <INTEGER>
Default 1
Description Specifies multiplier applied to a job's queue time (in minutes) to determine the job's queuetime priority factor.
Example
QUEUETIMEWEIGHT 20

A job that has been queued for 4:20:00 will have a queuetime priority factor of 20 * 260.

REALTIMEDBOBJECTS
Format Comma-delimited list of one or more of the following: JOB, NODE, RSV (reservation), TRIG (trigger), VC (virtual container). You can also specify ALL or NONE.
Default ALL
Description Specifies which objects Moab will store in the unixodbc database.
Example
REALTIMEDBOBJECTS JOB,RSV,TRIG

Moab stores jobs, reservations, and triggers in the uxodbc database. It will no longer record real time information about nodes and VCs.

RECORDEVENTLIST
Format One or more comma (',') or plus ('+') separated events of GEVENT, ALLSCHEDCOMMAND, AMCREATE, AMDELETE, AMEND, AMPAUSE, AMQUOTE, AMRESUME, AMSTART, AMUPDATE, JOBCANCEL, JOBCHECKPOINT, JOBEND, JOBFAILURE, JOBMIGRATE, JOBMODIFY, JOBPREEMPT, JOBREJECT, JOBRESUME, JOBSTART, JOBSUBMIT, NODEDOWN, NODEFAILURE, NODEUP, QOSVIOLATION, RMDOWN, RMPOLLEND, RMPOLLSTART, RMUP, RSVCANCEL, RSVCREATE, RSVEND, RSVMODIFY, RSVSTART, SCHEDCOMMAND, SCHEDCYCLEEND, SCHEDCYCLESTART, SCHEDPAUSE, SCHEDSTART, SCHEDSTOP, VMCREATE, VMDESTROY, VMMIGRATE, VMPOWEROFF, VMPOWERON, or ALL
Default JOBSTART, JOBCANCEL, JOBEND, JOBFAILURE, SCHEDPAUSE, SCHEDSTART, SCHEDSTOP, TRIGEND, TRIGFAILURE, TRIGSTART
Description Specifies which events should be recorded in the appropriate event file found in Moab's stats/ directory. These events are recorded for both local and remotely staged jobs. (See Event Log Overview) Note: If a plus character is included in the list, the specified events will be added to the default list; otherwise, the specified list will replace the default list.
Example
RECORDEVENTLIST JOBSTART,JOBCANCEL,JOBEND

When a local and/or remote job starts, is canceled, or ends, the respective event will be recorded.

REJECTDOSSCRIPTS
Format <BOOLEAN>
Default TRUE
Description Moab rejects DOS-formatted scripts submitted with the msub command. This is useful if you use SLURM as your resource manager, since it does not handle DOS scripts well. For REJECTDOSSCRIPTS to work correctly, FILTERCMDFILE must be FALSE. Otherwise, Moab modifies the script instead of rejecting it, leading to job errors.
Example
REJECTDOSSCRIPTS FALSE	

Moab does not reject DOS-formatted scripts submitted with msub.

REJECTINFEASIBLEJOBS
Format <BOOLEAN>
Default FALSE
Description If zero feasible nodes are found for a job among the currently available nodes on the cluster, the scheduler rejects the job. See JOBREJECTPOLICY for more information.
Example
REJECTINFEASIBLEJOBS TRUE
JOBREJECTPOLICY      MAIL,CANCEL

Any job with zero feasible nodes for execution will be rejected.

REJECTNEGPRIOJOBS
Format <BOOLEAN>
Default TRUE
Description If enabled, the scheduler will refuse to start any job with a negative priority. See Job Priority Overview and ENABLENEGJOBPRIORITY for more information.
Example
ENABLENEGJOBPRIORITY TRUE 
REJECTNEGPRIOJOBS    TRUE

Any job with a priority less than zero will be rejected.

REMAPCLASS
Format <ClassID>
Default ---
Description

Specifies which class/queue will be remapped based on the processors, nodes, and node features requested and the resource limits of each class. See Remap Class Overview for more information.

In order to use REMAPCLASS, you must specify a DEFAULTCLASS.

Example
RMCFG[internal]  DEFAULTCLASS=batch
REMAPCLASS       batch
CLASSCFG[small]  MAX.PROC=2
CLASSCFG[medium] MAX.PROC=16
CLASSCFG[large]  MAX.PROC=1024

Class batch will be remapped based on the number of processors requested.


REMAPCLASSLIST
Format Comma delimited list of class names
Default ---
Description Specifies the order in which classes will be searched when attempting to remap a class. Only classes included in the list will be searched and Moab will select the first class with matches. Note: If no REMAPCLASSLIST is specified, Moab will search all classes and will search them in the order they are discovered. See Remap Class Overview for more information.
Example
RMCFG[internal] DEFAULTCLASS=batch
REMAPCLASS      batch
REMAPCLASSLIST  short,medium,long

Class batch will be re-mapped to one of the listed classes.

REMOTEFAILTRANSIENT
Format <BOOLEAN>
Default FALSE
Description Only applicable to Moab configurations with multiple resource managers able to run jobs (such as in a grid environment). When Moab attempts to migrate a job to one of these resource managers, a remote failure may occur. For example, a destination peer in a grid that has an error accepting a job results in a remote error, and the job is rejected. REMOTEFAILTRANSIENT controls how Moab reacts to remote errors. By default, Moab considers such an error permanent and does not try to migrate the same job to that resource manager again. If REMOTEFAILTRANSIENT is set to TRUE, then Moab considers such an error as transient and will not exclude the erring resource manager in future migration attempts.
Example
REMOTEFAILTRANSIENT    TRUE
REMOVETRIGOUTPUTFILES
Format <BOOLEAN>
Default FALSE
Description When Moab launches external trigger actions, the standard output and error of those trigger actions are redirected to files located in Moab's spool directory. By default, these files are cleaned every 24 hours. (Files older than 24 hours are removed.) If, however, you wish to have Moab immediately remove the spool files after they are no longer needed, set RemoveTrigOutputFiles to TRUE.
Example
REMOVETRIGOUTPUTFILES  TRUE
RESCAP
Format <DOUBLE>
Default 0 (NO CAP)
Description Specifies the maximum allowed absolute pre-weighted job resource priority factor.
Example
RESCAP 1000

The total resource priority factor component of a job will be bound by +/- 1000

RESERVATIONDEPTH[X]
Format <INTEGER>
Default 1
Description Specifies the number of priority reservations which are allowed in the associated reservation bucket. Note: The array index, X, is the bucket label and can be any string up to 64 characters. This label should be synchronized with the RESERVATIONQOSLIST parameter. See Reservation Policies.
Example
RESERVATIONDEPTH[bigmem]   4
RESERVATIONQOSLIST[bigmem] special,fast,joshua

Jobs with QOS's of special, fast, or joshua can have a cumulative total of up to 4 priority reservations.

RESERVATIONPOLICY
Format One of the following: CURRENTHIGHEST, HIGHEST, NEVER
Default CURRENTHIGHEST
Description Specifies how Moab reservations will be handled. (See also RESERVATIONDEPTH) See Reservation Policies.
Example
RESERVATIONPOLICY          CURRENTHIGHEST
RESERVATIONDEPTH[DEFAULT]  2

Moab will maintain reservations for only the 2 currently highest priority jobs.

RESERVATIONQOSLIST[X]
Format One or more QOS values or [ALL]
Default [ALL]
Description Specifies which QOS credentials have access to the associated reservation bucket. Note: The array index, X, is the bucket label and can be any string up to 64 characters. This label should be synchronized with the RESERVATIONDEPTH parameter. See Reservation Policies.
Example
RESERVATIONDEPTH[big]   4
RESERVATIONQOSLIST[big] hi,low,med

Jobs with QOS's of hi, low, or med can have a cumulative total of up to 4 priority reservations.

RESERVATIONRETRYTIME
Format [[[DD:]HH:]MM:]SS
Default 60 seconds
Description Period of time Moab will continue to attempt to allocate resources to start a job after the time resources should be made available. This parameter takes into account resource manager node state race conditions, nodes with residual high load, network glitches, etc. (For related information, see Reservation Policies, DEFERSTARTCOUNT, DEFERTIME, NODEFAILURERESERVETIME, JOBRETRYTIME, and GUARANTEEDPREEMPTION.)
Example
RESERVATIONRETRYTIME   00:05:00

Moab will try for up to 5 minutes to maintain immediate reservations if the reservations are blocked due to node state, network, or batch system based race conditions.

RESOURCELIMITMULTIPLIER[<PARID>]
Format

<RESOURCE>:<MULTIPLIER>[,...]

Where <RESOURCE> is one of the following:

NODE, PROC, JOBPROC, MEM, JOBMEM, SWAP, DISK, or WALLTIME

Default 1.0
Description If set to less than one, then the hard limit will be the specified limit and the soft limit will be the specified limit multiplied by the multiplier. If set to a value greater than one, then the specified limit will be the soft limit and the hard limit will be the specified limit multiplied by the multiplier. See Usage-based Limits.
Example
RESOURCELIMITMULTIPLER  PROC:1.1,MEM:2.0

Sets hard limit for PROC at 1.1 times the PROC soft limit, and the hard limit of MEM to 2.0 times the MEM soft limit.

RESOURCELIMITPOLICY
Format <RESOURCE>:[<SPOLICY>,]<HPOLICY> :[<SACTION>,]<HACTION> [:[<SVIOLATIONTIME>,]<HVIOLATIONTIME>]...

Where RESOURCE is one of CPUTIME, DISK, JOBMEM, JOBPROC, MEM, MINJOBPROC, NETWORK, PROC, SWAP, or WALLTIME

where *POLICY is one of ALWAYS, EXTENDEDVIOLATION, or BLOCKEDWORKLOADONLY

and where *ACTION is one of CANCEL, CHECKPOINT, NOTIFY, REQUEUE, SIGNAL, or SUSPEND.

Default No limit enforcement.
Description Specifies how the scheduler should handle jobs which utilize more resources than they request. See Usage-based Limits.
Example
RESOURCELIMITPOLICY  MEM:ALWAYS,BLOCKEDWORKLOADONLY:REQUEUE,CANCEL

Moab will cancel all jobs which exceed their requested memory limits.

RESTARTINTERVAL
Format [[[DD:]HH:]MM:]SS
Default ---
Description Causes Moab daemon to recycle/restart when the given interval of time has transpired.
Example
RESTARTINTERVAL  20:00:00

Moab daemon will automatically restart every 20 hours.

RESOURCEQUERYDEPTH
Format <INTEGER>
Default 3
Description Maximum number of options which will be returned in response to an mshow -a resource query.
Example
RESOURCEQUERYDEPTH  1

The mshow -a command will return at most 1 valid collection of resources.

RESWEIGHT
Format <INTEGER>
Default 1
Description All resource priority components are multiplied by this value before being added to the total job priority. See Job Prioritization.
Example
RESWEIGHT  5
MEMWEIGHT  10
PROCWEIGHT 100
SWAPWEIGHT 0
RESCAP     2000

The job priority resource factor will be calculated as MIN(2000,5 * (10 * JobMemory + 100 * JobProc)).

RMCFG
Format One or more key-value pairs as described in the Resource Manager Configuration Overview
Default ---
Description Specifies the interface and policy configuration for the scheduler-resource manager interface. Described in detail in the Resource Manager Configuration Overview.
Example
RMCFG[TORQUE3] TYPE=PBS

The PBS server will be used for resource management.

RMMSGIGNORE
Format <BOOLEAN>
Default FALSE
Description

Specifies whether or not Moab should adjust node state based on generic resource manager failure messages. See Compute Node Health Check for more info.

For green or ONDEMAND computing, RMMSGIGNORE must be set to TRUE to prevent Moab from powering off a down node..

Example
RMMSGIGNORE TRUE

Moab will load and report resource manager failure messages but will not adjust node state as a result of them.

RMPOLLINTERVAL
Format [<MINPOLLTIME>,]<MAXPOLLTIME> where poll time is specified as [[[DD:]HH:]MM:]SS
Default 0,30
Description

Specifies the interval between RM polls. The poll interval will be no less than MINPOLLTIME and no more than MAXPOLLTIME. If you specify a single value, Moab interprets the value as the MAXPOLLTIME with a MINPOLLTIME of 0.

If you use TORQUE as your resource manager, prevent communication errors by giving tcp_timeout at least twice the value of the Moab RMPOLLINTERVAL.

Example
RMPOLLINTERVAL 30,45

Moab will refresh its resource manager information between a minimum of 30 seconds and a maximum of 45 seconds. Note: This parameter specifies the default global poll interval for all resource managers.

RMRETRYTIMECAP
Format [[[DD:]HH:]MM:]SS
Default 1:00:00
Description

Moab attempts to contact RMs that are in state 'corrupt' (not down). If the attempt is unsuccessful, Moab tries again later. If the second attempt is unsuccessful, Moab increases the gap (the gap grows exponentially) between communication attempts. RMRETRYTIMECAP puts a cap on the length between connection attempts.

Example
RMRETRYTIMECAP 24:00:00

Moab stops increasing the gap between connection attempts once the retry gap reaches 24 hours.

RSVLIMITPOLICY
Format HARD or SOFT
Default ---
Description Specifies what limits should be enforced when creating reservations.
Example
RSVLIMITPOLICY  HARD

Moab will limit reservation creation based on the HARD limits configured.

RSVNODEALLOCATIONPOLICY
Format One of the following:
FIRSTAVAILABLE, LASTAVAILABLE, MINRESOURCE, CPULOAD, LOCAL, CONTIGUOUS, MAXBALANCE, or PRIORITY
Default LASTAVAILABLE
Description Specifies how Moab should allocate available resources to reservations.
Example
RSVNODEALLOCATIONPOLICY MINRESOURCE

Moab will apply the node allocation policy MINRESOURCE to all reservations by default.

RSVNODEALLOCATIONPRIORITYF
Format User specified algorithm
Default ---
Description When RSVNODEALLOCATIONPOLICY is set to PRIORITY, this parameter allows you to specify your own priority algorithm. The priority functions available are the same as the node priority functions.
Example
RSVNODEALLOCATIONPOLICY PRIORITY
RSVNODEALLOCATIONPRIORITYF 'SPEED + .01 * AMEM - 10 * JOBCOUNT'
RSVPROFILE[X]
Format

One or more of the following:

Allowed:

TRIGGERACL (ACCOUNTLIST, CLASSLIST, GROUPLIST, MAXTIME, QOSLIST, USERLIST)

HostExp ( HOSTLIST)

Features (NODEFEATURES)

FLAGS

TASKCOUNT

RSVACCESSLIST

Note: Lists of more than one ACL value cannot be whitespace delimited. Such lists must be delimited with a comma, pipe, or colon.

Not allowed:

ACCESS

CHARGEACCOUNT

DAYS

DEPTH

ENDTIME

OWNER

PARTITION

PERIOD

PRIORITY

RESOURCES

STARTTIME

TPN

Default ---
Description Specifies attributes of a reservation profile using syntax similar to that for specifying a standing reservation. See Using Reservation Profiles for details.
Example
RSVPROFILE[fast] USERLIST=john,steve
RSVPROFILE[fast] QOSLIST=high,low
RSVPROFILE[fast] TRIGGER=ETYPE=start,OFFSET=5:00,ATYPE=exec,ACTION="/opt/moab/rp.pl"

Moab will create a reservation profile including trigger and ACL information.

RSVSEARCHALGO
Format LONG or WIDE
Default NONE
Description

When Moab is determining when and where a job can run, it either searches for the most resources (WIDE) or the longest range of resources (LONG). In almost all cases, searching for the longest range is ideal and returns the soonest starttime. In some rare cases, however, a particular job may need to search for the most resources. In those cases sites can configure this parameter to prevent the starvation of large jobs that fail to hold onto their reservation starttimes. See the WIDERSVSEARCHALGO job flag.

If this parameter is not set, it will be displayed in mschedctl -l as NONE but the algorithm that is used will be LONG.

Example
RSVSEARCHALGO WIDE
                                                        
SCHEDCFG
Format

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

FBSERVER, FLAGS, MAXJOBID, MAXRECORDEDCJOBID, MINJOBID, HTTPSERVERPORT, MODE, RECOVERYACTION, SERVER, or TRIGGER

Default ---
Description

Specifies scheduler policy and interface configuration.

The SERVER attribute can also be set using the environment variable $MOABSERVER. Using this variable allows you to quickly change the Moab server that client commands will connect to.

> export MOABSERVER=cluster2:12221
Example
SCHEDCFG[zylem3] SERVER=geronimo.scc.com:3422 MODE=NORMAL

Moab will execute in NORMAL mode on the host geronimo.scc.com.

SERVERHOST
Description

This parameter is deprecated. See SCHEDCFG for replacement parameter.

SERVERMODE
Description

This parameter is deprecated. See SCHEDCFG for replacement parameter.

SERVERNAME
Format <STRING>
Default <SERVERHOST>
Description Specifies the name the scheduler will use to refer to itself in communication with peer daemons. See SCHEDCFG for replacement parameter.
Example
SERVERNAME moabA
SERVERPORT
Format <INTEGER> (range: 1-64000)
Default 40559
Description Port on which moab will open its user interface socket. See SCHEDCFG for replacement parameter.
Example
SERVERPORT 30003

Moab will listen for client socket connections on port 30003.

SERVERSUBMITFILTER
Format <PATH>
Default ---
Description

Specifies the location of a global job submit filter script. When you configure a global job submit filter, Moab executes it on the head node and uses it to filter every job submission it receives. See Server-based submit filter for more information about job submit filters.

Example
SERVERSUBMITFILTER /opt/moab/scripts/globalfilter.pl

Moab uses /opt/moab/scripts/globalfilter.pl to filter every job submitted to Moab.

SERVICEWEIGHT
Format <INTEGER>
Default 1
Description Specifies the service component weight associated with the service factors. See Service (SERV) Factor for more information.
Example
SERVICEWEIGHT 2
SHOWMIGRATEDJOBSASIDLE
Format <BOOLEAN>
Default FALSE
Description By default, migrated jobs in the grid will show as blocked. This is to prevent jobs from counting against the idle policies of multiple clusters rather than just the cluster to which the job was migrated.
Example
SHOWMIGRATEDJOBSASIDLE TRUE

When set to TRUE, migrated jobs will show as idle and will count against the idle policies of the cluster showing the job as migrated.

SIMAUTOSHUTDOWN
Format <BOOLEAN>
Default TRUE
Description If TRUE, the scheduler will end simulations when the active queue and idle queue become empty.
Example
SIMAUTOSHUTDOWN TRUE

The simulation will end as soon as there are no jobs running and no idle jobs which could run.

SIMINITIALQUEUEDEPTH
Format <INTEGER>
Default 16
Description Specifies how many jobs the simulator will initially place in the idle job queue (see Simulation Overview).
Example
SCHEDCFG[sim1]         MODE=SIMULATION
SIMINITIALQUEUEDEPTH   64
SIMJOBSUBMISSIONPOLICY CONSTANTJOBDEPTH

Moab will initially place 64 idle jobs in the queue and, because of the specified queue policy, will attempt to maintain this many jobs in the idle queue throughout the duration of the simulation.

SIMJOBSUBMISSIONPOLICY
Format One of the following:
NORMAL, CONSTANTJOBDEPTH, CONSTANTPSDEPTH, or REPLAY
Default CONSTANTJOBDEPTH
Description Specifies how the simulator will submit new jobs into the idle queue. NORMAL mode causes jobs to be submitted at the time recorded in the workload trace file, CONSTANTJOBDEPTH and CONSTANTPSDEPTH attempt to maintain an idle queue of SIMINITIALQUEUEDEPTH jobs and proc-seconds respectively. REPLAY will force jobs to execute at the exactly the time specified in the simulation job trace file. This mode is most often used to generate detailed profile statistics for analysis in Moab Cluster Manager (see Simulation Overview).
Example
SIMJOBSUBMISSIONPOLICY NORMAL

Moab will submit jobs with the relative time distribution specified in the workload trace file.

SIMPURGEBLOCKEDJOBS
Format <BOOLEAN>
Default TRUE
Description Specifies whether Moab should remove jobs which can never execute (see Simulation Overview).
Example
SIMPURGEBLOCKEDJOBS  FALSE
SIMRMRANDOMDELAY
Format <INTEGER>
Default 0
Description Specifies the random delay added to the RM command base delay accumulated when making any resource manager call in simulation mode.
Example
SIMRMRANDOMDELAY  5

Moab will add a random delay of between 0 and 5 seconds to the simulated time delay of all RM calls.

SIMSTARTTIME
Format [HH[:MM[:SS]]][_MO[/DD[/YY]]]
Default ---
Description Specifies the time when the simulation starts.
Example
SIMSTARTTIME 00:00:00_01/01/00

Moab will set its clock to January 1, 2000 at 12:00:00 in the morning before starting the simulation

SIMSTOPTIME
Format [HH[:MM[:SS]]][_MO[/DD[/YY]]]
Default ---
Description Specifies the time when the simulation should pause.
Example
SIMSTOPTIME 00:00:00_01/01/04

Moab will stop scheduling when its internal simulation time reaches January 1, 2004.

SIMWORKLOADTRACEFILE
Format <STRING>
Default Traces/workload.trace
Description Specifies the file from which moab will obtain job information when running in simulation mode. Moab will attempt to locate the file relative to <MOABHOMEDIR> unless specified as an absolute path. See Simulation Overview and Workload Accounting Records.
Example
SIMWORKLOADTRACEFILE traces/jobs.2

Moab will obtain job traces when running in simulation mode from the <MOABHOMEDIR>/traces/jobs.2 file.

SPOOLDIR
Format <STRING>
Default ---
Description Specifies the directory for temporary spool files created by Moab while submitting a job to the RM.
Example
SPOOLDIR /tmp/moab/spool
SPOOLDIRKEEPTIME
Format <INTEGER> (seconds) or [[[DD:]HH:]MM:]SS
Default ---
Description Specifies the interval to delete spool files and other temporary files that have been left in the spool directory.
Example
SPOOLDIRKEEPTIME 4:00:00
SPVIOLATIONWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight to be applied to a job which violates soft usage limit policies (see Service (SERVICE) Component).
Example
SPVIOLATIONWEIGHT 5000
SRCFG[X]
Format One or more of the following <ATTR>=<VALUE> pairs
ACCESS, ACCOUNTLIST, CHARGE, CHARGEACCOUNT, CHARGEUSER, CLASSLIST, CLUSTERLIST, COMMENT, DAYS, DEPTH, DISABLE, ENDTIME, FLAGS, GROUPLIST, HOSTLIST, JOBATTRLIST, MAXTIME, NODEFEATURES, OWNER, PARTITION, PERIOD, PRIORITY, QOSLIST, REQUIREDTPN, RESOURCES, ROLLBACKOFFSET, RSVACCESSLIST, RSVGROUP, STARTTIME, TASKCOUNT, TIMELIMIT, TPN, TRIGGER, or USERLIST

Note:HOSTLIST and ACL list values must be comma delimited. For example: HOSTLIST=nodeA,nodeB

Default ---
Description Specifies attributes of a standing reservation. See Managing Reservations for details.
Example
SRCFG[fast] STARTTIME=9:00:00 ENDTIME=15:00:00
SRCFG[fast] HOSTLIST=node0[1-4]$
SRCFG[fast] QOSLIST=high,low

Moab will create a standing reservation running from 9:00 AM to 3:00 PM on nodes 1 through 4 accessible by jobs with QOS high or low.



STARTCOUNTCAP
Format <INTEGER>
Default 0
Description Specifies the max weighted value allowed from the startcount subfactor when determining a job's priority (see Priority Factors for more information).
Example
STARTCOUNTWEIGHT 5000
STARTCOUNTCAP    30000
STARTCOUNTWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight to be applied to a job's startcount when determining a job's priority (see Priority Factors for more information).
Example
STARTCOUNTWEIGHT 5000
STATDIR
Format <STRING>
Default stats
Description Specifies the directory in which Moab statistics will be maintained.
Example
STATDIR /var/adm/moab/stats
STATPROCMAX
Format <INTEGER>
Default 1
Description

Specifies the maximum number of processors requested by jobs to be displayed in matrix outputs (as displayed by the showstats -f command).

It is recommended that you not change any parameters via mschedctl -m or changeparam while Moab is running. Changing any of the parameters invalidates all past data and will start the collection over.

Example
STATPROCMAX     256
STATPROCSTEPCOUNT 4 
STATPROCSTEPSIZE  4

Each matrix output will display data in rows for jobs requesting between 4 and 256 processors.

Note

A NONE in services will still allow users to run showq and checkjob on their own jobs.


STATPROCMIN
Format <INTEGER>
Default 1
Description

Specifies the minimum number of processors requested by jobs to be displayed in matrix outputs (as displayed by the showstats -f command).

It is recommended that you not change any parameters via 'mschedctl -m' or 'changeparam' while Moab is running. Changing any of the parameters invalidates all past data and will start the collection over.

Example
STATPROCMIN       4
STATPROCSTEPCOUNT 4 
STATPROCSTEPSIZE  4

Each matrix output will display data in rows for jobs requesting between 4 and 256 processors.

Note

A NONE in services will still allow users to run showq and checkjob on their own jobs.


STATPROCSTEPCOUNT
Format <INTEGER>
Default 5
Description

Specifies the number of rows of processors requested by jobs to be displayed in matrix outputs (as displayed by the showstats -f command).

It is recommended that you not change any parameters via 'mschedctl -m' or 'changeparam' while Moab is running. Changing any of the parameters invalidates all past data and will start the collection over.

Example
STATPROCMIN       4
STATPROCSTEPCOUNT 4
STATPROCSTEPSIZE  4

Each matrix output will display data in rows for jobs requesting between 4 and 256 processors.

STATPROCSTEPSIZE
Format <INTEGER>
Default 4
Description

Specifies the processor count multiplier for rows of processors requested by jobs to be displayed in matrix outputs (as displayed by the showstats -f command).

It is recommended that you not change any parameters via 'mschedctl -m' or 'changeparam' while Moab is running. Changing any of the parameters invalidates all past data and will start the collection over.

Example
STATPROCMIN       4
STATPROCSTEPCOUNT 4
STATPROCSTEPSIZE  4

Each matrix output will display data in rows for jobs requesting between 4 and 256 processors.

STATTIMEMAX
Format [[DD:]HH:]MM:]SS
Default 00:15:00
Description

Specifies the maximum amount of time requested by jobs to be displayed in matrix outputs (as displayed by the showstats -f command).

It is recommended that you not change any parameters via 'mschedctl -m' or 'changeparam' while Moab is running. Changing any of the parameters invalidates all past data and will start the collection over.

Example
STATTIMEMAX       02:08:00
STATTIMESTEPCOUNT 4
STATTIMESTEPSIZE  4

Each matrix output will display data in columns for jobs requesting between 2 and 128 minutes.

STATTIMEMIN
Format [[DD:]HH:]MM:]SS
Default 00:15:00
Description

Specifies the minimum amount of time requested by jobs to be displayed in matrix outputs (as displayed by the showstats -f command).

It is recommended that you not change any parameters via 'mschedctl -m' or 'changeparam' while Moab is running. Changing any of the parameters invalidates all past data and will start the collection over.

Example
STATTIMEMIN       00:02:00
STATTIMESTEPCOUNT 4
STATTIMESTEPSIZE  4

Each matrix output will display data in columns for jobs requesting between 2 and 128 minutes.

STATTIMESTEPCOUNT
Format <INTEGER>
Default 6
Description

Specifies the number of columns of time requested by jobs to be displayed in matrix outputs (as displayed by the showstats -f command).

It is recommended that you not change any parameters via 'mschedctl -m' or 'changeparam' while Moab is running. Changing any of the parameters invalidates all past data and will start the collection over.

Example
STATTIMEMIN       00:02:00
STATTIMESTEPCOUNT 4
STATTIMESTEPSIZE  4

Each matrix output will display data in columns for jobs requesting between 2 and 128 minutes.

STATTIMESTEPSIZE
Format <INTEGER>
Default 4
Description

Specifies the time multiplier for columns of time requested by jobs to be displayed in matrix outputs (as displayed by the showstats -f command).

It is recommended that you not change any parameters via 'mschedctl -m' or 'changeparam' while Moab is running. Changing any of the parameters invalidates all past data and will start the collection over.

Example
STATTIMEMIN       00:02:00
STATTIMESTEPCOUNT 4
STATTIMESTEPSIZE  4

Each matrix output will display data in columns for jobs requesting between 2 and 128 minutes.

STOPITERATION
Format <INTEGER>
Default -1 (don't stop)
Description Specifies which scheduling iteration Moab will stop and wait for a command to resume scheduling.
Example
STOPITERATION 10

Moab should stop after iteration 10 of scheduling and wait for administrator commands.

STOREJOBSUBMISSION
Format <BOOLEAN>
Default ---
Description

When set to TRUE, specifies that Moab will save a job's submit arguments and script to $MOABHOMEDIR/stats/jobarchive/jobNumber.

If you use TORQUE as your resource manager, you can configure it to store completed job information, and it will store the same information returned by the qstat -f command. For more information, see Job Logging in the TORQUE documentation.

Example
STOREJOBSUBMISSION TRUE
STRICTPROTOCOLCHECK
Format <BOOLEAN>
Default FALSE
Description Specifies how Moab reacts to differences in XML protocols when communicating with other Moab peers. If set to TRUE, Moab will reject any communication that does not strictly conform to the expected protocol. If set to FALSE (the default), Moab will not reject XML that has extra or unknown attributes.
Example
STRICTPROTOCOLCHECK TRUE

Moab will reject any XML communication that does not strictly conform to the expected protocol definition.

SUBMITENVFILELOCATION
Format FILE or PIPE
Default ---
Description

If set to FILE, these behaviors are expected:

  • The environment file is owned by a user with 600 permissions.
  • Moab writes the environment variables ('\0' delimited) to a random file in Moab's spool directory.
  • Moab adds the --export-file=<path_to_file> on the sbatch command line.
  • Moab deletes the file after the job completes.

If set to PIPE, these behaviors are expected:

  • Moab creates a pipe and passes the read end of the pipe's file descriptor to sbatch.
  • Moab's parent process writes the environment ('\0' delimited) into the write end of the pipe.

Adaptive Computing recommends that you configure this parameter for a more secure environment.

Example
SUBMITENVFILELOCATION PIPE
SUBMITFILTER
Format <STRING>
Default ---
Description Specifies the directory of a given submit filter script.
Example
SUBMITFILTER /home/submitfilter/filter.pl
SUBMITHOSTS
Format space delimited list of host names
Default ---
Description If specified, SUBMITHOSTS specifies an explicit list of hosts where jobs can be submitted.
Example
SUBMITHOSTS hostA hostB
SUSPENDRESOURCES[<PARID>]
Format

<RESOURCE>[,...]

Where <RESOURCE> is one of the following:

NODE, PROC, MEM, SWAP, DISK

Default ---
Description List of resources to dedicate while a job is suspended (available in Moab version 4.5.1 and higher).
Example
SUSPENDRESOURCES[base]  MEM,SWAP,DISK

While a job is suspended in partition base, the memory, swap and disk for that job will remain dedicated to the job.

SYSCFG
Format

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

PRIORITY, FSTARGET, QLIST, QDEF, PLIST, FLAGS, or a fairness policy specification.

Default ---
Description Specifies system-wide default attributes. See the Attribute/Flag Overview for more information.
Example
SYSCFG PLIST=Partition1 QDEF=highprio

By default, all jobs will have access to partition Partition1 and will use the QOS highprio.

SWAPWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight assigned to the virtual memory request of a job.
Example
SWAPWEIGHT 10
SYSTEMMAXPROCPERJOB
Format <INTEGER>
Default -1 (NO LIMIT)
Description Specifies the maximum number of processors that can be requested by any single job.
Example
SYSTEMMAXPROCPERJOB 256

Moab will reject jobs requesting more than 256 processors.

SYSTEMMAXPROCSECONDPERJOB
Format <INTEGER>
Default -1 (NO LIMIT)
Description Specifies the maximum number of proc-seconds that can be requested by any single job.
Example
SYSTEMMAXJOBPROCSECOND 86400

Moab will reject jobs requesting more than 86400 procs seconds. i.e., 64 processors * 30 minutes will be rejected, while a 2 processor * 12 hour job will be allowed to run.

SYSTEMMAXJOBWALLTIME
Format [[[DD:]HH:]MM:]SS
Default -1 (NO LIMIT)
Description Specifies the maximum amount of wallclock time that can be requested by any single job.
Example
SYSTEMMAXJOBWALLTIME 1:00:00:00

Moab will reject jobs requesting more than 1 day of walltime.

TARGETQUEUETIMEWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight assigned to the time remaining until the queuetime is reached.
Example
TARGETQUEUETIMEWEIGHT 10
TARGETWEIGHT
Format <INTEGER>
Default 1
Description Specifies the weight to be applied to a job's queuetime and expansion factor target components (see Job Prioritization).
Example
TARGETWEIGHT 1000
TARGETXFACTORWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight assigned to the distance to the target expansion factor.
Example
TARGETXFACTORWEIGHT  10
TASKDISTRIBUTIONPOLICY
Format One of DEFAULT, PACK, RR (round-robin)
Default ---
Description Specifies how job tasks should be mapped to allocated resources. DEFAULT allows the resource manager to determine how the tasks are placed on the nodes. When PACK is used, a node is filled up with tasks before the next node is used. When RR is used, tasks are cycled through nodes, one task at a time, until there are no more tasks. See Task Distribution Overview for more information.
Example
TASKDISTRIBUTIONPOLICY DEFAULT

Moab should use standard task distribution algorithms.

THREADPOOLSIZE
Format <INTEGER>
Default 2X number of core processors (MAX: 64)
Description Governs the number of threads used when processing job scheduling. Scalability and performance may improve with multi-threading; to throttle, limit the number of threads used.
Example
THREADPOOLSIZE 10
TOOLSDIR
Format <STRING>
Default tools
Description Specifies the directory in which Moab tools will be maintained (commonly used in conjunction with Native Resource Managers, and Triggers).
Example
TOOLSDIR /var/adm/moab/tools
TRAPFUNCTION
Format <STRING>
Default ---
Description Specifies the functions to be trapped.
Example
TRAPFUNCTION UpdateNodeUtilization|GetNodeSResTime
TRAPJOB
Format <STRING>
Default ---
Description Specifies the jobs to be trapped.
Example
TRAPJOB pros23.0023.0
TRAPNODE
Format <STRING>
Default ---
Description Specifies the nodes to be trapped.
Example
TRAPNODE node001|node004|node005
TRAPRES
Format <STRING>
Default ---
Description Specifies the reservations to be trapped.
Example
TRAPRES interactive.0.1
TRIGCHECKTIME
Format <INTEGER> (milliseconds)
Default 2000
Description Each scheduling iteration, Moab will have a period of time where it handles commands and other UI requests. This time period is controlled by RMPOLLINTERVAL. During this time period, known as the UI phase, Moab will periodically evaluate triggers. Usually this only takes a fraction of a second, but if the number of triggers are large it could take up substantially more time (up to several seconds). While Moab is evaluating triggers, it doesn't respond to UI commands. This makes Moab feel sluggish and unresponsive. To remedy this, use the parameter TRIGCHECKTIME. This parameter tells Moab to only spend up to X milliseconds processing triggers during the UI phase. After X milliseconds has gone by, Moab will pause the evaluating of triggers, handle any pending UI events, and then restart the trigger evaluations where it last left off.
Example
TRIGCHECKTIME 4000
TRIGEVALLIMIT
Format <INTEGER>
Default 1
Description Each scheduling iteration, Moab will have a period of time where it handles commands and other UI requests. This time period is controlled by RMPOLLINTERVAL. During this time period, known as the UI phase, Moab will periodically evaluate triggers. The number of times Moab evaluates all triggers in the system is controlled by the TRIGEVALLIMIT parameter. By default, this is set to 1. This means that Moab will evaluate all triggers at most once during the UI phase. Moab will not leave the UI phase and start other scheduling tasks until ALL triggers are evaluated at least one time. If TrigEvalLimit is set to 5, then Moab will wait until all triggers are evaluated five times.
Example
TRIGEVALLIMIT   3
UIMANAGEMENTPOLICY
Format One of FORK or NONE
Default NONE
Description

When set with FORK, and with CLIENTUIPORT specified, Moab creates a new process to handle specific command requests in order to reduce command processing time. Currently these commands are supported:

See Appendix I: Considerations for Large Clusters for additional information on reducing command time (also known as low latency).

Example
UIMANAGEMENTPOILCY FORK
CLIENTUIPORT 42560
UJOBWEIGHT
Format <INTEGER>
Default 0
Description Weight assigned by jobs per user. -1 will reduce priority by number of active jobs owned by user.
Example
UJOBWEIGHT 10
UMASK
Format <INTEGER>
Default 0022 (octal) (produces 0644 permissions)
Description Specifies the file permission mask to use when creating new fairshare, stats, and event files. See the umask man page for more details.
Example
UMASK 0127

Create statistics and event files which are 'read-write' by owner and 'read' by group only.

UNSUPPORTEDDEPENDENCIES
Format Comma delimited string
Default ---
Description Specifies dependencies that are not supported and should not be accepted by job submissions. A maximum of 30 dependencies is supported.
Example
# moab.cfg
UNSUPPORTEDDEPENDENCIES before,beforeok,beforenotok,on
 
> msub -l depend=before:105 cmd.sh
ERROR: cannot submit job - error in extension string
UPROCWEIGHT
Format <INTEGER>
Default 0
Description Weight assigned by processors per user. -1 will reduce priority by number of active procs owned by user.
Example
UPROCWEIGHT 10
USAGECONSUMEDWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight assigned to per job processor second consumption.
Example
USAGECONSUMEDWEIGHT 10
USAGEEXECUTIONTIMEWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight assigned to the total job execution time (measured in seconds since job start). See Preemption Overview.
Example
USAGEEXECUTIONTIMEWEIGHT 10
USAGEPERCENTWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight assigned to total requested resources consumed.
Example
USAGEPERCENTWEIGHT 5
USAGEREMAININGWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight assigned to remaining usage.
Example
USAGEREMAININGWEIGHT 10
USAGEWEIGHT
Format <INTEGER>
Default 1
Description Specifies the weight assigned to the percent and total job usage subfactors.
Example
USAGEWEIGHT 100
USEANYPARTITIONPRIO
Format <BOOLEAN>
Default FALSE
Description

The FSTREE data from the first feasible FSTREE will be used when determining a job's start priority, rather than having no FSTREE data considered.

Note

Do not set USEANYPARTITIONPRIO if you use per-partition scheduling. Doing so causes to schedule jobs to the first partition listed, even if nodes from another partition will be available sooner.

Example
USEANYPARTITIONPRIO TRUE
USECPRSVNODELIST
Format <BOOLEAN>
Default TRUE
Description Specifies whether Moab should use the checkpointed reservation node list when rebuilding reservations on startup. If this is not used then Moab will use the reservation's specified host expression during rebuilding.
Example
USECPRSVNODELIST FALSE
USEDATABASE
Format INTERNAL
Default -
Description Specifies whether Moab should store profile statistics, checkpoint information, and event information in an integrated database. See Layout of Scheduler Components with Integrated Database Enabled for more information.
Example
USEDATABASE INTERNAL
USEJOBREGEX
Format BOOLEAN
Default FALSE
Description Specifies whether mjobctl supports regular expressions.
Example
USEJOBREGEX TRUE

										
[user@linux]$ mjobctl -c 8[1-3]

job '81' cancelled
job '82' cancelled
job '83' cancelled
USEMOABCTIME
Format <BOOLEAN>
Default FALSE
Description When Moab finds new jobs on the resource manager, it creates a job inside of Moab for each job in the resource manager. By default, when Moab creates a new job, it uses the time the job was submitted to the resource manager to calculate how long the job has been in the queue (Moab processing time - job creation in resource manager), which is then used in determining the job's priority.

In a system where more jobs are submitted to a resource manager than Moab can handle in one iteration, there is the possibility of jobs running out of order. For example, two jobs are both submitted at time 5. The first submitted job is processed first at time 6. So the first job's effective queue duration would be 1 (6-5). On the next iteration, the second job is processed at time 8. So the second job's effective queue duration would be 3 (8-5), indicating that it has been in the queue longer than the other job. Since the later job has a higher effective queue duration it will get a higher priority and could be scheduled to run before earlier submitted jobs.

Setting USEMOABCTIME to TRUE tells Moab to use the creation time of the job in Moab rather than the creation time in the resource manager. This corrects the possible problem of having later submitted jobs having higher priorities and starting before earlier submitted jobs.

Example
USEMOABCTIME TRUE
USEMOABJOBID
Format <BOOLEAN>
Default FALSE
Description Specifies whether to use the Moab job ID, or the resource manager's job ID.
Example
USEMOABJOBID TRUE
USERCFG[<USERID>]
Format

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

General Credential Flags, CDEF, DEFAULT.TPN, DEFAULT.WCLIMIT, EMAILADDRESS, ENABLEPROFILING, FSCAP, FSTARGET, JOBFLAGS, MAX.WCLIMIT, QLIST, QDEF, NOEMAIL, OVERRUN, PLIST, PRIORITY, or a usage limit.

Default ---
Description Specifies user specific attributes. For general user attribute information, See the Credential Overview. For a description of legal flag values, see flag overview.
Example
USERCFG[john] MAXJOB=50 QDEF=highprio
USERCFG[john] [email protected]

Up to 50 jobs submitted under the user ID john will be allowed to execute simultaneously and will be assigned the QOS highprio.

USERPRIOCAP
Format <INTEGER>
Default ---
Description Specifies the priority cap to be applied to the user specified job priority factor. Under Moab, only negative user priorities may be specified. See Credential (Service) Factor.
Example
USERPRIOWEIGHT 10
USERPRIOCAP    -10000
USERPRIOWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight to be applied to the user specified job priority. Under Moab, only negative user priorities may be specified. If this weight is set, users may reduce the priority of some of their jobs to allow other jobs to run earlier. See Credential (Service) Factor and User Selectable Prioritization.
Example
USERPRIOWEIGHT 10
USERWEIGHT
Format <INTEGER>
Default 1
Description Specifies the weight to be applied to the user priority of each job. See Credential (CRED) Factor.
Example
USERWEIGHT 10
USESYSLOG
Format <BOOLEAN>[<FACILITY>]
Default FALSE:daemon
Description Specifies whether or not the scheduler will report key events to the system syslog facility. If the <FACILITY> is specified, Moab will report events to this syslog facility. See Logging Facilities for more information.
Example
USESYSLOG TRUE:local3

Moab will report key events, commands, and failures to syslog using the local3 facility.

USESYSTEMQUEUETIME
Format <BOOLEAN>
Default FALSE
Description Specifies whether or not job prioritization should be based on the time the job has been eligible to run, i.e., idle and meets all fairness policies (TRUE) or the time the job has been idle (FALSE). See Priority Factors for more info. Note: This parameter has been superseded by the JOBPRIOACCRUALPOLICY parameter.
Example
USESYSTEMQUEUETIME FALSE

The queuetime and expansion factor components of a job's priority will be calculated based on the length of time the job has been in the idle state.

USEUSERHASH
Format <BOOLEAN>
Default FALSE
Description Enables searching of the user buffer using the user hash key instead of doing sequential searches of the user buffer.
Example
USEUSERHASH TRUE
VMCALCULATELOADBYVMSUM
Format <BOOLEAN>
Default FALSE
Description When false, vmmigrate using overcommits uses the CPU load from the node to determine if VM's need to be migrated off the hypervisor. When true, overcommit vmmigrates calculates the total node load using the total sum reported by each VM on the hypervisor.
Example
VMCALCULATELOADBYVMSUM  TRUE
VMCPURGETIME
Format [[[DD:]HH:]MM:]SS
Default 5:00
Description

When a VM completes, Moab stores it in a completed VM table for the specified amount of time. This prevents it from starting again if an RM reports it late. It also prevents a user from creating a VM with the same ID for a certain amount of time.

The VM will remain in the completed VM table for more than the specified amount of time if VMSTALETIME is greater than VMCPURGETIME. Both parameters must expire before Moab will remove the VM from the table.

Example
VMCPURGETIME 10:00

Moab holds completed VMs for 10 minutes to prevent a late RM from reporting and restarting it.

VMMIGRATETOZEROLOADNODES
Format <BOOLEAN>
Default FALSE
Description Allows VM migrations to occur to and from hypervisors that do not report a CPULoad or memory load.
Example
VMMIGRATETOZEROLOADNODES     TRUE
VMMIGRATETHROTTLE
Format <INTEGER>
Default ---
Description Sets the maximum allowable 'VM migrate' jobs at any given time.
Example
VMMIGRATETHROTTLE  20

Only 20 VM migrate jobs are allowed in the system at any given time.

VMMIGRATIONPOLICY
Format <STRING>; values include CONSOLIDATION and OVERCOMMIT
Default NONE
Description

Choose only one of these values:

  • CONSOLIDATION- If the CONSOLIDATION flag is set, Moab consolidates VMs to allow nodes to go idle. This flag also ensures that no hypervisors are overloaded.
  • OVERCOMMIT- If the OVERCOMMIT flag is set, VMs to be migrated will be selected from overloaded hypervisors to bring them below the selected thresholds. This flag must be set for the VMOCTHRESHOLD parameter to function.
Example
VMMIGRATIONPOLICY OVERCOMMIT
VMMINOPDELAY
Format [HH[:MM[:SS]
Default ---
Description The minimum time between automatic VM node operations, such as creating, modifying, and destroying VMs. May prevent thrashing.
Example
VMMINOPDELAY 30
VMOCTHRESHOLD
Format MEM:<0-1>,PROCS:<0-1>,DISK:<0-1>,SWAP:<0-1>,GMETRIC:<metric>:value
Default ---
Description Percentage threshold at which Moab begins to migrate virtual machines to other nodes. VMMIGRATIONPOLICY must be set to OVERCOMMIT for this to occur.
Example
NODECFG[DEFAULT]  VMOCTHRESHOLD=PROC:.7,MEM:.9,GMETRIC:mem_io:6000    # This is the default global policy
NODECFG[node42]   VMOCTHRESHOLD=PROC:.2,MEM:.1,GMETRIC:mem_io:12000   # This is a node-specific policy for node42

When a node surpasses .7 (70%) load of CPU or .9 (90%) of memory, Moab begins to migrate virtual machines to other nodes. When node42surpasses .2 (20%) load of CPU or .1 (10%) of memory, Moab begins to migrate virtual machines to other nodes.

VMPROVISIONSTATUSREADYVALUE
Format <INTEGER>
Default ---
Description Checks a VM for a special value or values (which Moab gets from the resource manager) and, based on the value, tells Moab that a VM was created..
Example
VMProvisionStatusReadyValue 2
VMProvisionStatusReadyValue 1-4,6,16
VMSARESTATIC
Format <BOOLEAN>
Default FALSE
Description When set to true, informs Moab that it can schedule under the assumption that no VMs will be migrated and no new VMs will be created, and disables Moab from scheduling any VM creations or migrations.
Example
VMSARESTATIC TRUE
VMSTALEACTION
Format One of the following: IGNORE, CANCELTRACKINGJOB, or DESTROY
Default IGNORE
Description

Specifies the action that is applied to a stale VM, or a VM that the resource manager has not reported to Moab recently (see VMSTALETIME).

  • IGNORE (default) specifies that Moab will take no action.
  • CANCELTRACKINGJOB specifies that Moab will remove the tracking job for stale VMs, but will not remove the actual VM (not recommended).
  • DESTROY specifies that Moab destroys stale VMs.
Note

If you specify DESTROY, you must also set the ENABLEVMDESTROY parameter to TRUE.

Example
VMSTALEACTION DESTROY                                                     
VMSTALETIME
Format [[HH:]MM:]SS
Default 10:00
Description

Specifies the amount of time a VM must be unreported by any resource manager before it is considered "stale."

To specify what happens with the VM after it has become stale, see VMSTALEACTION.

Example
VMSTALETIME 5:00                                                   

5 minutes must pass without a resource manager reporting a VM for it to be considered stale.

VMSTORAGEMOUNTDIR
Format <PATH>
Default ---
Description The specified path is used as the default location for storage mounts in all newly created VMs (created via the mvmctl command). This parameter defines the default storage mount directory if one is not specified.
Example
VMSTORAGEMOUTDIR   /var/spool

Moab uses /var/spool as a storage mount directory if a storage directory is not submitted (but additional storage is requested) at VM creation.

VMTRACKING
Format <STRING>
Default ---
Description When set to TRUE, VMTracking jobs are used to represent VMs in the queue.
Example
VMTRACKING TRUE
WALLTIMECAP
Format <DOUBLE>
Default 0 (NO CAP)
Description Specifies the maximum total pre-weighted absolute contribution to job priority which can be contributed by the walltime component. This value is specified as an absolute priority value, not as a percent.
Example
WALLTIMECAP 10000

Moab will bound a job's pre-weighted walltime priority component within the range +/- 10000.

WALLTIMEWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight to be applied to the amount of walltime requested by a job (in seconds) (see Resource (RES) Factor).
Example
RESWEIGHT      10
WALLTIMEWEIGHT 100

Increase the priority of longer duration jobs.

WCACCURACYCAP
Format <DOUBLE>
Default 0 (NO CAP)
Description Specifies the maximum total pre-weighted absolute contribution to job priority which can be contributed by the wallclock accuracy component. This value is specified as an absolute priority value, not as a percent.
Example
WCACCURACYCAP 10000

Moab will bound a job's pre-weighted wallclock accuracy priority component within the range +/- 10000.

WCACCURACYWEIGHT
Format <INTEGER>
Default 0
Description Specifies the priority weight to be applied to the job's historical user wallclock accuracy (range 0.0 to 1.0) (see Fairshare (FS) Factor).
Example
FSWEIGHT         10
WCACCURACYWEIGHT 100

Favor jobs with good wallclock accuracies by giving them a priority increase.

WCVIOLATIONACTION
Format one of CANCEL or PREEMPT
Default CANCEL
Description Specifies the action to take when a job exceeds its wallclock limit. If set to CANCEL, the job will be terminated. If set to PREEMPT, the action defined by PREEMPTPOLICY parameter will be taken. See JOBMAXOVERRUN or Usage-based limits.
Example
WCVIOLATIONACTION   PREEMPT
PREEMPTPOLICY       REQUEUE

Moab will requeue jobs which exceed their wallclock limit.

WEBSERVICESURL
Format <URL>
Default ---
Description If specified, Moab sends data to Moab Web Services (MWS) to be stored in a database. This allows Moab to spend more cycles on scheduling instead of database interaction. The sending occurs via HTTP PUT.
Example
WEBSERVICESURL http://mws-staging.ac:8080/mws/rm/moab/dump

Moab sends data that needs to be stored in a database to the specified URL.

WIKIEVENTS
Format <BOOLEAN>
Default TRUE
Description When set to true, Moab events are set to native wiki format (ATTR=VALUE pairs) to facilitate easier readability .
Example
WIKIEVENTS TRUE

Moab events will generate output in the format of the following sample:

09:26:40 1288279600:5 job 58 JOBEND 58 REQUESTEDNC=1 REQUESTEDTC=3 UNAME=wightman GNAME=wightman 
WCLIMIT=60 STATE=Completed RCLASS=[batch:1] SUBMITTIME=1288279493 RMEMCMP=>= RDISKCMP=>= 
RFEATURES=[NONE] SYSTEMQUEUETIME=1288279493 TASKS=1 FLAGS=RESTARTABLE PARTITION=pbs DPROCS=1 
ENDDATE=2140000000 TASKMAP=proxy,GLOBAL SRM=pbs EXITCODE=0   SID=2357 NODEALLOCATIONPOLICY=SHARED 
EFFECTIVEQUEUEDURATION=107
XFACTORCAP
Format <DOUBLE>
Default 0 (NO CAP)
Description Specifies the maximum total pre-weighted absolute contribution to job priority which can be contributed by the expansion factor component. This value is specified as an absolute priority value, not as a percent.
Example
XFACTORCAP 10000

Moab will bound a job's pre-weighted XFactor priority component within the range +/- 10000.

XFACTORWEIGHT
Format <INTEGER>
Default 0
Description Specifies the weight to be applied to a job's minimum expansion factor before it is added to the job's cumulative priority.
Example
XFACTORWEIGHT 1000

Moab will multiply a job's XFactor value by 1000 and then add this value to its total priority.

XFMINWCLIMIT
Format [[[DD:]HH:]MM:]SS
Default -1 (NO LIMIT)
Description Specifies the minimum job wallclock limit that will be considered in job expansion factor priority calculations.
Example
XFMINWCLIMIT 0:01:00

Jobs requesting less than 1 minute of wallclock time will be treated as if their wallclock limit was set to 1 minute when determining expansion factor for priority calculations.

© 2015 Adaptive Computing