Specifies the minimum job wallclock time for virtual scaling (optimistic-like backfilling.) Any job with a wallclock time less than this setting will not be virtually scaled. The value specified relates to a job's original walltime and not its virtually-scaled walltime.

Time between automatic Moab checkpoints.

If RMPOLLINTERVAL does not specify both a minimum and maximum poll time, Moab will ignore CHECKPOINTINTERVAL and checkpoint every iteration.

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

General Credential Flags, DEFAULT.ATTR, DEFAULT.DISK, DEFAULT.FEATURES, DEFAULT.GRES, DEFAULT.MEM, DEFAULT.NODE, DEFAULT.NODESET, DEFAULT.PROC, ENABLEPROFILING, EXCL.FEATURES, EXCLUDEUSERLIST, HOSTLIST, JOBEPILOG, JOBPROLOG, MAXPROCPERNODE, MAX.NODE, MAX.PROC, MAX.WCLIMIT, MIN.NODE, MIN.PROC, MIN.TPN, MIN.WCLIMIT, PARTITION, PRIORITY, PRIORITYF, QDEF, QLIST, REQ.FEATURES, REQUIREDACCOUNTLIST, REQUIREDUSERLIST, RESFAILPOLICY, SYSPRIO, WCOVERRUN, usage limit, or fairshare usage limit specification.

Disallows interactive jobs submitted with msub -I.

Note: It is possible for users to submit interactive jobs directly to a resource manager, which can bypass the DISABLEINTERACTIVEJOBS parameter. However, some resource managers (such as TORQUE) will check with Moab before allowing an interactive job.

ACCOUNTCENTRIC, HIDEBLOCKED, HIDEDRAINED, NODECENTRIC, or USEBLOCKING

Specifies flags that control how Moab client commands display varied information.

ACCOUNTCENTRIC will display account information in showq, rather than group information.

HIDEBLOCKED will prevent showq from listing information about blocked jobs which are not owned by the user if the user is not an admin.

HIDEDRAINED prevents mdiag -n from displaying nodes and mvmctl -q from displaying VMs in the DRAINED state. An override option of mdiag -n -w nodestate=drained lists only those nodes with a DRAINED state. Similarly, an override option of mvmctl -q -w state=drained lists only those VMs with a DRAINED state.

NODECENTRIC will display node allocation information instead of processor allocation information in showq.

USEBLOCKING disables threading for commands that support it; those commands include showq, mdiag -n, and mdiag -j.

Configures Moab so that it will accept msub submissions, start jobs, process triggers, etc., in a manner which minimizes their processing time. The downside is that Moab will return minimal information about these jobs at submit time.

If ENABLEHIGHTHROUGHPUT is TRUE, you must set NODEALLOCATIONPOLICY to FIRSTAVAILABLE.

Specifies the user password required for logging in to your web services. You should set this parameter in the moab-private.cfg file.

In conjunction with this parameter, you will also need to configure the following parameters to set up event logging to Moab Web Services:

• PUSHEVENTSTOWEBSERVICE
• EVENTLOGWSURL
• EVENTLOGWSUSER

Specifies your web services event log URL. For example, if you are using Moab Web Services:

In conjunction with this parameter, you will also need to configure the following parameters to set up event logging to Moab Web Services:

• PUSHEVENTSTOWEBSERVICE
• EVENTLOGWSUSER
• EVENTLOGWSPASSWORD

Specifies the username required for logging in to your web services. You should set this parameter in the moab-private.cfg file.

In conjunction with this parameter, you will also need to configure the following parameters to set up event logging to Moab Web Services:

• PUSHEVENTSTOWEBSERVICE
• EVENTLOGWSURL
• EVENTLOGWSPASSWORD

• Replace all comments with spaces (excludes Resource Manager directives)
• Strip empty lines
• Replace \r with \n
• Lock job to a PBS resource manager if $PBS is found in the script

Include the FILTERCMDFILE parameter in the moab.cfg file that resides on the clients.

FILTERCMDFILE must be FALSE for REJECTDOSSCRIPTS to work correctly.

List of zero or more space delimited <ATTR >=<VALUE> pairs where <ATTR> is one of the following:

TYPE, PRIVATE, INHERITREQFEATURES, INVERTTASKCOUNT, FEATUREGRES, or STARTDELAY

Specifies associations of generic resources into resource groups.

When PRIVATE is set to TRUE, Moab puts the requested generic resource on a separate job request.

By default a private request is a request with 1 task with X number of generic resources per task. When INVERTTASKCOUNTandPRIVATE are set to TRUE, Moab makes the generic resource's private request a request with X number of tasks with 1 generic resource per task. If INHERITREQFEATURES is also TRUE, then the private request will inherit the features of the primary request, causing Moab to place the private requests on the same nodes as the primary request.

See 12.6 Managing Consumable Generic Resources for more information.

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

General Credential Flags, PRIORITY, ENABLEPROFILING, QLIST, QDEF, PLIST, FLAGS, usage limits, or a fairshare usage limit specification.

Causes Moab to lock PREEMPTOR jobs until JOBRETRYTIME expires (essentially, waiting for the PREEMPTEE jobs to finish).

It may take some time for the PREEMPTEE jobs to clear out. During that time, the PREEMPTOR job might want to look elsewhere to run, which would be disruptive as it might preempt another set of jobs. If you wish it prevent this, it is recommended that you set GUARANTEEDPREEMPTION to TRUE.

For related information, see About preemption, Reservation Policies, DEFERSTARTCOUNT, DEFERTIME, RESERVATIONRETRYTIME, NODEFAILURERESERVETIME, and JOBRETRYTIME.

This parameter enables the identity manager interface allowing credential, policy, and usage information to be shared with an external information service.

Only one identity manager can be configured at a time.

The JOBCFG parameter supports the following attributes:

NONE, ACCOUNT, ACTION, AUTOSIZE, CLASS, CPULIMIT, DESCRIPTION, DGRES, FAILUREPOLICY, GROUP, IFLAGS, JOBSCRIPT MEM (for MEM=<value>), MEMORY (for MEMORY=$LEARN), NODEACCESSPOLICY, NODEMOD, PARTITION, PREF, QOS, RESTARTABLE, RM, RMSERVICEJOB, SELECT, STAGEIN, SOFTWARE, SRM, TEMPLIMIT, TFLAGS, USER, VMUSAGE, WALLTIME, WORK

It also supports the following Wiki attributes:

ARGS, DMEM, DDISK, DWAP, ERROR, EXEC, EXITCODE, GATTR, GEVENT, IWD, JNAME, NAME, PARTITIONMASK, PRIORITYF, RDISK, RSWAP, RAGRES, RCGRES, TASKPERNODE, TRIGGER, VARIABLE, NULL

Note: The index to the JOBCFG parameter can either be an admin-chosen job template name or the exact name of job reported by one or more workload queries. See Wiki Attributes and Job Template Extensions.

Extends the job walltime when Moab starts the job up to the lesser of the maximum or the next reservation (rounded down to the nearest minute).

JOBEXTENDSTARTWALLTIME TRUE and JOBEXTENDDURATION cannot be configured together. If they are in the same moab.cfg or are both active, then the JOBEXTENDDURATION will not be honored.

JOBEXTENDSTARTWALLTIME TRUE

Submit job with a minimum wallclock limit and a walltime; for example:

echo sleep 500 | msub -A ee -l
nodes=5,minwclimit=5:00,walltime=30:00,partition=g02

Specifies the number of times a job is requeued and restarted by Moab if the job fails (if the job itself returns a non-zero exit code). Some types of jobs may succeed if automatically retried several times in short succession. This parameter was created with these types of jobs in mind. Note that the job in question must also be restartable (the job needs to have the "RESTARTABLE" flag set on it) and the RM managing the job must support requeuing and starting completed jobs.

If a job fails too many times, and reaches the number of retries given by JobFailRetryCount, then a UserHold is placed on the job and a message is attached to it signifying that the job has a "restart count violation."

Soft and hard limit of the amount of time Moab will allow a job to exceed its wallclock limit before it first sends a mail to the primary admin (soft limit) and then terminates the job (hard limit). See WCVIOLATIONACTION or Usage-based Limits.

If you run Moab with the TORQUE resource manager, you must set the $ignwalltime parameter to true in the /var/spool/torque/mom_priv/config file, otherwise the pbs_mom will kill any job that exceeds its walltime. See the TORQUE documentation for more information.

Specifies how Moab should track the dynamic aspects of a job's priority. ACCRUE indicates that the job will accrue queuetime based priority from the time it is submitted unless it violates any of the policies not specified in JOBPRIOEXCEPTIONS. RESET indicates that it will accrue priority from the time it is submitted unless it violates any of the JOBPRIOEXCEPTIONS. However, with RESET, if the job does violate JOBPRIOEXCEPTIONS then its queuetime based priority will be reset to 0.

JOBPRIOACCRUALPOLICY is a global parameter, but can be configured to work only in QOSCFG:

QOSCFG[arrays]  JOBPRIOACCRUALPOLICY=ACCRUE

The following old JOBPRIOACCRUALPOLICY values have been deprecated and should be adjusted to the following values:

• QUEUEPOLICY= ACCRUE and JOBPRIOEXCEPTIONSSOFTPOLICY,HARDPOLICY
• QUEUEPOLICYRESET= RESET and JOBPRIOEXCEPTIONSSOFTPOLICY,HARDPOLICY
• ALWAYS= ACCRUE and JOBPRIOEXCEPTIONSALL
• FULLPOLICY= ACCRUE and JOBPRIOEXCEPTIONSNONE
• FULLPOLICYRESET= RESET and JOBPRIOEXCEPTIONSNONE

Specifies exceptions for calculating a job's dynamic priority (QUEUETIME, XFACTOR, TARGETQUEUETIME). Normally, when a job violates a policy, is placed on hold, or has an unsatisfied dependency, it will not accrue priority. Exceptions can be configured to allow a job to accrue priority in spite of any of these violations. With DEPENDS a job will increase in priority even if there exists an unsatisfied dependency. With SOFTPOLICY, HARDPOLICY, or IDLEPOLICY a job can accrue priority despite violating a specific limit. With DEFER, USERHOLD, BATCHHOLD, or SYSTEMHOLD a job can accrue priority despite being on hold.

JOBPRIOEXCEPTIONS is a global parameter, but can be configured to work only in QOSCFG:

QOSCFG[arrays]  JOBPRIOEXCEPTIONS=IDLEPOLICY

Moab will remove the specified environment variables from the job's environment before migrating the job to its destination resource manager. This is useful when jobs submit themselves from one cluster to another with the full environment.

This parameter is currently only supported with TORQUE resource managers.

If set to NONE, no mail is sent. If set to DEFAULT, Moab sends mail via the system's default mail program (usually /usr/bin/sendmail). If set to the local path of a mail program, Moab uses the specified mail program to send mail.

By default, Moab mail notification is disabled. To enable, you must set MAILPROGRAM to DEFAULT or specify some other locally available mail program. If the default mail domain is set, emails will be routed to this domain unless a per-user domain is specified using the EMAILADDRESS attribute of the USERCFG parameter. If neither of these values is set, Moab uses "@localhost" as the mail domain. See Notify Admins.

For jobs, the email address used on the msub -M option overrides all other user email addresses. Additionally, administrators are notified in the case of job violations.

Specifies the maximum quantity of jobs for which Moab should allocate memory used for tracking jobs. If Moab is tracking the maximum quantity of jobs specified by this parameter, it rejects subsequent jobs submitted by any user since it has no memory left with which to track newly submitted jobs.

If a user submitted a job with the msub command, this rejection behavior requires the user to resubmit the job at a later time after other jobs have completed, which frees memory in which Moab can place later-submitted jobs. If a user submitted a job with the TORQUE qsub command, TORQUE will automatically resubmit the job to Moab until Moab accepts it.

The mdiag -S command indicates if any job overflows have occurred.

If this parameter's value is changed, it does not go into effect until Moab restarts. Moab reads the parameter only on initial startup and uses its value to allocate the memory it uses to track jobs.

Specifies the maximum number of compute nodes supported.

Specifies the maximum number of reservations on a node.

For large SMP systems (>512 processors/node), Adaptive Computing advises adjusting the value to approximately twice the average sum of admin, standing, and job reservations present.

A second number, led by a comma, can also be specified to set a maximum number of reservations for nodes that are part of the SHARED partition.

The maximum possible value of MAXRSVPERNODE is 8192 for a global node and 4096 for any other node.

Moab must be restarted for any changes to this parameter to take effect. The command mdiag -S indicates whether any node reservation overflows have occurred. See Considerations for Large Clusters.

Do not lower the MAXRSVPERNODE value while there are active jobs in the queue. This can lead to queue instability and certain jobs could become stuck or disconnected from the system.

Specifies the length of the interval (in seconds) between job queries when using msub -K. Jobs submitted with the -K option query the scheduler every MSUBQUERYINTERVAL seconds until the job is completed.

MSUBQUERYINTERVAL can exist as an environment variable. Any value in moab.cfg overrides the environment variable.

Note: If MSUBQUERYINTERVAL is set to 0, the -K option will be disabled. Jobs will still submit correctly, but the client will not continue to check on the job.

One of the following:

SHARED, SHAREDONLY, SINGLEJOB, SINGLETASK, SINGLEUSER, or UNIQUEUSER

NODEACCESSPOLICY SINGLEUSER

Moab will allow resources on a node to be used by more than one job provided that the jobs are all owned by the same user.

One of the following:

FIRSTAVAILABLE, LASTAVAILABLE, MINRESOURCE, CPULOAD, LOCAL, CONTIGUOUS, MAXBALANCE, PRIORITY, or PLUGIN.

Specifies how Moab should allocate available resources to jobs. See Node Allocation Overview for more information.

If ENABLEHIGHTHROUGHPUT is TRUE, you must set NODEALLOCATIONPOLICY to FIRSTAVAILABLE.

One of the following:

CANCEL, HOLD, IGNORE, MIGRATE, NOTIFY, or REQUEUE

<POLICY>[:<RESOURCETYPE>] ...

where <POLICY> is one of COMBINED, DEDICATED, or UTILIZED

and <RESOURCETYPE> is one of PROC, MEM, SWAP, or DISK

where <LABEL> is any string and <NODECAT> is one of the defined node categories.

Causes Moab to attempt to span a job evenly across node sets unless doing so delays the job beyond the requested NODESETDELAY.

Must use with NODESETPLUS set to SPANEVENLY; if you do not want to use SPANEVENLY, use NODESETISOPTIONAL instead of NODESETDELAY.

Specifies how Moab distributes jobs among node sets. See Node Set Overview.

This parameter will not work with multi-req jobs or preemption.

Length of time Moab will assume untracked generic resources will remain unavailable for scheduling if a system reservation is not explicitly created for the node.

If NODEUNTRACKEDRESDELAYTIME is enabled and there is an untracked resource preventing a job from running, then the job remains in the idle queue instead of being deferred.

Disables all Moab OS credential lookups, including UID, GID, user to group mappings, and any other OS specific information.

Setting OSCREDLOOKUP by itself does not allow job submission; additional configuration is required. When submitting jobs from user accounts that do not exist on the head node (where Moab Workload Manager and TORQUE are running), you must also set the PROXYJOBSUBMISSION flag in addition to specifying configuration settings in the resource manager configuration file. See the example that follows for information on required resource manager settings.

By default Moab's scheduling routine schedules each job on each partition using the following algorithm:

prioritize

foreach (job)

     find the partition on which that job should run

     schedule job

In this model, a job's priority is the same on each partition as it uses a single global priority. Because a job's priority is the same on every partition, Moab prioritizes the queue once and then schedules the prioritized queue across all partitions.

When PERPARTITIONSCHEDULING TRUE is set, the following algorithm is used:

foreach (partition)

     prioritize

     foreach (job)

          schedule job

In this case, each partition may have a unique priority configuration and Moab will re-prioritize the jobs for each partition on the system. Each job is prioritized and scheduled on each partition. See PARCFG for more information. Also, note that Moab will order the partitions as they are discovered in the moab.cfg file. Partitions should be explicitly ordered via PARCFG in the moab.cfg file.

Note: If this policy is set to REQUEUE, preemptible jobs should be marked as RESTARTABLE. If this policy is set to SUSPEND, preemptible jobs should be marked as SUSPENDABLE. Note: Moab uses preemption escalation to preempt resources if the specified preemption facility is not applicable. This means if the policy is set to SUSPEND and the job is not SUSPENDABLE, Moab may attempt to requeue or even cancel the job.

Determines which jobs to preempt based on size or priority. The higher the value, the more emphasis is placed on the priority of the job, causing the lower priority jobs to be preempted first. The lower the value, the more emphasis is placed on the size of the job, causing the smaller jobs to be preempted first. If set to 0, job priority will be ignored, job size will take precedence and the smallest jobs will be preempted.

The special setting of -1 places the emphasis solely on resource utilization. This means that jobs will be preempted in a manner that keeps the resource utilization at the highest level, regardless of job priority or size.

Specifies the number of statistical profiles to maintain.

PROFILECOUNT must be set high enough that at least one day of statistics is maintained. The statistics time window can be determined by measuring PROFILEDURATION * PROFILECOUNT. If PROFILEDURATION is one hour then PROFILECOUNT must be at least 24 so 24 hours worth of statistics are maintained. If PROFILEDURATION is 30:00 then PROFILECOUNT must be set to at least 48. If PROFILECOUNT is not high enough for at least one day of statistics, Moab adjusts it automatically.

Specifies whether or not you want to send cache objects (nodes, jobs, services, etc.) to Moab Web Services.

Specifies whether or not you want to send event logs to web services for storage. For more information, see Event logging with web services.

In conjunction with this parameter, you will also need to configure the following parameters to set up event logging to Moab Web Services:

• EVENTLOGWSURL
• EVENTLOGWSUSER
• EVENTLOGWSPASSWORD

List of zero or more space delimited <ATTR>=<VALUE> pairs where <ATTR> is one of the following:

General Credential Flags, PRIORITY, ENABLEPROFILING, FSTARGET, JOBPRIOACCRUALPOLICY, JOBPRIOEXCEPTIONS, MEMBERULIST, QTWEIGHT, QTTARGET, XFWEIGHT, XFTARGET, PREEMPTMINTIME, PREEMPTMAXTIME, PREEMPTQTTHRESHOLD, PREEMPTXFTHRESHOLD, PREEMPTEES, RSVQTTHRESHOLD, RSVXFTHRESHOLD, ACLBLTHRESHOLD, ACLQTTHRESHOLD, ACLXFTHRESHOLD, PLIST, QFLAGS, or a usage limit.