5.645 Queue Attributes

This appendix provides information on the different queue attributes.

This appendix also lists some queue resource limits. See 5.645.2 Assigning Queue Resource Limits.

For Boolean attributes, T, t, 1, Y, and y are all synonymous with "TRUE," and F, f, 0, N, and n all mean "FALSE."

5.645.1 Attributes

acl_groups

acl_group_enable

acl_group_sloppy

acl_hosts

acl_host_enable

acl_logic_or

acl_users

acl_user_enable

disallowed_types

enabled

features_required

ghost_queue

keep_completed

kill_delay

max_queuable

max_running

max_user_queuable

max_user_run

priority

queue_type

req_information_max

req_information_min

required_login_property

resources_available

resources_default

resources_max

resources_min

route_destinations

started

acl_groups
Format <GROUP>[@<HOST>][+<USER>[@<HOST>]]...
Default ---
Description

Specifies the list of groups which may submit jobs to the queue. If acl_group_enable is set to true, only users with a primary group listed in acl_groups may utilize the queue.

If the PBSACLUSEGROUPLIST variable is set in the pbs_server environment, acl_groups checks against all groups of which the job user is a member.

Example

> qmgr -c "set queue batch acl_groups=staff"

> qmgr -c "set queue batch acl_groups+=ops@h1"

> qmgr -c "set queue batch acl_groups+=staff@h1"

Used in conjunction with acl_group_enable.

acl_group_enable
Format <BOOLEAN>
Default FALSE
Description If TRUE, constrains Torque to only allow jobs submitted from groups specified by the acl_groups parameter.
Example

qmgr -c "set queue batch acl_group_enable=true"

acl_group_sloppy
Format <BOOLEAN>
Default FALSE
Description If TRUE, acl_groups will be checked against all groups of which the job users is a member.
Example ---
acl_hosts
Format <HOST>[+<HOST>]...
Default ---
Description Specifies the list of hosts that may submit jobs to the queue.
Example

qmgr -c "set queue batch acl_hosts=h1+h1+h1"

Used in conjunction with acl_host_enable.

acl_host_enable
Format <BOOLEAN>
Default FALSE
Description If TRUE, constrains Torque to only allow jobs submitted from hosts specified by the acl_hosts parameter.
Example

qmgr -c "set queue batch acl_host_enable=true"

acl_logic_or
Format <BOOLEAN>
Default FALSE
Description If TRUE, user and group acls are logically OR'd together, meaning that either acl may be met to allow access. If FALSE or unset, then both acls are AND'd, meaning that both acls must be satisfied.
Example

qmgr -c "set queue batch acl_logic_or=true"

acl_users
Format <USER>[@<HOST>][+<USER>[@<HOST>]]...
Default ---
Description Specifies the list of users who may submit jobs to the queue. If acl_user_enable is set to TRUE, only users listed in acl_users may use the queue.
Example

> qmgr -c "set queue batch acl_users=john"

> qmgr -c "set queue batch acl_users+=steve@h1"

> qmgr -c "set queue batch acl_users+=stevek@h1"

Used in conjunction with acl_user_enable.

acl_user_enable
Format <BOOLEAN>
Default FALSE
Description If TRUE, constrains Torque to only allow jobs submitted from users specified by the acl_users parameter.
Example

qmgr -c "set queue batch acl_user_enable=true"

disallowed_types
Format <type>[+<type>]...
Default ---
Description Specifies classes of jobs that are not allowed to be submitted to this queue. Valid types are interactive, batch, rerunable, nonrerunable, fault_tolerant (as of version 2.4.0 and later), fault_intolerant (as of version 2.4.0 and later), and job_array (as of version 2.4.1 and later).
Example

qmgr -c "set queue batch disallowed_types = interactive"

qmgr -c "set queue batch disallowed_types += job_array"

enabled
Format <BOOLEAN>
Default FALSE
Description Specifies whether the queue accepts new job submissions.
Example

qmgr -c "set queue batch enabled=true"

features_required
Format feature1[,feature2[,feature3...]]
Default ---
Description

Specifies that all jobs in this queue will require these features in addition to any they may have requested. A feature is a synonym for a property.

Example

qmgr -c 's q batch features_required=fast'

ghost_queue
Format <BOOLEAN>
Default FALSE
Description

Intended for automatic, internal recovery (by the server) only. If set to TRUE, the queue rejects new jobs, but permits the server to recognize the ones currently queued and/or running. Unset this attribute in order to approve a queue and restore it to normal operation. See 5.615 Automatic Queue and Job Recovery for more information regarding this process.

Example

qmgr -c "unset queue batch ghost_queue"

keep_completed
Format <INTEGER>
Default 0
Description Specifies the number of seconds jobs should be held in the Completed state after exiting. For more information, see Keeping Completed Jobs.
Example

qmgr -c "set queue batch keep_completed=120"

kill_delay
Format <INTEGER>
Default 2
Description

Specifies the number of seconds between sending a SIGTERM and a SIGKILL to a job in a specific queue that you want to cancel. It is possible that the job script, and any child processes it spawns, can receive several SIGTERM signals before the SIGKILL signal is received.

All MOMs must be configured with $exec_with_exec true in order for kill_delay to work, even when relying on default kill_delay settings.

This setting overrides the server setting. See kill_delay in 5.649 Server Parameters.

Example

qmgr -c "set queue batch kill_delay=30"

max_queuable
Format <INTEGER>
Default unlimited
Description Specifies the maximum number of jobs allowed in the queue at any given time (includes idle, running, and blocked jobs).
Example

qmgr -c "set queue batch max_queuable=20"

max_running
Format <INTEGER>
Default unlimited
Description Specifies the maximum number of jobs in the queue allowed to run at any given time.
Example

qmgr -c "set queue batch max_running=20"

max_user_queuable
Format <INTEGER>
Default unlimited
Description Specifies the maximum number of jobs, per user, allowed in the queue at any given time (includes idle, running, and blocked jobs). Version 2.1.3 and greater.
Example

qmgr -c "set queue batch max_user_queuable=20"

max_user_run
Format <INTEGER>
Default unlimited
Description This limits the maximum number of jobs a user can have running from the given queue.
Example

qmgr -c "set queue batch max_user_run=10"

priority
Format <INTEGER>
Default 0
Description Specifies the priority value associated with the queue.
Example

qmgr -c "set queue batch priority=20"

queue_type
Format One of e, execution, r, or route (see Creating a Routing Queue)
Default ---
Description

Specifies the queue type.

This value must be explicitly set for all queues.

Example

qmgr -c "set queue batch queue_type=execution"

req_information_max
Format <STRING>
Default ---
Description

Specifies the maximum resource limits allowed for jobs submitted to a queue.

These limits apply only to the qsub -L job submission option.

 

Valid values are lprocs, node, socket, numachip, core, thread, memory, swap, and disk.

If a maximum core count is specified, jobs with usecores must have lprocs<= the maximum core count; jobs without usecores are rejected.

If a maximum thread count is specified, lprocs must be <= the maximum thread count.

Example

qmgr -c "set queue batch req_information_max.lprocs=8"

req_information_min
Format <STRING>
Default ---
Description

Specifies the minimum resource limits allowed for jobs submitted to a queue.

These limits apply only to the qsub -L job submission option.

 

Valid values are lprocs, node, socket, numachip, core, thread, memory, swap, and disk.

If a minimum core count is specified, jobs with usecores must have lprocs>= the minimum core count; jobs without usecores are rejected.

If a minimum thread count is specified, lprocs must be >= the minimum thread count.

Example

qmgr -c "set queue batch req_information_min.lprocs=2"

required_login_property
Format <STRING>
Default ---
Description Adds the specified login property as a requirement for all jobs in this queue.
Example

qmgr -c 's q <queuename> required_login_property=INDUSTRIAL'

resources_available
Format <STRING>
Default ---
Description Specifies to cumulative resources available to all jobs running in the queue. See qsub will not allow the submission of jobs requesting many processors for more information.
Example

qmgr -c "set queue batch resources_available.nodect=20"

You must restart pbs_server for changes to take effect.

Also, resources_available is constrained by the smallest of queue.resources_available and server.resources_available.

resources_default
Format <STRING>
Default ---
Description Specifies default resource requirements for jobs submitted to the queue.
Example

qmgr -c "set queue batch resources_default.walltime=3600"

resources_max
Format <STRING>
Default ---
Description Specifies the maximum resource limits for jobs submitted to the queue.
Example

qmgr -c "set queue batch resources_max.nodect=16"

resources_min
Format <STRING>
Default ---
Description Specifies the minimum resource limits for jobs submitted to the queue.
Example

qmgr -c "set queue batch resources_min.nodect=2"

route_destinations
Format

<queue>[@<host>]

Default ---
Description

Specifies the potential destination queues for jobs submitted to the associated routing queue.

This attribute is only valid for routing queues (see Creating a Routing Queue).

Example

> qmgr -c "set queue route route_destinations=fast"

> qmgr -c "set queue route route_destinations+=slow"

> qmgr -c "set queue route route_destinations+=medium@hostname"

To set multiple queue specifications, use multiple commands:

> qmgr -c 's s route_destinations=batch'

> qmgr -c 's s route_destinations+=long'

> qmgr -c 's s route_destinations+=short'

started
Format <BOOLEAN>
Default FALSE
Description Specifies whether jobs in the queue are allowed to execute.
Example

qmgr -c "set queue batch started=true"

5.645.2 Assigning Queue Resource Limits

Administrators can use resources limits to help direct what kind of jobs go to different queues. There are four queue attributes where resource limits can be set: resources_available, resources_default, resources_max, and resources_min. The list of supported resources that can be limited with these attributes are arch, mem, nodect, nodes, procct, pvmem, vmem, and walltime.

Resource Format Description
arch string Specifies the administrator defined system architecture required.
mem size Amount of physical memory used by the job. (Ignored on Darwin, Digital Unix, Free BSD, HPUX 11, IRIX, NetBSD, and SunOS. Also ignored on Linux if number of nodes is not 1. Not implemented on AIX and HPUX 10.)
ncpus integer

Sets the number of processors in one task where a task cannot span nodes.

You cannot request both ncpus and nodes in the same queue.

nodect integer Sets the number of nodes available. By default, Torque will set the number of nodes available to the number of nodes listed in the TORQUE_HOME/server_priv/nodes file. nodect can be set to be greater than or less than that number. Generally, it is used to set the node count higher than the number of physical nodes in the cluster.
nodes integer Specifies the number of nodes.
procct integer

Sets limits on the total number of execution slots (procs) allocated to a job. The number of procs is calculated by summing the products of all node and ppn entries for a job.

For example qsub -l nodes=2:ppn=2+3:ppn=4 job.sh would yield a procct of 16. 2*2 (2:ppn=2) + 3*4 (3:ppn=4).

pvmem size Amount of virtual memory used by any single process in a job.
vmem size Amount of virtual memory used by all concurrent processes in the job.
walltime seconds, or [[HH:]MM:]SS Amount of real time during which a job can be in a running state.

size

The size format specifies the maximum amount in terms of bytes or words. It is expressed in the form integer[suffix]. The suffix is a multiplier defined in the following table ("b" means bytes [the default] and "w" means words). The size of a word is calculated on the execution server as its word size.

Suffix Multiplier
b w 1
kb kw 1024
mb mw 1,048,576
gb gw 1,073,741,824
tb tw 1,099,511,627,776

Related Topics 

© 2016 Adaptive Computing