Appendix B: Server Parameters
TORQUE server parameters are specified using the qmgr command. The set subcommand is used to modify the server object. For example:
> qmgr -c 'set server default_queue=batch'
Parameters
| acl_hosts | |||
| Format: | <HOST>[,<HOST>]... or <HOST>[range] or <HOST*> where the asterisk (*) can appear anywhere in the host name | ||
| Default: | (only the host running pbs_server may submit jobs) | ||
| Description: |
Specifies a list of hosts from which jobs may be submitted. Hosts in the server nodes file located at $TORQUE/server_priv/nodes cannot be added to the list using the acl_hosts parameter. To submit batch or interactive jobs (See Configuring Job Submit Hosts) through hosts that are specified in the server nodes file, use the submit_hosts parameter.Qmgr: set queue batch acl_hosts = "hostA,hostB" Qmgr: set queue batch acl_hosts += "hostE,hostF,hostG" In version 2.5 and later, the wildcard (*) character can appear anywhere in the host name, and ranges are supported; these specifications also work for managers and operators. Qmgr: set server acl_hosts = "galaxy*.tom.org" Qmgr: set server acl_hosts += "galaxy[0-50].tom.org" Qmgr: set server managers+=tom@galaxy[0-50].tom.org |
||
| acl_host_enable | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | Specifies if the acl_hosts value is enabled | ||
| acl_logic_or | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | Specifies if user and group queue ACL's should be logically AND'd or logically OR'd | ||
| acl_roots | |||
| Format: | <username>@<domain> | ||
| Default: | --- | ||
| Description: | Specifies which root users are allowed to submit and run jobs. | ||
| allow_node_submit | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | Specifies if users can submit jobs directly from any trusted compute host directly or from within batch jobs (See Configuring Job Submit Hosts.) | ||
| allow_proxy_user | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | Specifies users can proxy from one user to another. Proxy requests will be either validated by ruserok() or by the scheduler. (See Job Submission Configuration.) | ||
| auto_node_np | |||
| Format: | <BOOLEAN> | ||
| Default: | DISABLED | ||
| Description: | Automatically configures a node's np (number of processors) value based on the ncpus value from the status update. Requires full manager privilege to set or alter. | ||
| checkpoint_defaults | |||
| Format: | <STRING> | ||
| Default: | --- | ||
| Description: | Specifies for a queue the default checkpoint values for a job that does not have checkpointing specified. The checkpoint_defaults parameter only takes effect on execution queues.
set queue batch checkpoint_defaults="enabled, periodic, interval=5" |
||
| clone_batch_delay | |||
| Format: | <INTEGER> | ||
| Default: | 1 | ||
| Description: | Specifies the delay (in seconds) between clone batches. | ||
| clone_batch_size | |||
| Format: | <INTEGER> | ||
| Default: | 256 | ||
| Description: | Job arrays are created in batches of size X. X jobs are created, and after the clone_batch_delay, X more are created. This repeats until all are created. | ||
| default_queue | |||
| Format: | <STRING> | ||
| Default: | --- | ||
| Description: | Indicates the queue to assign to a job if no queue is explicitly specified by the submitter | ||
| disable_server_id_check | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | Makes it so the user for the job doesn't have to exist on the server. The user must still exist on all the compute nodes or the job will fail when it tries to execute.
|
||
| disable_job_server_suffix | |||
| Format: | <BOOLEAN> | ||
| Default: | TRUE | ||
| Description: | If this parameter is set to TRUE, Torque will display both the job ID and the host name. If it is set to FALSE, only the job ID will be displayed.
|
||
| job_force_cancel_time | |||
| Format: | <INTEGER> | ||
| Default: | Disabled | ||
| Description: | If a job has been deleted and is still in the system after x seconds, the job will be purged from the system. This is mostly useful when a job is running on a large number of nodes and one node goes down. The job cannot be deleted because the MOM cannot be contacted. The qdel fails and none of the other nodes can be reused. This parameter can used to remedy such situations. | ||
| job_log_file_max_size | |||
| Format: | <INTEGER> | ||
| Default: | --- | ||
| Description: | This specifies a soft limit (in kilobytes) for the job log's maximum size. The file size is checked every five minutes and if the current day file size is greater than or equal to this value, it is rolled from <filename> to <filename.1> and a new empty log is opened. If the current day file size exceeds the maximum size a second time, the <filename.1> log file is rolled to <filename.2>, the current log is rolled to <filename.1>, and a new empty log is opened. Each new log causes all other logs to roll to an extension that is one greater than its current number. Any value less than 0 is ignored by pbs_server (meaning the log will not be rolled). | ||
| job_log_file_roll_depth | |||
| Format: | <INTEGER> | ||
| Default: | --- | ||
| Description: | This sets the maximum number of new log files that are kept in a day if the job_log_file_max_size parameter is set. For example, if the roll depth is set to 3, no file can roll higher than <filename.3>. If a file is already at the specified depth, such as <filename.3>, the file is deleted so it can be replaced by the incoming file roll, <filename.2>. | ||
| job_log_keep_days | |||
| Format: | <INTEGER> | ||
| Default: | --- | ||
| Description: | This maintains logs for the number of days designated. If set to 4, any log file older than 4 days old is deleted. | ||
| job_nanny | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | Enables the experimental "job deletion nanny" feature. All job cancels will create a repeating task that will resend KILL signals if the initial job cancel failed. Further job cancels will be rejected with the message "job cancel in progress." This is useful for temporary failures with a job's execution node during a job delete request. | ||
| job_stat_rate | |||
| Format: | <INTEGER> | ||
| Default: | 45 (set to 30 in TORQUE 1.2.0p5 and earlier) | ||
| Description: | Specifies the maximum age of MOM level job data which is allowed when servicing a qstat request. If data is older than this value, the pbs_server daemon will contact the MOMs with stale data to request an update.For large systems, this value should be increased to 5 minutes or higher. | ||
| job_start_timeout | |||
| Format: | <INTEGER> | ||
| Default: | --- | ||
| Description: | Specifies the pbs_server to pbs_mom TCP socket timeout in seconds that is used when the pbs_server sends a job start to the pbs_mom. It is useful when the MOM has extra overhead involved in starting jobs. If not specified, then the tcp_timeout parameter is used. | ||
| lock_file | |||
| Format: | <STRING> | ||
| Default: | torque/server_priv/server.lock | ||
| Description: | Specifies the name and location of the lock file used to determine which high availability server should be active.
If a full path is specified, it is used verbatim by TORQUE. If a relative path is specified, TORQUE will prefix it with torque/server_priv. |
||
| lock_file_update_time | |||
| Format: | <INTEGER> | ||
| Default: | 3 | ||
| Description: | Specifies how often (in seconds) the thread will update the lockfile. | ||
| lock_file_check_time | |||
| Format: | <INTEGER> | ||
| Default: | 9 | ||
| Description: | Specifies how often (in seconds) a high availability server will check to see if it should become active. |
||
| log_events | |||
| Format: | Bitmap | ||
| Default: | --- | ||
| Description: | By default, all events are logged. However, you can customize things so that only
certain events show up in the log file. These are the bitmaps for the
different kinds of logs: If you want to log only error, system, and job information, use qmgr to set log_events to 11: set server log_events = 11 |
||
| log_file_max_size | |||
| Format: | <INTEGER> | ||
| Default: | 0 | ||
| Description: | Specifies a soft limit, in kilobytes, for the server's log file. The filesize is checked every 5 minutes, and if the current day filesize is greater than or equal to this value then it will be rolled from X to X.1 and a new empty log will be opened. Any value ⇐ 0 will be ignored by pbs_server (the log will not be rolled). | ||
| log_file_roll_depth | |||
| Format: | <INTEGER> | ||
| Default: | 1 | ||
| Description: | This parameter controlls how deep the current day log files will be rolled, if log_file_max_size is set, before they are deleted. | ||
| log_keep_days | |||
| Format: | <INTEGER> | ||
| Default: | 0 | ||
| Description: | This specifies how long (in days) a server or MOM log should be kept. | ||
| log_level | |||
| Format: | <INTEGER> | ||
| Default: | 0 | ||
| Description: | Specifies the pbs_server logging verbosity. Maximum value is 7. | ||
| mail_body_fmt | |||
| Format: | A printf-like format string | ||
| Default: | PBS Job Id: %i Job Name: %j Exec host: %h %m %d | ||
| Description: | Override the default format for the body of outgoing mail messages. A number of printf-like format specifiers and escape sequences can be used: \n new line \t tab \\ backslash \' single quote \" double quote %d details concerning the message %h PBS host name %i PBS job identifier %j PBS job name %m long reason for message %r short reason for message %% a single % |
||
| mail_domain | |||
| Format: | <STRING> | ||
| Default: | --- | ||
| Description: | Override the default domain for outgoing mail messages. If set, emails will be addressed to <user>@<hostdomain>. If unset, the job's Job_Owner attribute will be used. If set to never, TORQUE will never send emails. | ||
| mail_subject_fmt | |||
| Format: | A printf-like format string | ||
| Default: | PBS JOB %i | ||
| Description: | Override the default format for the subject of outgoing mail messages. A number of printf-like format specifiers and escape sequences can be used: \n new line \t tab \\ backslash \' single quote \" double quote %d details concerning the message %h PBS host name %i PBS job identifier %j PBS job name %m long reason for message %r short reason for message %% a single % |
||
| managers | |||
| Format: | <user>@<host.sub.domain>[,<user>@<host.sub.domain>...] | ||
| Default: | root@localhost | ||
| Description: | List of users granted batch administrator privileges. The host, sub-domain, or domain name may be wildcarded by the use of an asterisk character (*). Requires full manager privilege to set or alter. | ||
| max_job_array_size | |||
| Format: | <INTEGER> | ||
| Default: | Unlimited | ||
| Description: | Sets the maximum number of jobs that can be in a single job array. | ||
| max_slot_limit | |||
| Format: | <INTEGER> | ||
| Default: | Unlimited | ||
| Description: | This is the maximum number of jobs that can run concurrently in any job array. Slot limits can be applied at submission time with qsub, or it can be modifed with qalter.
qmgr -c 'set server max_slot_limit=10' Requested slot limit is too large, limit is 10. |
||
| max_threads | |||
| Format: | <INTEGER> | ||
| Default: | 5 | ||
| Description: | This is the maximum number of threads that should exist in the threadpool at any time. | ||
| min_threads | |||
| Format: | <INTEGER> | ||
| Default: | 5 | ||
| Description: | This is the minimum number of threads that should exist in the threadpool at any time. | ||
| moab_array_compatible | |||
| Format: | <BOOLEAN> | ||
| Default: | TRUE | ||
| Description: | This parameter places a hold on jobs that exceed the slot limit in a job array. When one of the active jobs is completed or deleted, one of the held jobs goes to a queued state. |
||
| mom_job_sync | |||
| Format: | <BOOLEAN> | ||
| Default: | TRUE | ||
| Description: | Specifies that the pbs_server will synchronize its view of the job queue and resource allocation with compute nodes as they come online. If a job exists on a compute node in a pre-execution or corrupt state, it will be automatically cleaned up and purged. (enabled by default in TORQUE 2.2.0 and higher) | ||
| np_default | |||
| Format: | <INTEGER> | ||
| Default: | --- | ||
| Description: | Allows the administrator to unify the number of processors (np) on all nodes. The value can be dynamically changed. A value of 0 tells pbs_server to use the value of np found in the nodes file. The maximum value is 32767. | ||
| operators | |||
| Format: | <user>@<host.sub.domain>[,<user>@<host.sub.domain>...] | ||
| Default: | root@localhost | ||
| Description: | List of users granted batch operator privileges. Requires full manager privilege to set or alter. | ||
| node_check_rate | |||
| Format: | <INTEGER> | ||
| Default: | 600 | ||
| Description: | Specifies the minimum duration (in seconds) that a node can be unresponsive to server queries before being marked down by the pbs_server daemon | ||
| node_pack | |||
| Format: | <BOOLEAN> | ||
| Default: | --- | ||
| Description: | Controls how multiple processor nodes are allocated to jobs. If this attribute is set to true, jobs will be assigned to the multiple processor nodes with the fewest free processors. This packs jobs into the fewest possible nodes leaving multiple processor nodes free for jobs which need many processors on a node. If set to false, jobs will be scattered across nodes reducing conflicts over memory between jobs. If unset, the jobs are packed on nodes in the order that the nodes are declared to the server (in the nodes file). Default value: unset - assigned to nodes as nodes in order that were declared. | ||
| node_ping_rate | |||
| Format: | <INTEGER> | ||
| Default: | 300 | ||
| Description: | Specifies the maximum interval (in seconds) between successive pings sent from the pbs_server daemon to the pbs_mom daemon to determine node/daemon health. | ||
| poll_jobs | |||
| Format: | <BOOLEAN> | ||
| Default: | TRUE (FALSE in TORQUE 1.2.0p5 and earlier) | ||
| Description: | If set to TRUE, pbs_server will poll job info from MOMs over time and will not block on handling requests which require this job information. If set to FALSE, no polling will occur and if requested job information is stale, pbs_server may block while it attempts to update this information. For large systems, this value should be set to TRUE. | ||
| query_other_jobs | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | Specifies whether or not non-admin users may view jobs they do not own | ||
| record_job_info | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | This must be set to true in order for job logging to be enabled. | ||
| record_job_script | |||
| Format: | <BOOLEAN> | ||
| Default: | FALSE | ||
| Description: | If set to TRUE, this adds the contents of the script executed by a job to the log. | ||
| resources_available | |||
| Format: | <STRING> | ||
| Default: | --- | ||
| Description: | Allows overriding of detected resource quantity limits (see queue resources_available). pbs_server must be restarted for changes to take effect. Also, resources_available is constrained by the smallest of queue.resources_available and the server.resources_available. | ||
| submit_hosts | |||
| Format: | "<HOSTNAME>[,<HOSTNAME>]..." | ||
| Default: | --- | ||
| Description: | Indicates which hosts included in the server nodes file located at $TORQUE/server_priv/nodes can submit batch or interactive jobs (See Configuring Job Submit Hosts). For more information on adding hosts that are not included in the first nodes file, see the acl_hosts parameter. | ||
| tcp_timeout | |||
| Format: | <INTEGER> | ||
| Default: | 8 | ||
| Description: | Specifies the pbs_server to pbs_mom TCP socket timeout in seconds. (See Considerations for Large Clusters).
|
||
| thread_idle_seconds | |||
| Format: | <INTEGER> | ||
| Default: | -1 | ||
| Description: | This is the number of seconds a thread can be idle in the threadpool before it is deleted. If threads should not be deleted, set to -1 which is the default. TORQUE will always maintain at least min_threads number of threads, even if all are idle. | ||
