5.688 Adding Nodes

Torque can add and remove nodes either dynamically with qmgr or by manually editing the TORQUE_HOME/server_priv/nodes file. See Initializing/Configuring Torque on the Server (pbs_server).

Be aware of the following:

Nodes cannot be added or deleted dynamically if there is a mom_hierarchy file in the server_priv directory.
When you make changes to nodes by directly editing the nodes file, you must restart pbs_server for those changes to take effect. Changes made using qmgr do not require a restart.
When you make changes to a node's ip address, you must clear the pbs_server cache. Either restart pbs_server or delete the changed node and then re-add it.
Before a newly added node is set to a free state, the cluster must be informed that the new node is valid and they can trust it for running jobs. Once this is done, the node will automatically transition to free.
Adding or changing a hostname on a node requires a pbs_server restart in order to add the new hostname as a node.

5.688.1 Run-time Node Changes

Torque can dynamically add nodes with the qmgr command. For example, the following command will add node node003:

> qmgr -c 'create node node003[,node004,node005...] [np=n,][TTL=yyyy-mm-ddThh:mm:ssZ,][acl="user==user1:user2:user3",][requestid=n]'

The optional parameters are used as follows:

np – Number of virtual processors.
TTL – (Time to Live) Specifies the time in UTC format that the node is supposed to be retired by Moab. Moab will not schedule any jobs on a node after its time to live has passed.
acl – (Access control list) Can be used to control which users have access to the node in Moab.
requestid – An ID that can be used to track the request that created the node.

You can alter the parameters of a node using a set command as follows:

qmgr -c 'set node node003 np=y'

qmgr -c 'set node node003 TTL=yyyy-mm-ddThh:mm:ssZ'

qmgr -c 'set node node003 requestid=23234'

qmgr -c 'set node node003 acl="user10,user11,user12"'

qmgr -c 'set node node003 acl+="user5,user6"'

qmgr -c 'set node node003 acl-=user1'

Torque does not use the TTL, acl, and requestid parameters. Information for those parameters are simply passed to Moab.

The above command appends the TORQUE_HOME/server_priv/nodes file with:

node003 np=3 TTL=2014-08-06T14:30:00Z acl=user1,user2,user3 requestid=3210

node004 ...

Nodes can also be removed with a similar command:

> qmgr -c 'delete node node003[,node004,node005...]'