12.5 Managing Node State
There are multiple models in which Moab can operate allowing it to either honor the node state set by an external service or locally determine and set the node state. This section covers the following:
- identifying meanings of particular node states
- specifying node states within locally developed services and resource managers
- adjusting node state within Moab based on load, policies, and events
12.5.1 Node State Definitions
12.5.2 Specifying Node States within Native Resource Managers
Native resource managers can report node state implicitly and explicitly, using NODESTATE, LOAD, NETLOAD, and other attributes. See Managing Resources Directly with the Native Interface for more information.
12.5.3 Moab Based Node State Adjustment
Node state can be adjusted based on reported processor, memory, or other load factors. It can also be adjusted based on reports of one or more resource managers in a multi-resource manager configuration. Also, both generic events and generic metrics can be used to adjust node state.
- TORQUE health scripts (allow compute nodes to detect and report site specific failures).
12.5.4 Adjusting Scheduling Behavior Based on Reported Node State
Based on reported node state, Moab can support various policies to make better use of available resources.
12.5.4.1 Down State
- JOBACTIONONNODEFAILURE parameter (cancel/requeue jobs if allocated nodes fail).
- Triggers (take specified action if failure is detected).
See Also
- Managing Resources Directly with the Native Interface
- License Management
- Adjusting Node Availability
- NODEMAXLOAD parameter