(Click to open topic with navigation)

4.0 Scheduler Commands > Commands > mnodectl

Conventions

mnodectl

Synopsis

mnodectl -m attr{=|-=}val nodeexp
mnodectl -q [cat|diag|profile|wiki] nodeexp

Overview

Change specified attributes for a given node expression.

Access

By default, this command can be run by any Moab Administrator.

Format

-m - Modify
Format

<ATTR>{=|-=}<VAL>

Where <ATTR> is one of the following:
FEATURES
GEVENT,
GMETRIC,
MESSAGE,
OS,
POWER,
STATE,
VARIABLE

and -=, in this case, clears the attribute; it does not decrement the attribute's value.

Changing OS and POWER require a Moab Adaptive Computing Suite license and a provisioning resource manager.
Description Modify the state or attribute of specified node(s)
Example 
> mnodectl -m features=fastio,highmem node1
> mnodectl -m gevent=cpufail:'cpu02 has failed w/ec:0317' node1 
> mnodectl -m gmetric=temp:131.2 node1 
> mnodectl -m message='cpufailure:cpu02 has failed w/ec:0317' node1 
> mnodectl -m OS=RHAS30 node1 
> mnodectl -m power=off node1
> mnodectl -m state=idle node1
> mnodectl -m variable=IP=10.10.10.100,Location=R1S2 node1
-q - Query
Format {cat | diag | profile | wiki}
Description

Query node categories or node profile information (see ENABLEPROFILING for nodes).

The diag and profile options must use --xml.
Example 
> mnodectl -q cat ALL
node categorization stats from Mon Jul 10 00:00:00 to Mon Jul 10 15:30:00
Node: moab
 Categories:
                     busy: 96.88%
                     idle: 3.12%
Node: maka
 Categories:
                     busy: 96.88%
                     idle: 3.12%
Node: pau
 Categories:
                     busy: 96.88%
                     idle: 3.12%
Node: maowu
 Categories:
                     busy: 96.88%
                  down-hw: 3.12%
Cluster Summary:
                     busy: 96.88%
                  down-hw: 0.78%
                     idle: 2.34%
> mnodectl -v -q profile
...
> mnodectl -q wiki <ALL>
GLOBAL STATE=Idle PARTITION=SHARED
n0 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n1 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n2 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n3 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n4 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n5 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n6 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n7 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n8 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED
n9 STATE=Idle PARTITION=base APROC=4 CPROC=4 RM=base NODEACCESSPOLICY=SHARED

Query a node with the output displayed in a WIKI string.

Parameters

FEATURES
Format <STRING>

One of the following:

  • a comma-delimited list of features
  • [NONE] (to clear features on the node)
Description

Sets the features on a node.

These node features will be overwritten when an RM reports features.
Example 
mnodectl -m features=fastio,highmem node1

mnodectl -m features=[NONE] node1

 
GEVENT
Format <EVENT>:<MESSAGE>
Description Creates a generic event on the node to which Moab may respond (see Enabling Generic Events).
Example 
mnodectl -m gevent=powerfail:'power has failed' node1
GMETRIC
Format <ATTR>:<VALUE>
Description Sets the value for a generic metric on the node (see Enabling Generic Metrics).

When a gmetric set in Moab conflicts with what the resource manager reports, Moab uses the set gmetric until the next time the resource manager reports a different number.
Example 
mnodectl -m gmetric=temp:120 node1
MESSAGE
Format '<MESSAGE>'
Description Sets a message to be displayed on the node.
Example 
mnodectl -m message='powerfailure: power has failed' node1
NODEEXP
Format <STRING>
Where <NODEEXP> is a node name, regex or ALL
Description Identifies one or more nodes.
Example node1 — applies only to node1
fr10n* - all nodes starting with fr10n
ALL - all known nodes
OS
Format <STRING>
Description Operating System (see Resource Provisioning).
Example 
mnodectl node1 -m OS=RHELAS30
POWER
Format {off|on}
Description

Set the power state of a node. Action will NOT be taken if the node is already in the specified state.

If you power off a node, a green policy will try to turn it back on. If you want the node to remain powered off, you must associate a reservation with it.

If you request to power off a node that has active work on it, Moab will return a status indicating that the node is busy (with a job or VM) and will not be powered off. You will see one of these messages:

  • Ignoring node <name>: power ON in process (indicates node is currently powering on)
  • Ignoring node <name>: power OFF in process (indicates node is currently powering off)
  • Ignoring node <name>: has active VMs running (indicates the node is currently running active VMs)
  • Ignoring node <name>: has active jobs running (indicates the node is currently running active jobs)

Once you resolve the activity on the node (by preempting or migrating the jobs or VMs, for example), you can attempt to power the node off again.

You can use the --flags=force option to cause a force override. However, doing this will power off the node regardless of whether or not its jobs get migrated or preempted (i.e., you run the risk of losing the VMs/jobs entirely). For example:

> mnodectl node1 -m power=off --flags=force
Example 
> mnodectl node1 -m power=off
STATE
Format {drained|idle}
Description

Remove (drained) or add (idle) a node from scheduling.
Example 
mnodectl node1 -m state=drained

Moab ignores node1 when scheduling.
VARIABLE
Format <name>[=<value>],<name>[=<value>]...
Description Set a list of variables for a node.
Example 
> mnodectl node1 -m variable=IP=10.10.10.100,Location=R1S2

Related topics