momctl

(PBS MOM Control)

5.484.1 Synopsis

momctl -c { <JOBID> | all }
momctl -C
momctl -d { <INTEGER> | <JOBID> }
momctl -f <FILE>
momctl -h <HOST>[,<HOST>]...
momctl -l
momctl -p <PORT_NUMBER>
momctl -q <ATTRIBUTE>
momctl -r { <FILE> | LOCAL:<FILE> }
momctl -s
momctl -u

5.484.2 Overview

The momctl command allows remote shutdown, reconfiguration, diagnostics, and querying of the pbs_mom daemon.

5.484.3 Format

-c — Clear
Format	{ <JOBID> \| all }
Default	---
Description	Makes the MOM unaware of the job's existence. It does not clean up any processes associated with the job.
Example	momctl -c 2 -h node1

-C — Cycle
Format	---
Default	---
Description	Cycle pbs_mom (force the MOM to send a status update to pbs_server).
Example	momctl -C -h node1 Cycle pbs_mom on node1.

-d — Diagnose
Format	{ <INTEGER> \| <JOBID> }
Default	0
Description	Diagnose MOM(s). (For more details, see Diagnose detail below.)
Example	momctl -d 2 -h node1 Print level 2 and lower diagnostic information for the MOM on node1.

-f — Host File
Format	<FILE>
Default	---
Description	A file containing only comma or whitespace (space, tab, or new line) delimited hostnames.
Example	momctl -f hosts.txt -d 0 Print diagnose information for the MOMs running on the hosts specified in hosts.txt.

-h — Host List
Format	<HOST>[,<HOST>]...
Default	localhost
Description	A comma separated list of hosts.
Example	momctl -h node1,node2,node3 -d 0 Print diagnose information for the MOMs running on node1, node2, and node3.

-l — Layout
Format	---
Default	---
Description	Display the layout of the machine as it is understood by Torque.
Example	[root@c04a numa]# momctl -l Machine (50329748KB) Socket 0 (33552532KB) Chip 0 (16775316KB) Core 0 (1 threads) Core 1 (1 threads) Core 2 (1 threads) Core 3 (1 threads) Core 4 (1 threads) Core 5 (1 threads) Core 6 (1 threads) Core 7 (1 threads) Chip 1 (16777216KB) Core 8 (1 threads) Core 9 (1 threads) Core 10 (1 threads) Core 11 (1 threads) Core 12 (1 threads) Core 13 (1 threads) Core 14 (1 threads) Core 15 (1 threads) Socket 1 (16777216KB) Chip 2 (8388608KB) Core 16 (1 threads) Core 17 (1 threads) Core 18 (1 threads) Core 19 (1 threads) Core 20 (1 threads) Core 21 (1 threads) Core 22 (1 threads) Core 23 (1 threads) Chip 3 (8388608KB) Core 24 (1 threads) Core 25 (1 threads) Core 26 (1 threads) Core 27 (1 threads) Core 28 (1 threads) Core 29 (1 threads) Core 30 (1 threads) Core 31 (1 threads)

-p — Port
Format	<PORT_NUMBER>
Default	Torque's default port number.
Description	The port number for the specified MOM(s).
Example	momctl -p 5455 -h node1 -d 0 Request diagnose information over port 5455 on node1.

-q — Query
Format	<ATTRIBUTE>
Default	---
Description	Query or set <ATTRIBUTE>, where <ATTRIBUTE> is a property listed by pbsnodes -a (see Query attributes for a list of attributes).
Example	momctl -q physmem Print the amount of physmem on localhost. momctl -h node2 -q loglevel=7 Change the current MOM logging on node2 to level 7.

-r — Reconfigure
Format	{ <FILE> \| LOCAL:<FILE> }
Default	---
Description	Reconfigure MOM(s) with remote or local config file, <FILE>. This does not work if $remote_reconfig is not set to true when the MOM is started.
Example	momctl -r /home/user1/new.config -h node1 Reconfigure MOM on node1 with /home/user1/new.cofig on node1.

-s — Shutdown
Format	---
Default	---
Description	Have the MOM shut itself down gracefully (this includes reporting to server so that pbsnodes marks the node down).
Example	momctl -s Shut down the pbs_mom process on localhost.

-u — Update
Format	---
Default	---
Description	Update the hardware configuration on pbs_server with a layout from the MOM.
Example	momctl -u Update pbs_server hardware configuration.

5.484.4 Query attributes

Attribute	Description
arch	node hardware architecture
availmem	available RAM
loadave	1 minute load average
ncpus	number of CPUs available on the system
netload	total number of bytes transferred over all network interfaces
nsessions	number of sessions active
nusers	number of users active
physmem	configured RAM
sessions	list of active sessions
totmem	configured RAM plus configured swap

5.484.5 Diagnose detail

Level	Description
0	Display the following information: Local hostname Expected server hostname Execution version MOM home directory MOM config file version (if specified) Duration MOM has been executing Duration since last request from pbs_server daemon Duration since last request to pbs_server daemon RM failure messages (if any) Log verbosity level Local job list
1	All information for level 0 plus the following: Interval between updates sent to server Number of initialization messages sent to pbs_server daemon Number of initialization messages received from pbs_server daemon Prolog/epilog alarm time List of trusted clients
2	All information from level 1 plus the following: PID Event alarm status
3	All information from level 2 plus the following: syslog enabled

Example 5-230: MOM diagnostics

momctl -d 1

Host: nsrc/nsrc.fllcl.com    Server: 10.10.10.113    Version: torque_1.1.0p4
HomeDirectory:                  /usr/spool/PBS/mom_priv
ConfigVersion:          147
MOM active:             7390 seconds
Last Msg From Server:   7389 seconds (CLUSTER_ADDRS)
Server Update Interval: 20 seconds
Server Update Interval: 20 seconds
Init Msgs Received:        0 hellos/1 cluster-addrs
Init Msgs Sent:                1 hellos
LOGLEVEL:                            0 (use SIGUSR1/SIGUSR2 to adjust)
Prolog Alarm Time:      300 seconds
Trusted Client List:    12.14.213.113,127.0.0.1
JobList:                              NONE

diagnostics complete

Example 5-231: System shutdown

> momctl -s -f /opt/clusterhostfile

shutdown request successful on node001
shutdown request successful on node002
shutdown request successful on node003
shutdown request successful on node004
shutdown request successful on node005
shutdown request successful on node006