4.328 mschedctl

The mschedctl command controls various aspects of scheduling behavior. It is used to manage scheduling activity, shutdown the scheduler, and create resource trace files. It can also evaluate, modify, and create parameters, triggers, and messages.

With many flags, the --msg=<MSG> option can be specified to annotate the action in the event log.

4.328.3 Format

-A - ANNOTATE
Format	<STRING>
Description	Report the specified parameter modification to the event log and annotate it with the specified message. The RECORDEVENTLIST parameter must be set in order for this to work.
Example	mschedctl -A 'increase logging' -m 'LOGLEVEL 6' Adjust the LOGLEVEL parameter and record an associated message.

-c - CREATE
Format	One of: message <STRING> [-o <TYPE>:<VAL>] trigger<TRIGSPEC> -o <OBJECTTYPE>:<OBJECTID> gevent -n <NAME> [-m <message>] where <ATTR> is one of account, duration, ID, messages, profile, reqresources, resources, rsvprofile, starttime, user, or variables
Description	Create a message, trigger, or gevent and attach it to the specified object. To create a trigger on a default object, use the Moab configuration file (moab.cfg) rather than the mschedctl command.
Example	mschedctl -c message tell the admin to be nice Create a message on the system table. mschedctl -c trigger EType=start,AType=exec,Action="/tmp/email $OWNER $TIME" -o rsv:system.1 Create a trigger linked to system.1. Creating triggers on default objects via mschedctl -c trigger does not propagate the triggers to individual objects. To propagate triggers to all objects, the triggers must be created within the moab.cfg file; for example: `NODECFG[DEFAULT]TRIGGER`. mschedctl -c gevent -n diskfailure -m "node=n4" Create a gevent indicating a disk failure on the node labeled n4.

-d - DESTROY
Format	One of: trigger:<TRIGID> message:<INDEX>
Description	Delete a trigger or message.
Example	mschedctl -d trigger:3 Delete trigger 3. mschedctl -d message:5 Delete message with index 5.

-f - FLUSH
Format	{all\|fairshare\|usage}
Description	Reset all internally-stored Moab Scheduler statistics to the initial start-up state as of the time the command was executed. Flushing should only be used if you experience corrupt statistics. The best practice is to pause the Moab scheduler with mschedctl -p before running the flush command. After running the flush command, unpause the Moab scheduler with mschedctl -r and the jobs will start flowing again. For all external observers this will be a transparent flush unless they are watching the stats.
Example	mschedctl -f usage Flush usage statistics.

-k - KILL
Description	Stop scheduling and exit the scheduler
Example	mschedctl -k Kill the scheduler.

-l - LIST
Format	{config\|feature\|gmetric\|gres\|message\|opsys\|trans\|trigger} [-v] [--xml] Using the --xml argument with the trans option returns XML that states if the queried TID is valid or not.
Default	config
Description	List the generic metrics, generic resources, scheduler configuration, system messages, operating systems, triggers, transactions, or node features recognized by Moab. This command does not show credential parameters (such as user, group class, QoS, account).
Example	mschedctl -l config List system parameters. The config command without the -v flag does not show the settings of all scheduling parameters. To show the settings of all scheduling parameters, use the -v flag. This will provide an extended output. This output is often best used in conjunction with the grep command as the output can be voluminous. mschedctl -l feature List all node features recognized by Moab. mschedctl -l gmetric List all configured generic metrics. mschedctl -l gres List all configured generic resources. mschedctl -l message List all system messages. mschedctl -l opsys List all recognized operating systems mschedctl -l trans 1 List transaction id 1. mschedctl -l trigger List triggers.

-L - LOG
Format	[<LOGLEVEL>[: <LOG_FILE>]]
Default	7 $MOABHOMEDIR/log/moab.log
Description	Create a temporary log file with the specified loglevel. If no log file is given, Moab creates a log file in the log directory whose filename extension is the timestamp of when the command was run (for example, "/opt/moab/log/moab.log.20150405081227").
Example	mschedctl -L7:/tmp/moab.log

-m - MODIFY
Format	One of: config [<STRING>] [-e] <STRING> is any string which would be acceptable in moab.cfg If no string is specified, <STRING> is read from STDIN. If -e is specified, the configuration string will be evaluated for correctness but no configuration changes will take place. Any issues with the provided string will be reported to STDERR. mschedctl --flags=persistent -m <config> has been deprecated; use the following method instead: Run mschedctl -m <config> to put the change into effect dynamically. Manually add the setting to the moab.cfg file, so that it always goes into effect after any future Moab restarts/recycles. trigger:<TRIGID> <ATTR>=<VAL> where <ATTR> is one of action, atype, etype, iscomplete, oid, otype, offset, or threshold.
Description	Modify a system parameter or trigger. Moab only loads the following list of parameters when first starting up. Therefore, to change any of these, you must edit the setting in moab.cfg and then restart/recycle with mschedctl -R. JOBMAXNODECOUNT MAXGMETRIC MAXGRES MAXJOB MAXNODE MAXRSVPERNODE STATPROC* STATTIME*
Example	mschedctl -m config LOGLEVEL 9 Change the system loglevel to 9. mschedctl -m trigger:2 AType=exec,Offset=200,OID=system.1 Change aspects of trigger 2.

-p - PAUSE
Description	Disable scheduling but allow the scheduler to update its cluster and workload state information.
Example	mschedctl -p

-q QUERY PENDING ACTIONS
Default	mschedctl -q pactions --xml
Description	A way to view pending actions. Only an XML request is valid. Pending actions can be VMs or system jobs.
Example	mschedctl -q pactions --xml

-R - RECYCLE
Description	Recycle scheduler immediately (shut it down and restart it using the original execution environment and command line arguments). If Moab has been started under systemd, use systemctl restart moab.service instead of using this option.
Example	mschedctl -R Recycle scheduler immediately. To restart Moab with its last known scheduler state, use: mschedctl -R savestate

-r - RESUME
Format	mschedctl -r [[HH:[MM:]]SS]
Default	0
Description	Resume scheduling in the specified amount of time (or immediately if none is specified).
Example	mschedctl -r Resume scheduling immediately.

-s - STOP
Format	<INTEGER>
Default	0
Description	Suspend/stop scheduling at specified iteration (or at the end of the current iteration if none is specified). If the letter I follows <ITERATION>, Moab will not process client requests until this iteration is reached.
Example	mschedctl -s 100I Stop scheduling at iteration 100 and ignore all client requests until then.

-S - STEP
Format	<INTEGER>
Default	0
Description	Step the specified number of iterations (or to the next iteration if none is specified) and suspend scheduling If the letter I follows <ITERATION>, Moab will not process client requests until this iteration is reached.
Example	mschedctl -S Step to the next iteration and stop scheduling.

-W
Description	Preform a manual checkpoint file write.
Example	mschedctl -W

4.328.4 Examples

Example 4-107: Shutting down the Scheduler

mschedctl -k
scheduler will be shutdown immediately