13.5 Issues with Client Commands
13.5-A Client Overview
Moab client commands are implemented as links to the executable mclient. When a Moab client command runs, the client executable determines the name under which it runs and behaves accordingly. At the time Moab was configured, a home directory was specified. The Moab client attempts to open the configuration file, moab.cfg, in the etc/ folder of this home directory on the node where the client command executes. This means that the home directory specified at configure time must be available on all hosts where the Moab client commands are executed. This also means that a moab.cfg file must be available in the etc/ folder of this home directory. When the clients open this file, they will try to load the SCHEDCFG parameter to determine how to contact the Moab server.
The home directory value specified at configure time can be overridden by creating an /etc/moab.cfg file or by setting the MOABHOMEDIR environment variable.
Once the client has determined where the Moab server is located, it creates a message, adds an encrypted checksum, and sends the message to the server. The Moab client and Moab server must use a shared secret key for this to work. When the Moab server receives the client request and verifies the message, it processes the command and returns a reply.
13.5-B Diagnosing Client Problems
The easiest way to determine where client failures are occurring is to use built-in Moab logging. On the client side, use the --loglevel flag. For example:
> showq --loglevel=9
This will display verbose logging information regarding the loading of the configuration file, connecting to the Moab server, sending the request, and receiving a response.
This information almost always reveals the source of the problem. If it does not, the next step is to look at the Moab server side logs; this is done using the following steps:
- Stop Moab scheduling so that the only activity is handling Moab client requests.
> mschedctl -s
- Set the logging level to very verbose.
> mschedctl -m loglevel 7
- Watch Moab activity.
> tail -f log/moab.log | more
Now, in a second window, issue any failing client command, such as showq.
The moab.log file will record the client request and any reasons it was rejected.