Maui Scheduler

Maui Quick Start Guide


Maui is an advanced cluster scheduler used throughout the world to improve control over and the efficiency of cluster and supercomputer resources. This guide will highlight the key steps and considerations involved in installing, building, configuring, and testing Maui on a new system.

The first step is understanding what Maui is: Maui is a scheduler, it makes decisions about where, when, and how to run jobs as specified by a set of configurable policies, priorities, and limits. It makes and enforces its decisions by querying and controlling a resource management system such as OpenPBS, PBSPro, Loadleveler, SGE, etc.

Running Maui requires between 20 and 50 MB of RAM (for clusters up to 10 teraflops) and at least 20 MB of disk space to hold its source, binaries, statistics, and log files. It requires that a resource manager (such as PBS, SGE, Loadleveler, etc) be installed and operational. Since Maui must drive this resource manager, Maui must be run under a userid which has administrative access to the resource manager that will be used. The setup and configuration of Maui should be conducted under this userid.

The steps below outline how you can get started with testing out Maui. Before taking the following steps, the host and home directory which will host Maui should be determined. Most often, the host will same as that machine on which the resource manager is run and the home directory will be a standard local build location such as /usr/local/src.

Step One: Unpack and Build Maui

> cd /usr/local/src
(or directory of your choice)
> gtar -xzvf maui-<VERSION>.tar.gz
(this will create a directory named 'maui-<VERSION>'
> cd maui-<VERSION>
> ./configure
> make
> sudo make install

The 'configure' step will request some info about compilers, resource managers, etc. Upon completing these steps, several executables will be created in the maui-<VERSION>/bin directory. If you want to install these binaries system-wide, run make install to do so. See the Installation Overview for further information.

Step Two: Configure the Resource Manager

The next step is resource manager specific configuration. The PBS, SGE, and LoadLeveler Integration Guides describe what configuration, if any, is required to enable the Maui to resource manager interface.

Step Three: Configure Maui

Most often, the first step in evaluating Maui is to enable it without any special features, running it as a vanilla FIFO backfill scheduler. Maui configuration is handled by editing the flat text config file, maui.cfg, located in the Maui home directory (i.e., maui-<VERSION>). Although the configure script will have already setup the initial configuration, it may be desirable to check the parameters ADMIN1, SERVERHOST, and RMCFG located in this file. If there are any questions, the online parameters documentation should provide the needed information.

Step Four: Test Maui

With Maui configured, the next step is to test Maui. Maui possesses a unique feature, called test mode, which allows it to be safely tested even on a live system. In this mode, Maui can be started and attached to the resource manager with its ability to start, cancel, or otherwise modify jobs disabled.

This allows a site to verify the scheduler-resource manager interface and experiment with Maui commands on a live system in a risk free environment, all with the old scheduler still actively scheduling. To run Maui in test mode, set the SERVERMODE parameter to TEST in the maui.cfg file and start maui.

> vi maui.cfg
---
SERVERMODE TEST
---
> export PATH=$PATH:$(MAUI_HOME_DIR)/bin
> maui

The full suite of Maui commands can now be issued to evaluate Maui's view of your system. Commands such as showq, showbf, diagnose, or showstats may be good to get started with additional commands described in the online commands documentation. If these commands fail, or other anomalous behavior is detected, verify Maui is still running. If it is not, check the last few lines written into the maui.log file located in the maui-<VERSION>/log directory. If Maui shutdown, the log should clarify why. If the log does not fully identify the source of trouble, review the Troubleshooting section in the online admin manual or search the users mailing list archives.

Maui may be tested indefinitely and can be started and stopped repeatedly with no impact on the resource manager or other users. The online Maui Administrator's Guide will describe the more advanced configuration options available within Maui. Virtually all configuration options can be experimented with safely under test mode. The command showconfig -v can be used at any time to display current scheduler settings while the changeparam command can be used to modify these values. As mentioned earlier, the online parameters documentation will examine each configurable option in some detail

Step Five: Run Maui 'Live'

When testing has satisfactorily validated Maui configuration and behavior, Maui can be placed into production operation by setting Maui's SERVERMODE parameter to NORMAL, disabling your current scheduler, and restarting Maui. The PBS, SGE, and LoadLeveler Integration Guides cover how to disable the default scheduler if you are unfamiliar with this process.

The Maui Administrator's Guide will step you through the more advanced topics of truly managing your system and your workload. It contains case studies and detailed information on the issues associated with scheduling and the tools available within Maui for handling these issues. Its organization will allow you to cover only those topics of interest. If problems or questions arise, the Maui users mailing list or Supercluster support should be able to help.