Moab Workload Manager
Home Up

Moab-LSF Integration Guide

Overview

   Moab can be used as an external scheduler for LSF and significantly increase the cluster's capabilities in terms of cluster management, grid integration, accounting, and optimization services.  In this configuration, LSF manages the job queue and the compute resources while Moab queries the LSF daemons via the LSF scheduling API (only available in LSF 5.1 and higher) to obtain up-to-date job and node information.  Using this information, Moab directs LSF to manage jobs in accordance with specified Moab policies, priorities, and reservations.

Installing Moab

   To install Moab for LSF, from the Moab source code root directory, source the LSF environment file '$LSF_ENVDIR/lsf.conf', run 'configure', and then execute 'make install'.

Note: With LSF 5.x and earlier, use $LSF_CONFDIR in place of $LSF_ENVDIR.

Configuring Moab

   The 'configure' command should have already properly configured Moab to use LSF.  However, to verify this, edit the moab.cfg file and confirm that the following line is specified:

moab.cfg

RMCFG[base] TYPE=LSF FLAGS=ignqueuestate

   The resource manager flag ignqueuestate informs Moab to utilize queues which are disabled for usage by the LSF scheduler.

Note: Moab can pull information directly from the LSF API or can obtain it indirectly by parsing the output of LSF commands. See LSF Integration via the Native Interface for more information.

Configuring LSF

  Moab must be run as an LSF admin user.  To enable a user as an LSF admin, edit the ClusterAdmins stanza in the '$LSF_CONFDIR/lsf.cluster.<CLUSTER>' file as in the example below:

conf/lsf.cluster.<CLUSTER>

Begin   ClusterAdmins
Administrators = lsfadmin john steve
End    ClusterAdmins

  Additionally, LSF must be configured to use Moab as its external scheduler.  This can be done by adding the following line to the lsf.conf file and restarting the LSF head-node daemons:

lsf.conf

LSF_ENABLE_EXTSCHEDULER=Y

Disabling LSF Scheduler

   To disable LSF scheduling for all queues, issue the following command:

disable LSF queues

> badmin qinact all

Starting Moab

   To start Moab, source the LSF environment file, 'lsf.conf' into the current environment and then start the Moab daemon:

starting moab

> source /usr/lsf/conf/lsf.conf
> moab
Note > moabd is a safe and recommended method of starting Moab if things are not installed in their default locations.
Note It may also be necessary to add the path to your LSF libraries to the LD_LIBRARY_PATH variable as in 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lsf/6.0/linux2.4-glibc2.3-x86/lib'.

Troubleshooting

  • For LSF to function correctly, the mbatchd daemon must be running.  If Moab reports 'batch system daemon not responding ... still trying', check this process.
Home Up