12.4 Managing Shared Cluster Resources (Floating Resources) |
|
12.4 Managing Shared Cluster Resources (Floating Resources) |
|
This section describes how to configure, request, and reserve cluster file system space and bandwidth, software licenses, and generic cluster resources.
Shared cluster resources such as file systems, networks, and licenses can be managed through creating a pseudo-node. You can configure a pseudo-node via the NODECFG parameter much as a normal node would be but additional information is required to allow the scheduler to contact and synchronize state with the resource.
In the following example, a license manager is added as a cluster resource by defining the GLOBAL pseudo-node and specifying how the scheduler should query and modify its state.
NODECFG[GLOBAL] RMLIST=NATIVE NODECFG[GLOBAL] QUERYCMD=/usr/local/bin/flquery.sh NODECFG[GLOBAL] MODIFYCMD=/usr/local/bin/flmodify.sh
In some cases, pseudo-node resources may be very comparable to node-locked generic resources however there are a few fundamental differences which determine when one method of describing resources should be used over the other. The following table contrasts the two resource types.
| Attribute | Pseudo-Node | Generic Resource |
|---|---|---|
| Node-Locked | No Resources can be encapsulated as an independent node. |
Yes Must be associated with an existing compute node. |
| Requires exclusive batch system control over resource | No Resources (such as file systems and licenses) may be consumed both inside and outside of batch system workload. |
Yes Resources must only be consumed by batch workload. Use outside of batch control results in loss of resource synchronization. |
| Allows scheduler level allocation of resources | Yes If required, the scheduler can take external administrative action to allocate the resource to the job. |
No The scheduler can only maintain logical allocation information and cannot take any external action to allocate resources to the job. |
Consumable floating resources are configured in the same way as node-locked generic resources with the exception of using the GLOBAL node instead of a particular node.
NODECFG[GLOBAL] GRES=tape:4,matlab:2 ...
In this setup, four resources of type tape and 2 of type matlabare floating and available across all nodes.
Floating resources are requested on a per task basis using native resource manager job submission methods or using the GRES resource manager extensions.
Moab allows both the file space and bandwidth attributes or a cluster file system to be tracked, reserved, and scheduled. With this capability, a job or reservation may request a particular quantity of file space and a required amount of I/O bandwidth to this file system. While file system resources are managed as a cluster generic resource, they are specified using the FS attribute of the NODECFGparameter as in the following example:
NODECFG[GLOBAL] FS=PV1:10000@100,PV2:5000@100 ...
In this example, PV1 defines a 10 GB file system with a maximum throughput of 100 MB/s while PV2 defines a 5 GB file system also possessing a maximum throughput of 100 MB/s.
A job may request cluster file system resources using the fs resource manager extension. For a TORQUE based system, the following could be used:
>qsub -l nodes=1,walltime=1:00:00 -W x=fs:10@50
Jobs may request and reserve software licenses using native methods or using the GRES resource manager extension. If the cluster license manager does not support a query interface, license availability may be specified within Moab using the GRES attribute of the NODECFG parameter.
Example
Configure Moab to support four floating quickcalc and two floating matlab licenses.
NODECFG[GLOBAL] GRES=quickcalc:4,matlab:2 ...
Submit a TORQUE job requesting a node-locked or floating quickcalc license.
> qsub -l nodes=1,software=quickcalc,walltime=72000 testjob.cmd
Copyright © 2012 Adaptive Computing Enterprises, Inc.®