Moab-NUMA Integration Guide

Scheduling a NUMA type system requires some special configuration. Moab uses NODESETs and NODEAVAILABILITYPOLICY to determine when and where jobs can run in a NUMA environment.

This guide assumes Moab is scheduling a single NUMA system. Each node in the system must be configured with the same feature.

To integrate Moab and NUMA, follow these steps:

  1. Configure the NODESETs to use the node features. Note: Currently with a SharedMem setup a node can only reside in one nodeset.
  2. NODESETPOLICY ONEOF
    NODESETATTRIBUTE FEATURE
    NODESETISOPTIONAL FALSE
    NODESETPRIORITYTYPE MINLOSS
    NODESETLIST uv
    
  3. Configure Moab to use the "PRIORITY" NODEALLOCATIONPOLICY.
  4. NODEALLOCATIONPOLICY PRIORITY
  5. Configure Moab so that any job that runs on the NUMA partition requests SharedMem.
  6. RMCFG[base] TYPE=PBS
    PARCFG[base] FLAGS=SharedMem

If Moab is scheduling a single partition the following flag can be used:

PARCFG[ALL] FLAGS=SharedMem

Jobs requesting shared memory should be submitted using the "-l flags=sharedmem" option.

Copyright © 2012 Adaptive Computing Enterprises, Inc.®